Information processing device, information processing method and program

Abstract

To provide an authentication device for improving authentication accuracy and preventing impersonation by forgery of biological information.SOLUTION: An information processing method detects a gesture by an authentication object on the basis of distance information showing distance up to the authentication object, determines that authentication of the authentication object can be started on the basis of the biological information of the authentication object when the gesture based on the distance information is detected, and performs face authentication on the basis of a face image of an authentication object after determining that the authentication of the authentication object can be started.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] In recent years, authentication devices have been developed that enhance security by authenticating biometric characteristics such as faces, fingerprints, and vein patterns. These devices are used for authentication when unlocking doors or accessing information stored in computers. For example, Patent Document 1 describes a biometric authentication device that detects a motion pattern corresponding to a specific part of a living body by a sensor and performs authentication by comparing the biometric information of the part corresponding to the detected motion pattern with the biometric information generated from the detected motion pattern. Patent Document 2 describes a biometric characteristic input device that does not contact fingerprints and blood vessels. Authentication methods that use biometric characteristics are effective against issues that arise in authentication using passwords, keys, security cards, etc. (such as theft, forgetting, or loss of passwords or keys). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Republished WO2013 / 114806 [Patent Document 2] JP 2009-175810 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, authentication methods using biometric features also have new problems, such as a decrease in authentication accuracy due to ambiguity specific to living bodies and spoofing due to forged biometric information. Other problems include the following. For example, in the case of fingerprint authentication, a finger needs to be touched to a sensor, but there is a demand for authentication in a non-contact state to ensure the cleanliness of the human body and to simplify operation. In addition, when performing authentication using a finger vein pattern, it is often necessary to touch the finger to a stand because the authentication accuracy is problematic due to positional deviation. However, when the finger is touched to the stand, the vein in that part is compressed, the vein pattern changes, and there is also a problem that the authentication accuracy decreases. In addition, in face authentication, it is easy to ensure non-contact, but compared to fingerprints and blood vessels, there are few features that are easy to digitize, so it is generally difficult to ensure authentication accuracy. In addition, for example, when face authentication is used to allow passage through a door, if the authentication device is constantly monitoring the face, the door will open just when a person with a registered face image passes in front of the door, which causes inconvenience when there is a reason for not wanting to open the door. An authentication method that addresses the multiple problems of such biometric authentication has not been provided so far.

[0005] SUMMARY OF THE PRESENT DISCLOSURE An object of the present invention is to provide an information processing device, an information processing method, and a program that can solve the above-mentioned problems. [Means for solving the problem]

[0006] The present invention is an information processing device comprising an acquisition unit that acquires an image of a subject captured by an imaging device, a subject recognition unit that recognizes the subject from the acquired image, a gesture recognition unit that recognizes a gesture of the subject after the subject recognition unit has successfully recognized the subject, and a control unit that controls the operation of a control object linked to the gesture, wherein the control unit outputs a notification to the subject that the subject recognition has been successful.

[0007] The present invention is also an information processing method comprising: an acquisition step of acquiring an image of a subject captured by an imaging device; a subject recognition step of recognizing the subject from the acquired image by an information processing device; a gesture recognition step of recognizing a gesture of the subject by the information processing device after the subject recognition step is successful; and a control step of controlling, by the information processing device, the operation of a control object linked to the gesture, wherein the control step outputs a notification to the subject that the subject recognition has been successful.

[0008] The present invention is also a program comprising an acquisition step of acquiring an image of a subject captured by an imaging device, a subject recognition step of recognizing the subject from the acquired image by an information processing device, a gesture recognition step of recognizing a gesture of the subject by the information processing device after the subject recognition step is successful, and a control step of controlling, by the information processing device, the operation of a control object linked to the gesture, wherein the control step outputs a notification to the subject that the subject recognition has been successful. Effect of the Invention

[0009] According to the authentication of the present invention, it is possible to improve the accuracy of authentication and prevent impersonation through forged biometric information while maintaining the convenience of contactless authentication. [Brief description of the drawings]

[0010] [Figure 1] FIG. 2 is a diagram illustrating a minimum configuration of an authentication device according to the first embodiment of the present invention. [Diagram 2] 1 is a block diagram of a security system according to a first embodiment of the present invention. [Diagram 3] FIG. 2 is a first diagram showing an example of a gesture for instructing the authentication device according to the first embodiment of the present invention to start authentication processing; [Figure 4]1 is a first diagram showing an example of biometric information used for authentication by the authentication device according to the first embodiment of the present invention; [Diagram 5] 13 is a second diagram showing an example of biometric information used for authentication by the authentication device according to the first embodiment of the present invention. FIG. [Figure 6] 11 is a third diagram showing an example of biometric information used for authentication by the authentication device according to the first embodiment of the present invention. FIG. [Figure 7] FIG. 4 is a fourth diagram showing an example of biometric information used for authentication by the authentication device according to the first embodiment of the present invention. [Figure 8] FIG. 5 is a fifth diagram showing an example of biometric information used for authentication by the authentication device according to the first embodiment of the present invention. [Figure 9] FIG. 6 is a sixth diagram showing an example of biometric information used for authentication by the authentication device according to the first embodiment of the present invention. [Figure 10] 3A to 3C are diagrams illustrating an example of a biometric gesture used for authentication by the authentication device according to the first embodiment of the present invention. [Figure 11] FIG. 2 is a first diagram showing an example of a biometric gesture for instructing locking or unlocking a door according to the first embodiment of the present invention; [Figure 12] FIG. 11 is a second diagram showing an example of a biometric gesture for instructing locking or unlocking of a door according to the first embodiment of the present invention. [Figure 13] 13 is an example of an image when detecting that a person to be authenticated is in an emergency according to the first embodiment of the present invention. [Figure 14] 5 is a flowchart showing an example of authentication processing according to the first embodiment of the present invention. [Figure 15] 5 is a flowchart illustrating an example of a notification process according to the first embodiment of the present invention. [Figure 16] FIG. 4 is a block diagram of a security system according to a second embodiment of the present invention. [Figure 17] 13 is a flowchart showing an example of authentication processing according to the second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] <First embodiment> An authentication device according to a first embodiment of the present invention will now be described with reference to Figures 1 to 15. Figure 1 is a diagram showing the minimum configuration of an authentication device according to a first embodiment of the present invention. As shown in FIG. 1, the authentication device 100 includes at least an imaging unit 110 and an authentication unit 130. The imaging unit 110 captures an image of a person to be authenticated (hereinafter, referred to as a subject). The authentication unit 130 performs authentication of the living body based on one or more pieces of biometric information included in the image of the subject captured by the imaging unit 110 and information indicated by a gesture of the living body included in the image captured by the imaging unit 110. The biometric information is, for example, a face image, a vein pattern of each part of the living body, an iris pattern, etc. Also, the information indicated by a gesture of the living body is, for example, a number or a letter indicated by the subject with a finger.

[0012] FIG. 2 is a block diagram of a security system according to a first embodiment of the present invention. The crime prevention system 1 in this figure includes an authentication device 100, a reporting device 200, a storage device 210, a door 300, and an electric lock 310. The crime prevention system 1 is a system that authenticates whether or not a target person 10 who is about to pass through the door 300 has the authority to pass through the door 300, and permits the target person 10 to pass through the door 300 if the target person 10 has the authority. The crime prevention system 1 also includes a reporting device 200 that reports to a security guard or the like when a third party tries to pass through the door 300 at the same time as the target person 10 passes through the door 300. The storage device 210 also stores biometric information and the like used to authenticate the target person 10.

[0013] The authentication device 100 captures an image of the subject 10, performs authentication processing based on the captured image, and determines whether the subject 10 has the authority to pass through the door 300. If the subject 10 has the authority to pass through the door 300, the authentication device 100 outputs an unlocking signal to the electric lock 310 to control the door 300 to unlock it. If the authentication device 100 detects that the subject 10 has been attacked, for example, it operates the notification device 200 to notify the subject. The authentication device 100 includes a near-infrared camera 111, a near-infrared light source 112, a near-infrared light source 113, a pattern filter 114, a light source switch 115, a control unit 120, an authentication unit 130, and an alarm lamp 140.

[0014] The near-infrared camera 111 is a camera that captures an image of a subject using a near-infrared light source 112 or a near-infrared light source 113 as a light source. The near-infrared light source 112 is a light source of uniform near-infrared light for obtaining an image for face authentication, vein pattern authentication, and gesture recognition. When the near-infrared light source 112 is used as the light source, the near-infrared camera 111 captures a near-infrared image (called an image for authentication) of the subject 10. Meanwhile, a pattern filter 114 is provided in front of the near-infrared light source 113. The pattern filter 114 is a filter that provides special pattern light to parts such as the face and hands with light from the near-infrared light source 113 as a light source. The pattern filter 114 is, for example, a random dot pattern or striped pattern filter. By irradiating each part of the body with the special pattern light that has passed through the pattern filter 114 and capturing an image, an image including information on the distance between those parts and the near-infrared camera 111 can be obtained. That is, when the near-infrared light source 113 is used as the light source, the near-infrared camera 111 captures an image (called a distance image) including information indicating the distance from each part of the subject 10. The distance image can be used to improve the accuracy of face and vein pattern authentication and to strengthen gesture recognition. The light source switch 115 switches the light source of the near-infrared camera 111 to the near-infrared light source 112 or the near-infrared light source 113. The switching of the light source is controlled by the control unit 120. The imaging unit 110 in FIG. 1 is configured to include the near-infrared camera 111, the near-infrared light source 112, the near-infrared light source 113, the pattern filter 114, and the light source switch 115.

[0015] The control unit 120 controls the unlocking and locking of the electric lock 310 based on the authentication result by the authentication unit 130. The control unit 120 also operates the notification device 200 based on the recognition result of the gesture of the target person 10 by the authentication unit 130. The control unit 120 also instructs the near-infrared camera 111 to capture an image while switching between the near-infrared light source 112 and the near-infrared light source 113 via the light source switch 115. The control unit 120 also outputs the image captured by the near-infrared camera 111 to the authentication unit 130 and instructs the authentication unit 130 to perform authentication processing. The control unit 120 also turns on the alarm lamp 140 based on the detection of a third party by the authentication unit 130.

[0016] The authentication unit 130 acquires an authentication image of the subject 10 captured by the near-infrared camera 111, and performs a predetermined image processing and image recognition processing (hereinafter collectively referred to as image processing). For example, in the case of face authentication, the authentication unit 130 performs processing to extract the area of ​​the face and the feature points of the face from the authentication image in which the face of the subject 10 is captured. Alternatively, the authentication unit 130 recognizes the distance between the subject 10 and the near-infrared camera 111 based on the distance image of the subject 10. Then, the authentication unit 130 performs processing such as correcting the size of the face image included in the authentication image to a size that can be matched with the face image of an authorized person registered in advance in the storage device 210. Since these image processing can be performed using known techniques, the specific contents of the image processing will not be mentioned in this specification. After performing these image processing, the authentication unit 130 matches the face image of the subject 10 after the image processing with the face image registered in the storage device 210, and authenticates the subject 10. The authentication unit 130 may perform authentication using a plurality of pieces of biometric information (such as face, vein pattern, and iris pattern) among other biometric information obtained from the authentication image. The authentication unit 130 also recognizes the gesture of the subject 10 included in the authentication image by image processing. The gesture of the subject 10 is, for example, a gesture in which the subject 10 shakes his / her hand back and forth or bends and straightens his / her fingers toward the near-infrared camera 111. The authentication unit 130 may perform authentication using information indicated by the gesture of the subject 10 in addition to the authentication using the biometric information. The authentication unit 130 may also determine whether to lock or unlock the electric lock 310 based on the information indicated by the gesture of the subject 10. Specific examples of the authentication process by the authentication unit 130 and the control of the electric lock 310 will be described later.

[0017] FIG. 3 is a first diagram showing an example of a gesture for instructing the authentication device according to the first embodiment of the present invention to start authentication processing. 3 shows a state in which the subject 10 holds his / her hand up near his / her face toward the near-infrared camera 111. The authentication unit 130 acquires an authentication image exemplified in FIG. 3, and can recognize, for example, a gesture in which the subject 10 holds one of his / her hands up near his / her face by image processing. It is predetermined that when this gesture is recognized, an instruction to start authentication processing is issued, and the authentication unit 130 decides to start authentication processing for the subject 10. When it is decided to start authentication, the authentication unit 130 may perform, for example, face authentication processing using the authentication image of FIG. 3.

[0018] 4 is a first diagram showing an example of biometric information used for authentication by the authentication device according to the first embodiment of the present invention. FIG 4 shows an example of a throat vein pattern of a subject 10.

[0019] 5 is a second diagram showing an example of biometric information used for authentication by the authentication device according to the first embodiment of the present invention. FIG 5 shows an example of the cheek vein pattern of the subject 10.

[0020] 6 is a third diagram showing an example of biometric information used for authentication by the authentication device according to the first embodiment of the present invention. FIG 6 shows an example of the vein pattern of the palm of the subject 10.

[0021] 7 is a fourth diagram showing an example of biometric information used for authentication by the authentication device according to the first embodiment of the present invention. FIG 7 shows an example of the vein pattern on the back of the hand of the subject 10.

[0022] 8 is a fifth diagram showing an example of biometric information used for authentication by the authentication device according to the first embodiment of the present invention. Fig. 8 shows an example of the vein pattern of the arm of the subject person 10.

[0023] Fig. 9 is a sixth diagram showing an example of biometric information used for authentication by the authentication device according to the first embodiment of the present invention. Fig. 9 shows an example of the iris pattern of a subject 10. As shown in the example of Fig. 9, the resolution of recent cameras has become extremely high and it is possible to extract an iris pattern.

[0024] The near-infrared camera 111 using the near-infrared light source 112 as the light source can capture an image including the vein pattern and iris pattern exemplified in FIG. 4 to FIG. 9. The near-infrared camera 111 captures an authentication image including at least one of the above biometric information together with a facial image of the subject 10. As described above, the authentication unit 130 has a function of authenticating a face from the authentication image. The authentication unit 130 may also have a function of extracting a vein pattern from the authentication image captured by the near-infrared camera 111, comparing it with the vein pattern of an authorized person registered in advance in the storage device 210, and performing authentication processing of the subject 10. The authentication unit 130 may also have a function of extracting an iris pattern from the authentication image captured by the near-infrared camera 111, comparing it with the iris pattern of an authorized person registered in advance in the storage device 210, and performing authentication processing of the subject 10. The authentication unit 130 can perform authentication using multiple pieces of biometric information of the subject 10, including the face, throat vein pattern, cheek vein pattern, finger vein pattern, palm vein pattern, back of hand vein pattern, arm vein pattern, and iris pattern. For example, the authentication process may be performed using the face, any one vein pattern, and iris pattern. Alternatively, the authentication process may be performed using vein patterns of multiple parts. This can improve authentication accuracy compared to when authentication is performed using only the face. In addition, when authenticating, the subject 10 only needs to point each part of the body toward the near-infrared camera 111 so that the face, vein patterns of each part, and iris pattern can be captured, and the subject 10 can keep his or her body clean since the subject 10 does not come into contact with sensors during authentication. In addition, since multiple types of biometric information are used, it is more difficult to forge biometric information than a single piece of biometric information, such as the face or veins. Therefore, it is effective in preventing impersonation by a third party. Furthermore, as illustrated in FIG. 9, when an authentication image is captured using near-infrared light as a light source, authentication using the iris pattern is also possible. By adding the authentication result of the iris pattern, the authentication accuracy can be further improved.

[0025] FIG. 10 is a diagram showing an example of a biometric gesture used for authentication by the authentication device according to the first embodiment of the present invention. The near-infrared camera 111 using the near-infrared light source 112 as a light source can capture not only biometric information but also the gestures of the fingers of the subject 10. FIG. 10(a) is an example of a gesture in which the subject 10 expresses the number "2" with the number of fingers. FIG. 10(b) is an example of a gesture in which the subject 10 expresses the number "3" with the number of fingers. FIG. 10(c) is an example of a gesture in which the subject 10 expresses the number "4" with the number of fingers. When the subject 10 expresses numbers with finger gestures as in these examples, the authentication unit 130 can recognize the numbers as a secret number in the authentication process. For example, when the subject 10 makes the finger gestures shown in FIG. 10(a) to FIG. 10(c) in this order, the authentication unit 130 recognizes the secret number "234" from a plurality of authentication images captured of the state by using a known image processing technique. Furthermore, the authentication unit 130 performs authentication processing of the subject 10 by comparing the PIN "234" recognized from the authentication image captured by the near-infrared camera 111 with the PIN registered in advance in the storage device 210. Even in authentication using the PIN indicated by this finger gesture, the subject 10 only needs to make the finger gesture toward the near-infrared camera 111, and can maintain non-contact with the authentication device and keep the body clean. Furthermore, by combining this with authentication processing using biometric information such as face authentication, it is expected that the authentication accuracy can be improved and spoofing by a third party can be prevented.

[0026] As shown in Fig. 10(a) to Fig. 10(c), the near-infrared camera 111 can capture an authentication image including the vein pattern of the finger or palm of the subject 10 at the same time as the subject 10 makes a finger gesture. The authentication unit 130 can use the authentication image to perform authentication processing that combines authentication based on the vein pattern and authentication based on a personal identification number. In this case, the subject 10 can perform authentication based on biometric information (vein pattern) and authentication based on a personal identification number simply by making a finger gesture toward the near-infrared camera 111. Therefore, the subject 10 can improve security by performing two-factor authentication that combines biometric information and information indicated by a biometric gesture (or multi-factor authentication that includes other biometric information) without feeling bothered.

[0027] The gestures made by the subject 10 for authentication are not limited to the example in Fig. 10. The information indicated by the gestures of the subject 10 may be letters such as alphabets instead of numbers. Also, the subject 10 may be authenticated by forming his / her fingers into a certain shape.

[0028] As described above, in this embodiment, the authentication accuracy can be improved and security can be enhanced by performing an authentication process using multiple types of biometric information, or an authentication process that combines authentication based on one or multiple types of biometric information and authentication based on information indicated by a biometric gesture. If this authentication process is successful, the control unit 120 may control the electronic lock 310 to unlock and allow the subject 10 to pass through the door 300, but the authentication unit 130 may also be configured to recognize an instruction gesture made by the subject 10 to the door 300, and the control unit 120 may control the electronic lock 310 based on the instruction gesture.

[0029] FIG. 11 is a first diagram showing an example of a biometric gesture for instructing to lock and unlock a door according to the first embodiment of the present invention. The near-infrared camera 111 using the near-infrared light source 112 as a light source can capture the facial expression of the subject 10. FIG. 11 is an image capturing the facial expression of the subject 10 with one eye closed. The authentication unit 130 can recognize that the subject 10 has closed his / her right eye from the image shown in FIG. 11 by using a known image processing technique. By predetermining unlocking / locking depending on which eye is closed, it becomes possible to perform operations such as unlocking the electric lock 310 when the right eye is closed and locking the electric lock 310 when the left eye is closed. In this way, the subject 10 can indicate the timing of unlocking / locking the door 300 by facial expression.

[0030] FIG. 12 is a second diagram showing an example of a biometric gesture for instructing to lock and unlock the door according to the first embodiment of the present invention. 12(a) to 12(c) are diagrams showing gestures for instructing to lock and unlock the door 300. FIG. 12(a) is a gesture that means the start of an instruction gesture. When the subject 10 moves his / her hand laterally from the hand position in FIG. 12(a) to the position shown in FIG. 12(b), the authentication unit 130 recognizes the gesture from a series of authentication images and distance images. When the authentication unit 130 recognizes a gesture of the subject 10 moving his / her hand laterally, the authentication unit 130 determines, for example, that the gesture is a signal to unlock, and instructs the control unit 120 to unlock the electric lock 310. Also, when the subject 10 moves his / her hand from the hand position in FIG. 12(b) in a direction approaching the hand position shown in FIG. 12(c), the authentication unit 130 recognizes the gesture and determines, for example, that the gesture is a signal to lock. While the subject 10 is making a gesture, the control unit 120 controls the light source switch 115 to switch between the near-infrared light source 112 and the near-infrared light source 113, and the near-infrared camera 111 captures authentication images and distance images of the series of gestures. In the case of a gesture of moving the hand closer and farther away as in this example, the authentication unit 130 can accurately recognize the gesture of the subject 10 from the distance images captured by the near-infrared camera 111 using the near-infrared light source 113 as a light source.

[0031] 11 and 12, in this embodiment, the electric lock 310 is controlled based on the instruction gesture by the subject 10 for the operation of the door 300, so that the subject 10 can decide the timing to lock and unlock the door 300. The security system 1 of this embodiment can also detect the presence of a third party sneaking up behind the subject 10 and warn him / her, or report an emergency such as when the subject is attacked by a third party.

[0032] FIG. 13 shows an example of an image when detecting that a person to be authenticated is in an emergency according to the first embodiment of the present invention. FIG. 13 is a diagram showing a state in which the subject 10 has a surprised expression. The authentication unit 130 can recognize that the subject 10 has a surprised expression by detecting, for example, that the subject 10 opens his / her eyes or mouth using a known face recognition technique. When the subject 10 has a surprised expression or a frightened expression, the authentication unit 130 may determine that the subject is in an emergency situation, and control the control unit 120 to cause the reporting device 200 to report an emergency alert. In addition, when the authentication unit 130 detects that the subject's mouth is covered by a third party, for example, it may control the reporting device 200 to report an emergency alert.

[0033] Although not shown, if a third party is hiding behind the subject, the authentication unit 130 can detect a face or hand vein pattern that is not registered in the storage device 210 and detect the presence of a third party behind the subject. In this case, the control unit 120 can, for example, turn on an alarm lamp 140 installed beside the camera to notify the subject 10 of the presence of a suspicious person. It is also possible to register the suspicious person's biometric information in the storage device 210 in advance, and have the authentication unit 130 compare the biometric information included in the authentication image with the biometric information of the criminal to detect the presence of the criminal. In addition, when a suspicious third party is present or when the third party threatens and forces the user to open the lock, it is possible to use a pre-registered emergency gesture, such as intentionally covering the face with one's hand, to lock the electric lock 310 in a manner that the third party cannot detect, or to have the reporting device 200 report the incident.

[0034] In addition, information reported in an emergency through the reporting device 200 and images of the emergency situation captured by the near-infrared camera 111 can be recorded in the storage device 210 for use in crime prevention.

[0035] Next, the flow of processing of the authentication process, the locking / unlocking control of the electric lock 310, and the notification control when a third party is detected in this embodiment will be described. FIG. 14 is a flowchart showing an example of the authentication process in the first embodiment of the present invention. It is assumed that the memory device 210 pre-stores a gesture that is a condition for starting the authentication process of a person appearing in the authentication image (for example, a gesture in which the person holds one hand next to the face), biometric information of the subject 10 used for authentication, a PIN number used for authentication, and instruction gestures for the operation of the subject 10's electric lock 310 and the reporting device 200. First, the subject 10 stands in front of the near-infrared camera 111 and holds one hand up next to his / her face. The near-infrared camera 111 captures an image of the subject 10 based on an image capturing instruction from the control unit 120. At this time, the control unit 120 switches the light source of the near-infrared camera 111 between the near-infrared light source 112 and the near-infrared light source 113 by the light source switch 115. The near-infrared camera 111 captures an authentication image and a distance image of the subject 10 (step S100). The control unit 120 acquires the authentication image and the distance image captured by the near-infrared camera 111 and outputs them to the authentication unit 130. Note that in the subsequent processing, the near-infrared camera 111 continues to capture the authentication image and the distance image at, for example, a predetermined time interval and output them to the control unit 120.

[0036] Next, the authentication unit 130 executes image processing of the authentication image and the distance image based on an instruction from the control unit 120 (step S110). Next, the authentication unit 130 detects an authentication start gesture of the subject 10 by image processing of the authentication image (step S120). For example, the authentication unit 130 uses the acquired authentication image and distance image to recognize that the subject 10 is holding one hand up next to his / her face. The authentication unit 130 determines that authentication of the subject 10 is to be started based on this gesture.

[0037] Next, the authentication unit 130 executes the authentication process (step S130). At this time, the authentication unit 130 performs the authentication process by combining at least one authentication by biometric information and authentication by gesture, such as face authentication and input of a PIN by finger gesture. For example, the authentication unit 130 compares the face image of the subject 10 shown in the authentication image with a plurality of face images registered in the storage device 210, and determines whether or not the face image of the subject 10 is present among the face images of persons with authority registered in the storage device 210. Also, for example, the subject 10 holds one hand up next to his / her face, and then inputs a PIN by finger gesture as illustrated in FIG. 10. The authentication unit 130 recognizes the PIN input by the subject 10 by performing image processing on the authentication image. The authentication unit 130 determines whether or not the recognized PIN matches the PIN of the person who was able to be matched by the face image. If the authentication by the face image and the authentication by the PIN are successful, the authentication unit 130 determines that the authentication is successful. If not, the authentication unit 130 determines that the authentication has failed and outputs the authentication result to the control unit 120.

[0038] The authentication method of the subject 10 is not limited to face authentication and a PIN number based on gestures. For example, in addition to these, authentication may be performed based on the vein pattern of the finger or palm of the subject 10 included in an authentication image captured when the subject 10 holds up one hand or inputs a PIN number. Increasing the types of biometric information can improve authentication accuracy and security. In addition, when using multiple pieces of biometric information, even if authentication based on one of them fails, authentication may be determined to be successful if authentication based on the other biometric information is successful. For example, when authentication is performed based on the face, finger vein pattern, palm vein pattern, and PIN number, authentication is determined to be successful if authentication based on the face, palm vein pattern, and PIN number is successful even if authentication based on the finger vein pattern fails. For example, when a PIN number is input by finger gestures and authentication is performed based on the finger vein pattern of the image captured at that time, the finger vein pattern may not be properly captured depending on the angle at which the finger is captured and the degree to which the finger is bent. Even in such a case, if the authentication is determined to be successful based on other biometric information and the password, the subject 10 can pass through the door 300 without having to repeat the authentication, ensuring convenience for the subject 10. Furthermore, if authentication is performed based on multiple types of biometric information as in the above example, authentication can be performed based on the vein patterns of the face and palm even in circumstances such as when the subject 10 is injured and has a bandage on his / her finger. Even in such a case, authentication is performed based on one or more pieces of biometric information and a password, so sufficient security can be maintained. Furthermore, in this authentication process, the subject 10 does not touch his / her fingers or the like to sensors, so the subject 10 can be authenticated easily while maintaining a clean appearance. It is also possible to use an iris pattern as biometric information. Also, instead of inputting a PIN number by gesture, authentication processing can be performed by a certain gesture of the subject 10 (for example, a shape of the subject 10's hand or fingers, or a facial expression of the subject 10 such as blinking a predetermined number of times). Furthermore, the authentication unit 130 may perform authentication using a plurality of pieces of biometric information, such as face authentication, throat or cheek vein pattern authentication, and iris pattern authentication, instead of performing authentication using gestures.

[0039] Next, the control unit 120 controls the unlocking and locking of the electric lock 310 based on the authentication result of the authentication unit 130. If the authentication fails (step S140; No), the control unit 120 notifies the authentication failure by, for example, blinking the alarm lamp 140 (step S150). Then, this process flow ends. Alternatively, the authentication process may be retried a predetermined number of times. If the authentication succeeds (step S140; Yes), the authentication unit 130 judges whether or not a third party is present near the subject 10 (step S160). Here, a third party refers to a person who does not have the authority to pass through the door 300. The authentication unit 130 acquires the authentication image and the distance image captured by the near-infrared camera 111 via the control unit 120. The authentication unit 130 performs image authentication on the authentication image and judges whether or not a person other than the subject 10 is included in the authentication image. For example, the authentication unit 130 cuts out the vein patterns and iris patterns of each part of the face, hands, etc. of a person other than the subject 10 from the authentication image. If an image including such biometric information can be cut out, the authentication unit 130 compares it with an image of the corresponding biometric information registered in the storage device 210. At this time, the authentication unit 130 appropriately corrects the size of the face and vein pattern using the distance image. If the comparison is successful, the authentication unit 130 determines that the person shown in the authentication image is a person who has the authority to pass through the door 300. If the comparison is not successful, the authentication unit 130 determines that the person in the authentication image is a third party who does not have the authority to pass through the door 300.

[0040] When it is determined that a third party is present (step S160; Yes), the authentication unit 130 instructs the control unit 120 to notify the target person 10 of a warning to alert the target person 10 to be on guard. The control unit 120 turns on the alarm lamp 140 to notify the warning (step S190). For example, the target person 10 judges whether a person existing near the target person 10 is a suspicious person by looking at the lighting display of the alarm lamp 140. For example, in the case of a person who is intentionally allowed to pass through the door 300, such as when inviting a customer from outside, the third party may pass through the door 300 together with the target person 10. In this case, the target person 10 makes an instruction gesture to unlock the electric lock 310. However, if an unintentional suspicious person is present near the target person 10, it is not appropriate to allow the suspicious person to pass through. In that case, the target person 10 makes an instruction gesture to lock the electric lock 310, and the authentication unit 130 detects the instruction gesture (step S200). For example, the target person 10 closes one eye. Then, the near-infrared camera 111 captures the state, and the authentication unit 130 recognizes the instruction gesture of the subject 10 from the authentication image captured by the near-infrared camera 111. The authentication unit 130 compares the recognized instruction gesture with a predetermined instruction gesture for instructing unlocking or locking the electric lock 310. The authentication unit 130 outputs an instruction signal for unlocking or locking indicated by the detected instruction gesture to the control unit 120. Next, the control unit 120 unlocks or locks the electric lock 310 based on the acquired instruction signal. The subject 10 may also perform an instruction gesture for instructing the reporting device 200 to make an emergency call. When the authentication unit 130 recognizes the instruction gesture, the control unit 120 controls the reporting device 200 to make an emergency call.

[0041] When it is determined that no third party is present (step S160; No), the control unit 120 notifies the subject 10 that the authentication has been successful, for example by flashing the alarm lamp 140. The subject 10, upon seeing this display, makes an instruction gesture to unlock the electric lock 310, and the authentication unit 130 detects the instruction gesture (step S170). The authentication unit 130 outputs information indicating the detected instruction gesture to the control unit 120. The control unit 120 unlocks the electric lock 310 based on the acquired information (step S180). This allows the subject 10 to pass through the door 300.

[0042] FIG. 15 is a flowchart showing an example of the notification process in the first embodiment of the present invention. A process flow when an emergency occurs to the subject 10 will be described with reference to FIG. 15. For example, in FIG. 14, a case where an emergency is determined to occur to the subject 10 instead of step S160 will be described as an example. Steps S100 to 140 are the same as those in FIG. 14, and therefore will not be described. When the authentication is successful (step S140; Yes), the authentication unit 130 determines whether an emergency occurs to the subject 10 (step S163). For example, when the authentication unit 130 recognizes that the subject 10 has an expression of surprise or fear by performing image processing on the authentication image, the authentication unit 130 determines that an emergency occurs to the subject 10. Alternatively, when the authentication unit 130 recognizes a situation in which the subject 10 is attacked from behind or has his mouth covered, the authentication unit 130 determines that an emergency occurs to the subject 10. In that case (step S163; Yes), the authentication unit 130 notifies the control unit 120 that a report should be made to the subject 10. The control unit 120 causes the reporting device 200 to make an emergency report (step S213). Alternatively, the reporting device 200 may emit an alarm sound to scare off suspicious individuals. This allows the target person 10 to be notified of the situation promptly to the outside even if the target person 10 is attacked. At this time, the control unit 120 may not only cause the reporting device 200 to make an emergency report, but also lock the electric lock 310. The control unit 120 may also record in the storage device 210 an authentication image when the target person 10 is attacked, information on the history of making an emergency report, and the like.

[0043] When it is determined that no emergency has occurred to the subject 10 (step S163; No), the process is the same as that of step S170 and thereafter in Fig. 14. Note that this process can be performed in parallel with step S160 in Fig. 14 to perform these processes.

[0044] Generally, biometric authentication has a risk of spoofing due to forged biometric information. For example, in face authentication, there is a possibility that a person may pose as another person using an elaborate mask made of silicon rubber or the like and gain malicious access. In addition, forgery methods using clay or resin are known for fingerprint authentication, and examples of forgery using latent fingerprints left on cups and the like have been reported. In addition, there is a possibility that vein pattern authentication may be captured by a near-infrared camera and copied to spoof. However, in this embodiment, in addition to authentication using biometric information, authentication using a PIN number entered by finger gestures is also performed, so that even if biometric information is forged, security will not be breached by itself. In addition, in this embodiment, multiple pieces of biometric information can be obtained from one image captured by one near-infrared camera 111. For example, when a face is captured, in addition to the face, the vein pattern of the throat or cheek and the iris pattern may be obtained. Alternatively, when an authentication start gesture (raising one hand close to the face) is performed, the face, the vein pattern of the fingers or palm, and the iris pattern may be obtained at the same time. In this embodiment, this feature is utilized to efficiently perform authentication using multiple independent pieces of biometric information. By performing authentication using multiple pieces of biometric information, the possibility of spoofing by forging biometric information can be further reduced.

[0045] In addition, in the case of authentication using only one piece of biometric information, there are cases where the biometric authentication cannot be performed due to injury or other reasons. In general, in many cases, another authentication means is prepared in preparation for such cases. For example, if a PIN number is input using a numeric keypad as an alternative method, this authentication means loses the non-contact nature and ease of operation, and requires a numeric keypad operation panel for inputting the PIN number. In that case, restrictions are added, such as the need for a place to install the operation panel. According to this embodiment, since authentication using multiple pieces of biometric information and authentication using gestures can be used, even if one piece of biometric information cannot be used, if such a situation is assumed in advance and authentication is performed using other biometric information or gestures that can be used for authentication (that can be extracted from an authentication image), there is no need to prepare an additional device.

[0046] Furthermore, conventional authentication devices have no function for dealing with situations in which a third party sneaks up behind the user at the moment the user unlocks the door, or the user is forced to unlock the door under duress. According to this embodiment, the above-mentioned emergency situations can be recognized by image processing, and a warning can be displayed or an alarm can be issued.

[0047] In this way, according to this embodiment, it is possible to improve the accuracy of biometric authentication while maintaining the advantage of contactlessness, such as ensuring the cleanliness of face authentication, by utilizing other biometric authentication features of the face, and to make it difficult to impersonate someone by forging biometric features. Furthermore, the user can control the timing of unlocking and locking the door instead of using a button while maintaining contactlessness by recognizing gestures. Furthermore, it is possible to realize the provision of a crime prevention function by detecting the presence of a third party, warning the user even when the user is unaware, and, if necessary, making an emergency call without the third party being aware of it, or sounding an alarm to repel the third party. This makes it possible to protect the subject 10 from crime. Furthermore, in this embodiment, authentication using multiple types of biometric information and gestures is possible with just a single camera and authentication unit 130.

[0048] Second Embodiment An example of a security system according to the second embodiment of the present invention will now be described with reference to FIGS. In the second embodiment, in addition to the functions of the first embodiment, the second embodiment has a function of detecting a fingerprint of the subject person 10 and a function of performing fingerprint authentication. FIG. 16 is a block diagram of a security system according to the first embodiment of the present invention. In addition, among the configurations of the second embodiment of the present invention, the functional units and various devices that are the same as those that constitute the security system of the first embodiment of the present invention are given the same symbols, and their respective descriptions are omitted.

[0049] The security system 2 in this figure is configured to include an authentication device 100, a notification device 200, a storage device 210, a door 300, and an electric lock 310. The authentication device 100 has a near-infrared camera 111, a near-infrared light source 112, a near-infrared light source 113, a pattern filter 114, a light source switch 115, a control unit 120, an authentication unit 130, an alarm lamp 140, a fingerprint scanner 150, and an emergency switch 160. The fingerprint scanner 150 detects the fingerprint of the subject person 10, and outputs the detected fingerprint information to the control unit 120. The authentication unit 130 compares the fingerprint information with fingerprint information registered in the storage device 210 to perform fingerprint authentication. The emergency switch 160 is a hidden switch provided for emergency situations. The emergency switch 160 is provided in an inconspicuous place, and when the subject 10 presses the emergency switch 160 in an emergency, the notification device 200 operates to notify an emergency call.

[0050] FIG. 17 is a flowchart showing an example of authentication processing in the second embodiment of the present invention. Next, the flow of the authentication process, the unlocking / locking control of the electric lock 310, and the notification process when a third party is detected in the second embodiment will be described with reference to Fig. 17. The same processes as in the first embodiment will be briefly described. First, the target person 10 stands in front of the near-infrared camera 111 and makes an authentication start gesture. The near-infrared camera 111 captures an authentication image and a distance image in which the target person 10 appears (step S100). Next, the authentication unit 130 executes image processing of the authentication image and the distance image (step S110). Next, the authentication unit 130 detects an authentication start gesture of the subject 10 by image processing of the authentication image (step S120). The authentication unit 130 determines that authentication of the subject 10 should be started based on this gesture. The subject 10 then inputs a PIN number using a finger gesture, and aims an image of the vein pattern of the finger, palm, arm, etc. at the near-infrared camera 111. The biometric information required for this authentication is determined in advance, and the near-infrared camera 111 captures the authentication image including the biometric information required for authentication processing.

[0051] Next, the authentication unit 130 judges whether or not to detect a fingerprint (step S123). For example, the authentication unit 130 extracts biometric information required for authentication from the authentication image. If all biometric information required for authentication can be extracted, the authentication unit 130 judges that fingerprint detection is unnecessary. If necessary biometric information cannot be extracted due to the influence of an injury, the authentication unit 130 judges that fingerprint detection is necessary. If it is judged that fingerprint detection is unnecessary (step 123; No), the process proceeds to step S131 described later. If it is judged that fingerprint detection is necessary (step 123; Yes), the authentication unit 130 outputs a signal indicating that fingerprint authentication is necessary to the control unit 120. For example, the control unit 120 displays a message on a display unit of the fingerprint scanner 150 (for example, turns on a lamp) to prompt the subject 10 to perform fingerprint authentication. The subject 10 places a finger on the fingerprint scanner 150, and the fingerprint scanner 150 acquires fingerprint information of the subject 10 (step S125). The fingerprint scanner 150 outputs the acquired fingerprint information to the control unit 120. Next, the authentication unit 130 executes authentication processing (step S131). The authentication unit 130 acquires an authentication image and a target image captured by the near-infrared camera 111 from the control unit 120. The authentication unit 130 compares an image including biometric information such as a cut-out face, a vein pattern of each part, and an iris pattern with an image including corresponding biometric information registered in the storage device 210. Furthermore, when fingerprint information is acquired in step S125, the authentication unit 130 acquires fingerprint information from the control unit 120 and compares it with the fingerprint information registered in the storage device 210. Based on these comparison results, the authentication unit 130 authenticates whether the target person 10 has the authority to pass through the door 300. For example, the authentication unit 130 determines that authentication is successful when all of the comparisons based on one or more pieces of biometric information, the comparison based on a fingerprint, and the comparison based on a PIN number input by a gesture are successful. Alternatively, the authentication unit 130 may determine that authentication is successful when one or more of the biometric information and fingerprint matching is successful and the PIN matching is successful. Alternatively, the authentication unit 130 may determine that authentication is successful when either one or more of the biometric information and fingerprint matching is successful and the PIN matching is successful.In this manner, in this embodiment, adding fingerprint authentication as an option enables flexible authentication in response to changing circumstances (e.g., circumstances in which biometric information cannot be collected), thereby enhancing security.

[0052] Next, the control unit 120 controls the unlocking and locking of the electric lock 310 based on the authentication result of the authentication unit 130. If the authentication fails (step S140; No), the control unit 120 notifies the user of the authentication failure, for example by flashing the alarm lamp 140 (step S150). Then, this processing flow ends. If the authentication is successful (step S140; Yes), the authentication unit 130 determines whether a third party is present near the subject person 10 (step S160). The authentication unit 130 processes the authentication image and determines whether a person other than the subject person 10 is included in the authentication image.

[0053] When it is determined that a third party is present (step S160; Yes), the authentication unit 130 instructs the control unit 120 to notify the target person 10 of a warning. The control unit 120 turns on the alarm lamp 140 and displays a warning (step S190). In response to this warning display, the target person 10 uses the fingerprint scanner 150 to instruct the electric lock 310 and the notification device 200 to perform operations. For example, the target person 10 registers in advance in the storage device 210 fingerprint information of a finger (e.g., index finger) that instructs to unlock the electric lock 310, a finger (e.g., middle finger) that instructs to lock, and a finger (e.g., ring finger) that makes an emergency call. Then, if the target person 10 is not a suspicious person, the target person 10 places his / her index finger on the fingerprint scanner 150. If the third party is an unknown person, the target person 10 places his / her middle finger on the fingerprint scanner 150, and if the third party is clearly behaving suspiciously, the target person 10 places his / her ring finger on the fingerprint scanner 150. The authentication unit 130 identifies the placed finger (step S203). Specifically, the authentication unit 130 compares the fingerprint information acquired by the fingerprint scanner 150 with fingerprint information registered in advance for the subject 10, and recognizes the finger placed by the subject 10 on the fingerprint scanner 150. The authentication unit 130 outputs the operation content for the electric lock 310 and the notification device 200 corresponding to the recognized finger to the control unit 120. The control unit 120 controls the electric lock 310 and the notification device 200 based on the operation content indicated by the identified finger (step S215). For example, the control unit 120 unlocks / locks the electric lock 310 or makes an emergency call using the notification device 200, depending on the operation content acquired from the authentication unit 130. In addition, in a situation where a burglar or the like is about to break into the room and a fingerprint instruction gesture is not enough, the emergency switch 160 can be pressed hard, and the control unit 120 can activate the notification device 200 to report the emergency.

[0054] When it is determined that no third party is present (step S160; No), the process is the same as that shown in FIG.

[0055] According to this embodiment, in addition to the effects of the first embodiment, security can be improved by using the fingerprint scanner 150. Also, when priority is given to ensuring cleanliness and convenience of a non-contact system, the fingerprint scanner 150 can be used for emergency use and not used normally. Also, by providing an emergency switch 160, emergency calls can be made more reliably in the event of an emergency.

[0056] The security systems 1 and 2 of the first and second embodiments can be used, for example, to manage entry and exit to a server room, but can also be applied to other purposes such as logging into a computer and opening and closing the front door of an ordinary home.

[0057] The authentication device 100 described above has a computer inside. Each process of the authentication device 100 described above is stored in a computer-readable recording medium in the form of a program, and the computer reads and executes this program to perform the above processes. Here, the computer-readable recording medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, etc. Also, the computer program may be distributed to a computer via a communication line, and the computer that receives the program may execute the program. The program may be for implementing some of the functions described above, or may be a so-called differential file (differential program) that can implement the functions described above in combination with a program already recorded in the computer system.

[0058] In addition, the components in the above-described embodiment may be replaced with known components as appropriate without departing from the spirit of the present invention. Furthermore, the technical scope of the present invention is not limited to the above-described embodiment, and various modifications may be made without departing from the spirit of the present invention. The subject 10 is an example of a living body.

[0059] Note that some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0060] (Supplementary Note 1) An imaging unit that captures an image of a living body to be authenticated; an authentication unit that performs authentication of the living body based on one or more pieces of biometric information of the living body included in an image captured by the imaging unit and information indicated by a gesture of the living body included in the image captured by the imaging unit; An authentication device having the above configuration.

[0061] (Appendix 2) The biometric information is at least one of the face, vein pattern, fingerprint, and iris pattern of the biometric information, the authentication unit performs authentication by comparing biometric information included in the image captured by the imaging unit with a previously captured image including biometric information corresponding to the biometric information. 2. An authentication device as described in claim 1.

[0062] (Supplementary Note 3) The information indicated by the gesture of the living body included in the image is one or more numbers, The authentication unit performs authentication by comparing the one or more numbers with a predetermined number. 13. An authentication device according to claim 1 or 2.

[0063] (Appendix 4) The character is a shape indicated by the living body through a finger gesture, The authentication unit performs authentication based on the one or more shapes. 4. An authentication device as described in claim 3.

[0064] (Appendix 5) A control unit that controls the operation of a device connected to the device itself; Further equipped with When the authentication by the authentication unit is successful, the authentication unit recognizes the operation content to be instructed by the living body to the device based on at least one of a facial expression shown in a face image of the living body and information shown in a gesture of the living body, and the control unit controls the device based on the operation content. 5. An authentication device according to any one of claims 1 to 4.

[0065] (Supplementary Note 6) The device is a warning device that notifies a warning to the biometric subject to authentication, and when the authentication unit recognizes a biometric subject other than the biometric subject to authentication during authentication processing, The control unit controls the warning device to issue a warning notification. 6. An authentication device as described in claim 5.

[0066] (Appendix 7) The device is a notification device, When the authentication unit recognizes a biometric entity that fails to be authenticated, other than the biometric entity to be authenticated, the control unit controls the notification device to perform a notification operation based on a gesture of the biometric entity. 7. An authentication device according to claim 5 or 6.

[0067] (Appendix 8) The device is a notification device, When the authentication unit recognizes that an emergency has occurred in the living body to be authenticated, The control unit controls the notification device to perform a notification operation. 8. An authentication device according to any one of claims 5 to 7.

[0068] (Appendix 9) The device is a notification device, The authentication unit recognizes whether the information instructs the notification device to perform a notification operation based on at least one of a facial expression shown in a facial image of the living body and information shown in a gesture of the living body, and the control unit controls the notification device to perform a notification operation based on the recognition by the authentication unit. 9. An authentication device according to any one of claims 5 to 8.

[0069] (Appendix 10) The device is a control device that controls unlocking and locking of a door through which the living body passes, and the authentication unit recognizes whether the information instructs unlocking or locking the door based on at least one of an expression shown in a facial image of the living body and information shown in a gesture of the living body, and the control unit controls unlocking or locking the door based on the recognition by the authentication unit. 10. An authentication device according to any one of claims 5 to 9.

[0070] (Appendix 11) The imaging unit captures a distance image including information on a distance to the living body in addition to an image of the living body, The authentication device according to any one of Supplementary Note 1 to Supplementary Note 10, wherein the authentication unit performs image processing on the image captured by the imaging unit using the distance image.

[0071] (Appendix 12) A fingerprint detection unit for detecting a fingerprint of a living body to be authenticated; Further equipped with the authentication unit performs authentication using a fingerprint in addition to the biometric information and the information indicated by the biometric gesture; 12. An authentication device according to any one of claims 1 to 11.

[0072] (Appendix 13) When the authentication unit recognizes the biometric information to be authenticated and a biometric information other than the biometric information, the authentication unit requests authentication using a fingerprint detected by the fingerprint detection unit. 13. The authentication device of claim 12.

[0073] (Appendix 14) The authentication unit identifies a part of the body of the subject to be authenticated that corresponds to the fingerprint detected by the fingerprint detection unit, the control unit controls an operation of a device connected to the device itself by a control method that is predetermined in association with the part identified by the authentication unit. 13. An authentication device according to claim 11 or 12.

[0074] (Appendix 15) An imaging unit that captures an image of a living body to be authenticated; an authentication unit that performs authentication using a plurality of pieces of biometric information among the face of the living body captured by the imaging unit, the vein pattern of the throat of the living body, the vein pattern of the cheek of the living body, the vein pattern of the fingers of the living body, the vein pattern of the palm of the living body, the vein pattern of the back of the hand, the vein pattern of the arm, and the iris pattern of the living body; An authentication device having the above configuration.

[0075] (Appendix 16) The imaging unit is composed of one camera. 16. An authentication device according to any one of claims 1 to 15.

[0076] (Supplementary Note 17) An authentication device according to any one of Supplementary Note 1 to Supplementary Note 16; A reporting device; having The notification device issues a notification based on an instruction from the authentication device. Security system.

[0077] (Appendix 18) The notification device has an emergency button, The notification device issues a notification when the emergency button is operated. 18. A security system as described in claim 17.

[0078] (Appendix 19) Taking an image of the biometric subject to be authenticated; performing authentication of the living body based on one or more pieces of biometric information of the living body included in the captured image and information indicated by a gesture of the living body included in the captured image; Authentication method.

[0079] (Appendix 20) The computer of the authentication device, A means for acquiring an image of a living body to be authenticated; a means for authenticating the living body based on one or more pieces of biometric information of the living body included in the captured image and information indicated by a gesture of the living body included in the captured image; A program to function as a [Explanation of symbols]

[0080] 1, 2: Security system 100 Authentication device 111 Near infrared camera 112, 113...Near infrared light source 114...Pattern Filter 115···Light source switch 120...Control section 130 Authentication section 140... Warning lamp 150··Fingerprint scanner 160 Emergency switch 200...Notification device 210...Storage device 300···door 310... Electric lock

Claims

1. An acquisition unit that acquires an image of a subject captured by an imaging device; a subject recognition unit that recognizes the subject from the acquired image; a gesture recognition unit that recognizes a gesture of the target person after the target person recognition unit has successfully recognized the target person; A control unit that controls an action of a control target associated with the gesture; Equipped with The control unit outputs a notification to the target person that the target person recognition has been successful.

2. The apparatus further includes an acquisition means for acquiring a distance to the target person, When the gesture performed by the subject is detected based on the distance, the control unit controls the operation of the control target associated with the gesture. The information processing device according to claim 1 .

3. The control unit notifies whether the test was successful or not based on the state of light emission of the light-emitting component. The information processing device according to claim 1 .

4. The gesture recognition unit recognizes a predetermined movement of the subject. The information processing device according to claim 1 .

5. An acquisition step of acquiring an image of a subject captured by an imaging device; a subject recognition step of recognizing the subject from the acquired image by an information processing device; a gesture recognition step of recognizing a gesture of the target person by the information processing device after the target person recognition step is successful; a control step of controlling, by the information processing device, an action of a control target associated with the gesture; Equipped with The information processing method, wherein the control step outputs a notification to the subject that the subject recognition has been successful.

6. An acquisition step of acquiring a distance to the target person by the information processing device, When the gesture performed by the subject is detected based on the distance, the control step controls the operation of the control target associated with the gesture. The information processing method according to claim 5.

7. The control step notifies whether the operation has been successful or not based on the state of light emission of the light-emitting component. The information processing method according to claim 5.

8. An acquisition step of acquiring an image of a subject captured by an imaging device; a subject recognition step of recognizing the subject from the acquired image by an information processing device; a gesture recognition step of recognizing a gesture of the target person by the information processing device after the target person recognition step is successful; a control step of controlling, by the information processing device, an action of a control target associated with the gesture; Equipped with The control step outputs a notification to the subject that the subject recognition was successful.

9. An acquisition step of acquiring a distance to the target person by the information processing device, When the gesture performed by the subject is detected based on the distance, the control step controls the operation of the control target associated with the gesture. The program according to claim 8.

10. The control step notifies whether the operation has been successful or not based on the state of light emission of the light-emitting component. The program according to claim 8.

Images