Information processing device, control method, program, and storage medium
The information processing device enhances facial recognition authentication by assessing face detection reliability, improving efficiency and speed through a systematic verification process.
Patent Information
- Application Number
- JP2023086119
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-25
- Publication Date
- 2024-12-05
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing facial recognition systems for authentication are inefficient and slow due to the lack of reliable face detection and verification processes.
An information processing device equipped with an image acquisition unit, detection unit, calculation unit, and determination unit to assess the reliability of face detection and verification, allowing for quicker authentication by optimizing the verification process based on reliability scores.
Facial authentication is accelerated by implementing a system that evaluates face detection reliability, ensuring efficient and accurate verification processes.
Smart Images

Figure 2024168978000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device, a control method, a program, and a storage medium. [Background technology]
[0002] There is a known technology that captures the face of a person to be authenticated with a tablet PC camera, displays the authentication result on the display, and unlocks the door.
[0003] Patent Document 1 discloses a technique in which standards are set for the size and position of a face, and permission to pass is given based on face recognition if the standards are met. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] Patent 6151582 Summary of the Invention [Problem to be solved by the invention]
[0005] The problem that the present invention aims to solve is to perform authentication processing more quickly. [Means for solving the problem]
[0006] In order to solve the above problem, an information processing device according to one embodiment of the present invention has an image acquisition unit that acquires an image, a detection unit that detects a face from the image, a calculation unit that calculates a reliability of the detection result of the detection unit, and a judgment unit that judges whether or not to continue the face matching process by a matching unit based on the reliability calculated by the calculation unit. Effect of the Invention
[0007] According to the present invention, the authentication process can be performed more quickly. [Brief description of the drawings]
[0008] [Figure 1] Hardware and functional configuration diagram of information processing device [Diagram 2] A flowchart showing the operation of authentication processing by an information processing device. [Diagram 3] A flowchart showing the operation of a reliability determination process by an information processing device. [Figure 4] Diagram showing how a door is unlocked using facial recognition [Diagram 5] A diagram showing facial recognition images and the facial recognition scores for each frame. [Figure 6] A diagram showing facial recognition images and the facial recognition scores for each frame. [Figure 7] A diagram showing facial recognition images and the facial recognition scores for each frame. [Figure 8] Diagram showing face recognition [Figure 9] A flowchart showing a modified example of the operation of the information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] <Embodiment 1> (Hardware configuration) 1A is a diagram showing an example of a hardware configuration of an information processing device 100 according to this embodiment. A CPU (Central Processing Unit) 101 is a central processing unit that performs calculations and logical determinations for various processes and controls each component connected to a system bus 110.
[0010] Memories including a program memory and a data memory are mounted on the information processing device 100. A ROM (Read Only Memory) 102 is a program memory, and stores programs for control by the CPU 101, including various processing procedures described later.
[0011] The RAM (Random Access Memory) 103 is a data memory, and has a work area for the above programs of the CPU 101, an area for saving data during error processing, an area for loading the above control programs, and the like.
[0012] The program memory may be realized by loading a program into the RAM 103 from an external storage device connected to the information processing device 100. The HDD 104 is a hard disk for storing a plurality of electronic data and programs according to the present embodiment. An external storage device may be used to perform a similar function. Here, the external storage device may be realized by, for example, a medium (recording medium) and an external storage drive for realizing access to the medium. Examples of such media include a flexible disk (FD), a CD-ROM, a DVD, a USB memory, an MO, and a flash memory. The external storage device may also be a server device connected via a network.
[0013] The input device 105 is a device for inputting user operation information into the information processing device 100. For example, it includes a pointing device such as a mouse or a touch panel, a joystick, a keyboard, and the like. The output device 106 is a device for display output. The communication interface (I / F) 107 performs wired or wireless two-way communication with other information processing devices, communication devices, external storage devices, and the like using known communication technology. In this embodiment, the communication I / F 107 is connected to an imaging device 108, which is a network camera, via a LAN and receives captured images in real time. The imaging device 108 is a network camera that is installed in a state connected to a network and captures images that include the face of a person to be authenticated within the angle of view. The imaging device 108 may be incorporated in the information processing device 100, or a camera connected via USB other than a network camera may be used.
[0014] The entrance / exit management system 109 executes various processes related to locking and unlocking the electric lock according to the results of the processes performed by the information processing device 100 .
[0015] (Functional configuration) 1B is a block diagram showing an example of a functional configuration of the information processing device 100. The information processing device 100 has a control unit 111, an image acquisition unit 112, a detection unit 113, a matching unit 114, a registration unit 115, a calculation unit 116, a determination unit 117, a storage unit 118, and a communication unit 119. Each of these functional units is realized by the CPU 101 expanding a program stored in the ROM 102 into the RAM 103 and executing processing according to each flowchart described later. The storage unit 118 is a functional unit of the RAM 103. However, the embodiment of the present invention can also be realized by an information processing device that realizes these functional units by hardware. Each element will be described below.
[0016] The control unit 111 is composed of the CPU 101 and the like, and executes various processes by reading out programs and data stored in the ROM 102 into the RAM 103. The image acquisition unit 112 acquires an image captured by the imaging device .
[0017] The detection unit 113 performs processing to detect a person's face from the image acquired by the image acquisition unit 112 and extract features of the detected face. The matching unit 114 performs processing to match the face detected by the detection unit 113. That is, by performing processing to match the detected face with a registered face acquired from the registration unit 115, a similarity score between the detected face and the registered face is calculated. More specifically, a similarity score between a feature amount extracted from the detected face image and a feature amount extracted from the registered face image (or a registered feature amount) is calculated. The matching result is the similarity score and the registered name of the face with the highest similarity score. Note that if there is no registered face whose similarity score with the detected face is equal to or higher than a predetermined threshold, only the similarity score may be output as the matching result.
[0018] The registration unit 115 registers / deletes a face (image) used in the matching process by the matching unit 114 in the storage unit 118, and reads out a registered face (image) from the storage unit 118.
[0019] The calculation unit 116 calculates the reliability of the detection result of the detection unit 113. The reliability here is a score indicating whether the face detected by the detection unit 113 is a face (image) of sufficient quality to be used in the matching process of the matching unit 114. That is, a face (image) with high reliability is, for example, an image in which the face faces the imaging device 108 directly and an image of a sufficient size to be used in the matching process. A specific method of calculating the reliability will be described later.
[0020] The determination unit 117 determines whether or not to continue the matching process by the matching unit 114 based on the determination of the matching result by the matching unit 114 and the reliability.
[0021] The storage unit 118 stores various information such as registered faces (images), registered names, calculated reliability, etc. The communication unit 119 transmits information related to unlocking / locking the electronic lock to the entrance / exit management system 109. In this embodiment, information related to unlocking / locking the electronic lock is transmitted, but for example, information related to the opening / closing control of automatic doors or payment information of an authenticated person may also be transmitted.
[0022] Among the functions of the information processing device 100 according to this embodiment, at least some of the functions of the matching unit 114, the registration unit 115, and the storage unit 118 may be functions that are separated from the information processing device 100. That is, the functions of the matching unit 114, the registration unit 115, and the storage unit 118 may be realized by an external information processing device (authentication server). In that case, the face (image) detected by the detection unit 113 and a request for matching processing are transmitted from the communication unit 119 to the authentication server via a network, and the communication unit 119 receives the result of the matching processing by the authentication server. At this time, the function of extracting facial features among the functions of the detection unit 113 can also be realized by the authentication server. When the facial features are extracted on the information processing device 100 side, the features are transmitted to the authentication server instead of the face (image).
[0023] A part of the functions of the storage unit 118 of the information processing device 100 may still remain in the information processing device 100. This is because the reliability calculated by the calculation unit 116 is stored in the storage unit 118. The registered face (image), registered name (or registered ID), etc. are stored in the storage unit 118 on the authentication server side.
[0024] In the authentication server, a matching unit 114 performs a matching process (authentication process) to determine whether the face (image) or its features received from the information processing device 100 is a registered face (image or feature), and the matching result is transmitted to the information processing device 100. In this way, the matching unit 114, the registration unit 115, and the storage unit 118 are not essential components of the information processing device 100, and may be implemented by another information processing device.
[0025] (Authentication process) 2 is a flowchart of processing executed by the control unit 111 of the information processing device 100 according to this embodiment. The processing of this flowchart is realized by the CPU 101 loading a program for executing each process of this flowchart stored in the ROM 102 into the RAM 103 and executing it. The processing of this flowchart can be repeatedly executed even after a series of processes has ended.
[0026] The user moves to the front of the information processing device 100 as shown in 400 to 401 in Fig. 4, face authentication is performed, and if it is determined that the authentication is OK in this flowchart, the electric lock of the corresponding door is unlocked as shown in 402 in Fig. 4. Each process (step) will be explained below by adding an S to the beginning of each reference number.
[0027] In S200, the image acquisition unit 112 acquires an image of the person to be authenticated captured by the imaging device 108. Note that until the image of the person to be authenticated is acquired in S200, a message such as 403 is displayed to prompt the person to bring their face closer.
[0028] In S201, the detection unit 113 detects a face from the image acquired in S200. In this embodiment, the detection result includes at least the position of the center of the face in the image (X coordinate, Y coordinate), the orientation of the face (pitch angle, roll angle, yaw angle), and the size (number of pixels), width, and height of the face.
[0029] In S202, the detection unit 113 performs a process of extracting features. In S203, the matching unit 114 performs a process of matching the features of the face obtained in the process of S202 with the registered face obtained from the registration unit 115. As a result of the matching, the similarity score and the registered name of the face determined to be most similar are obtained.
[0030] In S204, the matching unit 114 determines whether the similarity score of the face matching result in S203 is equal to or greater than a predetermined threshold value. If it is equal to or greater than the predetermined threshold value, the process proceeds to S205, and if it is less than the predetermined threshold value, the process proceeds to S206.
[0031] In S205, the judgment unit 117 judges how many times the similarity score was judged to be equal to or greater than the predetermined threshold in the judgment in S204. In this embodiment, if it is judged to be equal to or greater than the predetermined threshold N times in a row, the process proceeds to S208, and if not, the process returns to S200. The judgment result is stored in the storage unit 118. N can be set to any integer equal to or greater than 1, and is set by the user or designer. It is not necessary that the judgments that the similarity score is equal to or greater than the predetermined threshold be consecutive. For example, if the cumulative number of times that the similarity score is judged to be equal to or greater than the predetermined threshold within a predetermined period is N times, the authentication may be deemed OK (proceed to S208).
[0032] In S206, the judgment unit 117 judges whether or not the total number of judgments in S204 is L. If the total number of judgments is L, it is judged that sufficient judgments have been performed for authentication and the process proceeds to S209, and if the total number of judgments is less than L, the process proceeds to S207.
[0033] L can be set to any integer equal to or greater than 1, and is set by the user or designer. If both N and L are set to 1, the determinations of S205 and S206 are not actually performed, and if it is determined that the similarity score is equal to or greater than a predetermined threshold, the process proceeds to S208, and if it is determined that the similarity score is less than the predetermined threshold, the process proceeds to S207. In the subroutine process of S207, as will be described later in detail, if L is 1, a determination is made to return to step S200, and face detection processing is performed again.
[0034] As described above, the process of this flowchart works if N and L are 1 or greater, but if both N and L are set to 1, authentication will be determined based on a single similarity assessment, so it is desirable for the values of N and L to be 2 or greater. An example of suitable settings for N and L will be described later in "More detailed operation explanation."
[0035] In S207, a subroutine process for determining the reliability is performed by the determination unit 117. Details will be described later in "Reliability Determination Process".
[0036] In S208, the control unit 111 determines that the authentication is OK, and the process proceeds to step S210. At this time, the display on the output device 106 becomes as shown in 404. Note that the judgment result of the judgment unit 117 and the consecutive / cumulative number of times stored in the memory unit 118 are reset.
[0037] In S209, the control unit 111 judges that the authentication is NG, and ends the process. At this time, the display on the output device 106 becomes as shown in 405. Note that the judgment result of the judgment unit 117 and the consecutive / cumulative number thereof stored in the memory unit 118 are reset.
[0038] In S210, the communication unit 119 instructs the entrance / exit management system to unlock the door, and the process of this flowchart ends.
[0039] As described above, among the processes in this flowchart, the processes of S202 and S203 may be executed by an information processing device (authentication server) other than the information processing device 100. In that case, instead of the processes of S202 and S203, a process of transmitting a matching process request against the detected face (image) to the authentication server and a process of receiving a result of the matching process by the authentication server are executed.
[0040] (Reliability determination process) Next, the reliability determination process of S207 will be described with reference to the flowchart of Fig. 3. Like the authentication process, the process of this flowchart is also realized by the CPU 101 loading a program stored in the ROM 102 into the RAM 103 and executing it.
[0041] In S300, the calculation unit 116 calculates the reliability of the detection result based on the detection result of the detection unit 113 in S201. The detection result by the detection unit 113 from the start of the process of the flowchart in FIG. 2 until the start of the subroutine process of S207 is stored in the storage unit 118. The reliability is represented as SCORE, and the reliability based on the face position is Score. p The face size is used as the confidence score. s The reliability based on the face orientation is Score o Then, for example, it can be calculated using the following formula 1. SCORE=Score p +Score s +Score o ...(Formula 1)
[0042] It is desirable to set the score for the face position so that the closer it is to the center of the screen, the higher the score. p If the coordinates of the center of the screen are (Xs, Ys) and the coordinates of the center of the face are (Xf, Yf), then it can be calculated, for example, using the following formula, where A is a coefficient.
[0043]
number
[0044] It is desirable to set the score according to the face size so that the larger the face, the higher the score. s If the face size is SF, it can be calculated using the following formula, where B is a coefficient. However, if either the width or height of the face is larger than the size of the screen, it is set to 0. Score s =B*SF...(Formula 3)
[0045] It is desirable to set the score based on the face direction so that the score is higher when the face is facing forward. o If the pitch angle of the face is PF, the roll angle of the face is RF, and the yaw angle of the face is YF, then, for example, γ can be calculated by the following formula: a , γ b , γ c is a coefficient.
[0046]
number
[0047] Note that these calculation methods are merely examples, and are not limited to these calculation methods. Each reliability can be weighted, and a user or designer can set a weight for each reliability. Depending on the weighting settings, the reliability may be calculated using only the reliability based on the face position. The same applies to the face size and face direction, and the reliability score is calculated based on at least one of the face detection position, the face size, and the face direction.
[0048] As another example of calculating the reliability score, the reliability may be calculated by using a score according to the brightness of the captured image, and the brighter the image, the higher the score. Also, the reliability may be calculated by detecting blur and blur in the image acquired in S200, and the lower the blur and blur, the higher the score. In this way, the reliability score may be calculated based on at least one of the blur, blur, and brightness of the image acquired by the image acquisition unit 112.
[0049] In S300, the reliability value is held in the storage unit 118 until the processing of the flowchart in FIG. 2 ends.
[0050] In S301, the determination unit 117 determines whether or not to continue the matching process by the matching unit 114 based on the reliability calculated by the calculation unit 116. More specifically, the determination is made based on a change in the reliability. In this embodiment, a description is given of a determination based on the reliability of the past M face detection results. M is an integer of 2 or more that can be set by a user or a designer.
[0051] In S300, if the amount of change in reliability is within a predetermined range, the process proceeds to S302. If the change in reliability is on a decreasing trend, the process may proceed to S302. If the amount of change in reliability is outside the predetermined range, the process proceeds to S303. Note that if the reliability held in the storage unit 118 is less than M times (i.e., if the number of times the reliability has been calculated is less than a predetermined threshold), the process proceeds to S303 regardless of whether the amount of change in reliability is within the predetermined range or not. If the change in reliability is on an increasing trend, the process may proceed to S303. If no detection results are stored in the storage unit 118, the number of times the reliability has been calculated will inevitably be 0, and the process automatically proceeds to S303.
[0052] The amount of change in the reliability is calculated by the calculation unit 116. The amount of change may be calculated based on the reliability score initially stored in the storage unit 118, or based on the previous reliability score (i.e., the amount of change in the reliability score between frames in which a face is detected).
[0053] In S302, the judgment unit 117 judges that the matching process by the matching unit 114 is to be ended, and in S303, it judges that the matching process is to be continued. The subroutine process ends. As shown in FIG. 2, if it is judged in the subroutine process of S207 that the matching process by the matching unit 114 is to be continued, the process returns to S200. On the other hand, if it is judged that the matching process is to be ended, the process proceeds to S209, where it is judged that the authentication is NG. By performing the above processes, the authentication process can be performed more quickly.
[0054] (More detailed operation explanation) Next, an example in which authentication NG is determined in the flowchart of FIG. 2 described above will be described using a specific example in FIG. 5(A). FIG. 5(A) is an image acquired in S200 during authentication, and is described as a display example displayed on the output device 106. FIG. 5(B) shows the relationship between the similarity score and time for each frame processed. The same reference numerals are attached to information corresponding to the diagram shown in FIG. 5(A). FIG. 5(C) shows the relationship between the reliability score and time for each frame processed. The similarity threshold value for determining that the two people are the same person is the value indicated by TH in the diagram. The same reference numerals are attached to information corresponding to the diagram shown in FIG. 5(A). In the flowchart described above, N is 2, L is 13, and M is 4. Also, the person to be authenticated is an unregistered person. 500 in FIG. 5(A) shows the state immediately after the person to be authenticated enters the captured image.
[0055] In S200, an image captured by the imaging device 108 is acquired, and the processes of S202 to S204 are carried out. Since the similarity is less than a predetermined threshold, the process proceeds to S206. Since the cumulative number of judgments is 1 at this point, the process proceeds to S207. Next, in S300, the reliability is calculated and stored in the storage unit 118. In S301, it is determined whether or not to continue the matching process based on the reliability of the past four times. Since the reliability of four times has not been stored, the process proceeds to S303, where it is determined to continue the matching process, and the process returns to S200.
[0056] The process when an image corresponding to 501 in FIG. 5A is input will be described. In S200, the corresponding image is acquired, and the processes from S202 to S204 are carried out. Since the similarity is less than a predetermined threshold, the process proceeds to S206. Since the cumulative number of judgments is four, the process proceeds to S207. Next, in S300, the reliability is calculated and stored in the memory unit 118. In S301, it is determined whether or not to continue the matching process based on the reliability of the past four times. Since the amount of change in the reliability of the four times is outside a predetermined range, the process proceeds to S303, where it is determined to continue the matching process, and the process returns to S200.
[0057] Next, the process when an image corresponding to 502 in FIG. 5A is input will be described. In S200, the corresponding image is acquired, and the processes from S202 to S204 are carried out. Since the similarity is less than a predetermined threshold, the process proceeds to S206. Since the cumulative number of judgments is 4, the process proceeds to S207. Next, in S300, the reliability is calculated and stored in the memory unit 118.
[0058] In S301, it is determined whether or not to continue the matching process based on the reliability of the past four times. Since the amount of change in reliability over the four times is within a predetermined range, the process proceeds to S302, where it is determined that the matching process is to be ended, and the process proceeds to S209. In S209, it is determined that authentication is NG, and the determination result of the determination unit 117 and its consecutive / cumulative counts are reset. Even after the determination of authentication is NG, the process of the flowchart in FIG. 2 is executed again.
[0059] The process when an image corresponding to 503 in FIG. 5A is input will be described. In S200, the corresponding image is acquired, and the processes from S202 to S204 are carried out. Since the similarity is less than a predetermined threshold, the process proceeds to S206. Since the cumulative number of judgments is 1, the process proceeds to S207. Next, in S300, the reliability is calculated and stored in the memory unit 118. In S301, it is determined whether or not to continue the matching process based on the reliability of the past four times. Since the reliability of four times has not been stored, the process proceeds to S303, where it is determined that the matching process should be continued, and the process returns to S200. As described above, the operation is performed to prevent a mistaken authentication NG judgment from being made when the person to be authenticated goes outside the shooting range.
[0060] An example of the determination of authentication OK in the flowchart of FIG. 2 described above will be described in more detail with reference to FIG. 6(A). FIG. 6(A) shows an image acquired in S200 during authentication, and is shown as an example of display displayed on the output device 106. FIG. 6(B) shows the relationship between the similarity score for each frame processed and time. The same reference numerals are attached to information corresponding to the diagram shown in FIG. 6(A). FIG. 6(C) shows the relationship between the reliability score for each frame processed and time. The similarity threshold value for determining that the person is the same is the value indicated by TH in the diagram. The same reference numerals are attached to information corresponding to the diagram shown in FIG. 6(A). In the flowchart described above, N is 2, L is 13, and M is 4. Also, the person to be authenticated is a person who has already been registered. 600 in FIG. 5(A) shows the state immediately after the person to be authenticated enters the captured image.
[0061] In S200, a corresponding image is acquired, and the processes from S202 to S204 are carried out. Since the similarity is less than a predetermined threshold, the process proceeds to S206. Since the cumulative number of judgments is 1, the process proceeds to S207. Next, in S300, the reliability is calculated and stored in the memory unit 118. In S301, it is determined whether or not to continue the matching process based on the reliability of the past four times. Since the reliability of four times has not been stored, the process proceeds to S303, where it is determined to continue the matching process, and the process returns to S200.
[0062] The process will be described when an image corresponding to 601 in FIG. 6A is input. In S200, the corresponding image is acquired, and the processes from S202 to S204 are carried out. Since the similarity is less than a predetermined threshold, the process proceeds to S206. Since the cumulative number of judgments is four, the process proceeds to S207. Next, in S300, the reliability is calculated and stored in the memory unit 118. In S301, it is determined whether or not to continue the matching process based on the reliability of the past four times. Since the amount of change in the reliability of the four times is outside a predetermined range, the process proceeds to S303, where it is determined to continue the matching process, and the process returns to S200.
[0063] Next, the process when an image corresponding to 602 in Fig. 6(A) is input will be described. In S200, the corresponding image is acquired, and the processes from S202 to S204 are carried out. Since the similarity is equal to or greater than the predetermined threshold, the process proceeds to S205. Since the number of times that the similarity has been determined to be equal to or greater than the predetermined threshold is 2, the process proceeds to S208. In S208, authentication is determined to be OK, and the determination result of determination unit 117 and its consecutive / cumulative number are reset. Even after a determination of authentication NG is made, the process in the flowchart of Fig. 2 continues until the person to be authenticated leaves the shooting range.
[0064] The process when an image corresponding to 603 in FIG. 6A is input will be described. In S200, the corresponding image is acquired, and the processes from S202 to S204 are carried out. Since the similarity is less than a predetermined threshold, the process proceeds to S206. Since the cumulative number of judgments is 1, the process proceeds to S207. Next, in S300, the reliability is calculated and stored in the memory unit 118. In S301, it is determined whether or not to continue the matching process based on the reliability of the past four times. Since the reliability of four times has not been stored, the process proceeds to S303, where it is determined that the matching process should be continued, and the process returns to S200. As described above, the process operates to prevent a mistaken authentication NG judgment from being made when going out of the shooting range.
[0065] Next, an example in which authentication NG is determined in the flowchart of FIG. 2 described above will be described using a specific example in FIG. 7(A). FIG. 7(A) is an image acquired in S200 during authentication, and is described as a display example displayed on the output device 106. FIG. 7(B) shows the relationship between the similarity score for each frame processed and time. The same reference numerals are attached to information corresponding to the diagram shown in FIG. 7(A). FIG. 7(C) shows the relationship between the reliability score for each frame processed and time. The similarity threshold for determining that the two people are the same person is the value indicated by TH in the diagram. The same reference numerals are attached to information corresponding to the diagram shown in FIG. 7(A). In the flowchart described above, N is 2, L is 13, and M is 4. Also, the person to be authenticated is an unregistered person. 700 in FIG. 7(A) shows the state immediately after the person to be authenticated enters the captured image.
[0066] In S200, a corresponding image is acquired, and the processes from S202 to S204 are carried out. Since the similarity is less than a predetermined threshold, the process proceeds to S206. Since the cumulative number of judgments is 1, the process proceeds to S207. Next, in S300, the reliability is calculated and stored in the memory unit 118. In S301, it is determined whether or not to continue the matching process based on the reliability of the past four times. Since the reliability of four times has not been stored, the process proceeds to S303, where it is determined to continue the matching process, and the process returns to S200.
[0067] The process will be described when an image corresponding to 701 in FIG. 7A is input. In S200, the corresponding image is acquired, and the processes from S202 to S204 are carried out. Since the similarity is less than a predetermined threshold, the process proceeds to S206. Since the cumulative number of judgments is four, the process proceeds to S207. Next, in S300, the reliability is calculated and stored in the memory unit 118. In S301, it is determined whether or not to continue the matching process based on the reliability of the past four times. Since no increasing trend is observed in the reliability, the process proceeds to S303, where it is determined to continue the matching process, and the process returns to S200.
[0068] Next, the process when an image corresponding to 702 in FIG. 7A is input will be described. In S200, the corresponding image is acquired, and the processes from S202 to S204 are carried out. Since the similarity is less than a predetermined threshold, the process proceeds to S206. Since the cumulative number of judgments is 8, the process proceeds to S207. Next, in S300, the reliability is calculated and stored in the memory unit 118. In S301, it is determined whether or not to continue the matching process based on the reliability of the past four times. Since the reliability has not shown any tendency to increase even once, the process proceeds to S303, where it is determined to continue the matching process, and the process returns to S200.
[0069] Next, the process when an image corresponding to 702 in Fig. 7(A) is input will be described. In S200, the corresponding image is acquired, and the processes from S202 to S204 are carried out. Since the similarity is less than a predetermined threshold, the process proceeds to S206. Since the cumulative number of judgments is 13, the process proceeds to S209. In S209, the authentication is judged as NG, and the judgment result of the judgment unit 117 and its consecutive / cumulative number are reset. If the authentication result remains undetermined through the above process flow, it may be judged as NG.
[0070] (Modification) An example of displaying a guide to an authentication target person using the reliability will be described. Fig. 9 is a flowchart of a process executed by the control unit 111 of the information processing device 100 when displaying a guide. When performing the same process as in the flowchart of Fig. 2 described above, the same reference numerals will be used and the description thereof will be omitted. Each step (step) will be described below with an S added to the beginning of the reference numerals.
[0071] In S900, the control unit (display control unit) 111 determines the content of the guide to be displayed on the output device (display unit) 106 based on the reliability calculated by the calculation unit 116. p , score according to face size s , Score based on face orientation o In this case, the guide corresponding to the lowest score is determined. Note that a predetermined threshold value may be set for each score, and an abstract guide such as "move your face closer" may be displayed until there is one score that does not reach the threshold. In S901, the control unit (display control unit) 111 causes the output device (display unit) 106 to display a guide based on the guide content determined in S900.
[0072] Next, a specific example of displaying a guide to a person to be authenticated using the reliability will be described with reference to FIG. 8. p , score according to face size s , Score based on face orientation o At the 800 mark, the face is almost facing forward. o exceeds the threshold, but the face position and face size score Score p , Score s does not satisfy a predetermined threshold. Since there are multiple scores that do not reach the predetermined threshold, in S900, an abstract guide is set to "move your face closer," and in S901, the guide is displayed.
[0073] Next, at the time of 801, the face was facing almost forward. o has a high score, but the score for face position and face size is p , Score s does not meet the predetermined threshold. Since there are multiple scores that do not reach the predetermined threshold, in S900, an abstract guide is set to "please move your face closer", and the guide is displayed in S901. At the point of 802, the score Score according to the face position is calculated as shown in the flowchart of FIG. p If the face orientation score exceeds a certain threshold, o and face position score p exceeds a certain threshold, the face size score Score s Since only 1 is below the threshold, Score s A guide is set to improve the recognition quality. The guide may be displayed as a character string or an icon. If the person to be authenticated follows the guide and aligns his / her face with the camera as shown in 803, the face recognition will be performed under better conditions, and the person will be more likely to be judged as authenticated.
[0074] As described above, by applying the present invention, it is possible to perform accurate and rapid determination at the timing when the person to be authenticated is able to match the face.
[0075] <Other embodiments> The present invention can be realized by a process of reading and executing a program that realizes one or more functions of the above-mentioned embodiment 1. The program is supplied to a system or device via a network or a storage medium, and is read and executed by one or more processors in the computer of the system or device. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.
Claims
1. an image acquisition unit that acquires an image; A detection unit that detects a face from the image; A calculation unit that calculates a reliability of a detection result of the detection unit; a determination unit that determines whether or not to continue the face matching process by the matching unit based on the reliability calculated by the calculation unit; 13. An information processing device comprising:
2. The information processing apparatus according to claim 1 , wherein the calculation unit calculates the reliability based on at least one of a position where the face is detected, a size of the face, and a direction of the face.
3. The information processing device according to claim 1 , wherein the determination unit determines whether or not to continue the matching process by the matching unit when the matching unit determines that the face is not a registered face as a result of the matching process by the matching unit.
4. The information processing device according to claim 3 , wherein the determination unit does not determine whether or not to continue the matching process by the matching unit when the matching unit determines that the face is a registered face as a result of the matching process by the matching unit.
5. The information processing apparatus according to claim 1 , wherein the determination unit determines whether to continue the matching process by the matching unit based on an amount of change in the reliability.
6. The information processing apparatus according to claim 1 , wherein the determination unit determines that the matching process by the matching unit is to be terminated when an amount of change in the reliability is within a predetermined range.
7. The information processing apparatus according to claim 1 , wherein the determination unit determines to continue the matching process by the matching unit when the amount of change in the reliability is outside a predetermined range.
8. The information processing apparatus according to claim 1 , wherein the determination unit determines that the matching process by the matching unit is to be terminated when the reliability is showing a decreasing trend.
9. The information processing apparatus according to claim 1 , wherein the determination unit determines to continue the matching process by the matching unit when the reliability is showing an increasing trend.
10. The information processing apparatus according to claim 1 , wherein the calculation section calculates a higher reliability as the position of the face detected by the detection section is closer to the center of the image.
11. The information processing apparatus according to claim 1 , wherein the calculation unit calculates a higher reliability as the size of the face detected by the detection unit increases.
12. The information processing apparatus according to claim 1 , wherein the calculation unit calculates a higher reliability as the orientation of the face detected by the detection unit approaches the front.
13. The information processing apparatus according to claim 1 , wherein the calculation unit calculates the reliability based on at least one of blur, blurring, and brightness of the image acquired by the image acquisition unit.
14. 2 . The information processing device according to claim 1 , further comprising a display control unit that causes a guide for increasing the reliability to be displayed on a display unit when the determination unit determines that the matching process by the matching unit should be continued.
15. The information processing apparatus according to claim 1 , further comprising a storage unit that stores the reliability calculated by the calculation unit.
16. The information processing apparatus according to claim 1 , further comprising the collation unit.
17. A registration unit that registers information about a person's face, 17. The information processing apparatus according to claim 16, wherein the matching process is a process of matching the face detected by the detection section with a face registered in the registration section.
18. an image acquisition step of acquiring an image; a detection step of detecting a face from the image; a calculation step of calculating a reliability of a detection result by the detection step; a determination step of determining whether or not to continue the face matching process based on the reliability calculated in the calculation step; 13. A method for controlling an information processing apparatus comprising the steps of:
19. A program for causing a computer to execute the method for controlling an information processing device according to claim 18.
20. A computer-readable storage medium storing the program according to claim 19.
Citation Information
Patent Citations
Direction searching device
JP1986051582A
