Image processing device, image processing method, and program

Abstract

To provide an image processing device, image processing method, and program, which suppress unauthorized authentication breakthrough in facial authentication.SOLUTION: An image processing device is provided, comprising a feature point detection unit configured to detect feature points K1 of a first image P1 of a background B and feature points K2 of a second image P2 of an authentication target T with the background B, and a determination unit configured to determine the presence or absence of an unauthorized authentication breakthrough attempt by comparing the feature points K1 of the first image P1 and the feature points K2 of the second image P2.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] Conventionally, as one of biometric authentication technologies, a face authentication technology is known that acquires a face image of a person to be authenticated and verifies whether or not the person is the person by comparing the face image with a face image registered in advance. For example, Patent Document 1 discloses an authentication system that performs face authentication using a mobile terminal carried by a user and a face image transmitted from the mobile terminal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-47102 Summary of the Invention [Problem to be solved by the invention]

[0004] If the portable terminal used in the authentication system disclosed in Patent Document 1 falls into the hands of a third party, the third party may be able to illegally breach the authentication system using the portable terminal and a facial image stored within the portable terminal.

[0005] The present invention has been made to solve the above-mentioned problems, and has an object to provide an image processing device, an image processing method, and a program that prevent fraudulent authentication hacking in face authentication. [Means for solving the problem]

[0006] In order to achieve the above object, the image processing device of the present invention comprises: a feature point detection unit that detects feature points of a first image showing a background and feature points of a second image showing an object to be authenticated against the background; a discrimination unit that discriminates whether or not an attempt has been made to break through the authentication by comparing a feature point of the first image with a feature point of the second image; Equipped with. Effect of the Invention

[0007] According to the present invention, it is possible to prevent fraudulent authentication breakthroughs in face authentication. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1A is a diagram showing a first image used in an image processing device according to an embodiment of the present invention, and FIG. 1B is a diagram showing a second image. [Diagram 2] FIG. 1A is a diagram showing feature points of a first image, FIG. 1B is a diagram showing feature points of a second image, and FIG. 1C is a diagram showing feature points of a third image. [Diagram 3] 13 is a diagram illustrating example feature points detected in a second image when an unauthorized authentication breakthrough is attempted. FIG. [Figure 4] 1 is a block diagram of an image processing apparatus according to an embodiment of the present invention; [Diagram 5] 1A to 1C are diagrams illustrating a guidance image according to an embodiment of the present invention. [Figure 6] 5(a) to (c) are diagrams for explaining images when the authentication target has moved according to FIGS. 5(a) to (c). [Figure 7] 11 is a flowchart of a face authentication entry / exit process according to an embodiment of the present invention. [Figure 8] 11 is a flowchart of a human detection process according to an embodiment of the present invention. [Figure 9] 4 is a flowchart of a face recognition process according to an embodiment of the present invention. [Figure 10] FIG. 2 is a diagram illustrating parallel processing of an image processing device according to an embodiment of the present invention. [Figure 11] 13A and 13B are diagrams illustrating a method for determining whether or not an unauthorized authentication has been broken through according to a modified example. [Figure 12]13 is a diagram illustrating parallel processing of an image processing device according to a modified example of the present invention. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An image processing apparatus according to an embodiment of the present invention will be described in detail below with reference to the drawings.

[0010] [Image processing device overview] An image processing device according to an embodiment of the present invention performs face authentication when entering or leaving a managed area (security area) where only a limited number of staff members (authentication target T) are permitted to enter, such as a train station or an airport. For face authentication, for example, a first image P1 showing a background B as shown in Fig. 1(a) and a second image P2 showing a face F and a part of a body C of the authentication target T with the background B in the background as shown in Fig. 1(b) are used. In addition, authentication using an IC (Integrated Circuit) card can be performed together with or alone the face authentication.

[0011] As shown by circles in Figs. 2(a) and (b), the image processing device 10 detects a feature point K1 in the first image P1 and a feature point K2 in the second image P2. For example, feature points K1 and K2 are detected at the ends and intersections of an object by Harris' corner detection method. The authentication target T is shown in the second image P2, and the authentication target T is not shown in the first image P1. Therefore, when comparing the feature points K1 and K2 in the first image P1, the feature points in the position corresponding to the feature point K1H shown in Fig. 2(c) in number and position among the feature points K1 cannot be confirmed in Fig. 2(b). At this time, for example, if a third party attempts to fraudulently break through face authentication using a tablet, smartphone, or output photo showing the authentication target T, for example, as shown in Fig. 3, a feature point K2S constituting a corner of a rectangle surrounding the authentication target T is detected in the second image P2 in addition to the feature point K2 of the authentication target T. When such a feature point K2S is detected, the image processing device 10 determines that an unauthorized face authentication has been attempted, and does not permit the third party to enter or leave. Thus, the image processing device 10 can prevent unauthorized authentication breakthroughs in face authentication.

[0012] [Image processing device configuration] The following describes the configuration of the image processing device 10. As shown in Fig. 4, the image processing device 10 includes a control unit 20 that detects and compares feature points K1 and K2, a storage unit 30 that stores a face image of the authentication target T, etc., an imaging unit 40 that captures an image of the authentication target T and a background B, a communication unit 50 that communicates with an entrance / exit gate, a display unit 60 that displays the captured image of the authentication target T and the background B, etc., and an input unit 70 that inputs information on an IC card, etc.

[0013] The control unit 20 is composed of at least one processor such as a CPU (Central Processing Unit), and realizes the functions of each unit (image acquisition unit 21, feature point detection unit 22, object detection unit 23, discrimination unit 24, guidance unit 25, registration information authentication unit 26, face authentication unit 27, and correction unit 28) described below by executing programs and the like stored in the storage unit 30. The control unit 20 also has a clock (not shown) and can acquire the current date and time, count the elapsed time, and the like.

[0014] The storage unit 30 is composed of at least one memory such as a ROM (Read Only Memory) or a RAM (Random Access Memory), and a part or the whole of the ROM is composed of an electrically rewritable memory (such as a flash memory). The storage unit 30 functionally has an image storage unit 31, a feature point storage unit 32, a detection area storage unit 33, an object storage unit 34, an illegal condition storage unit 35, and a personal information storage unit 36. The ROM stores programs executed by the CPU of the control unit 20 and data required in advance to execute the programs. The RAM stores data that is created or changed during program execution.

[0015] In the image storage section 31, a first image P1, a second image P2, etc. captured by the imaging section 40 are stored.

[0016] The feature point memory unit 32 stores the feature points K1 and K2 detected by the feature point detection unit 22 together with the first image P1 (Figure 2(a)) and the second image P2 (Figure 2(b)), and after comparing the first image P1 and the second image P2, stores the third image P3 (Figure 2(c)).

[0017] The detection area storage unit 33 stores a detection area for detecting feature points K1 and K2 in an image captured by the imaging unit 40. The detection area can be set arbitrarily by the user. During face authentication, it is preferable to set the detection area so that areas that show people other than the authentication target T are not included in the detection area. Instead of setting a detection area, a non-detection area may be set in which detection of feature points within the area is not performed.

[0018] In the object storage section 34, a face image, which is an object detected from an image captured by the imaging section 40, is stored.

[0019] The fraudulent condition storage unit 35 stores fraudulent conditions that the image processing device 10 determines to be fraudulent authentication breakthroughs. For example, the fraudulent conditions stored include the feature points K2S (FIG. 3) constituting a rectangle surrounding the authentication target T described above, a facial expression recognition time described below, and a feature point disappearance zone V. The user can also arbitrarily set or add fraudulent conditions.

[0020] The personal information storage unit 36 ​​stores personal information of the IC card owner. For example, name, age, sex, department, face image, etc. are stored. When both IC card authentication and face authentication are performed, the face image in the captured image is compared with the face image stored in the personal information storage unit 36.

[0021] The imaging unit 40 includes an imaging device 41 that captures an image of the authentication object T and a background B, and a driving device 42 that changes the attitude of the imaging device 41.

[0022] The imaging device 41 includes a CMOS (Complementary Metal Oxide Semiconductor) camera. The imaging device 41 captures an image including the background B in its imaging range, and generates a first image P1, a second image P2, and so on.

[0023] The driving device 42 changes the attitude of the imaging device 41. This changes the imaging range of the imaging device 41, and the positions, number, etc. of the feature point K1 in the first image P1 and the feature point K2 in the second image P2 can be changed. In order to prevent a part of the feature point K1 in the first image P1 from being recognized by the authentication target T in the second image P2, while allowing a part of the feature point K1 to be recognized as the feature point K2 in the second image P2 even by the authentication target T, the imaging range of the imaging device 41 is set using the driving device 42 so that, for example, the ceiling is included above the first image P1 and the second image P2. The driving device 42 changes the attitude of the imaging device 41 to change the imaging range of the imaging device 41, and the positions, number, etc. of the feature point K1 in the first image P1 and the feature point K2 in the second image P2 can be changed.

[0024] The communication unit 50 has a communication device 51 which is a module for communicating with external devices such as an entrance / exit gate. The communication device 51 is a wireless module including an antenna when communicating with an external device. For example, the communication device 51 is a wireless module for performing short-range wireless communication based on Bluetooth (registered trademark). Through the communication unit 50, the image processing device 10 controls the entrance / exit gate and manages the security of the management target area.

[0025] The display unit 60 includes a display device 61 configured from a liquid crystal display panel (LCD: Liquid Crystal Display).

[0026] The display device 61 is configured with a thin film transistor (TFT), liquid crystal, organic EL, etc. The display device 61 displays a first image P1, a second image P2, a guidance image Y, etc.

[0027] The input unit 70 has an input device 71 for the authentication subject T to input various information to the image processing device 10. The input device 71 is an IC card reader that reads an IC card held by the authentication subject T. The input unit 70 also has, for example, a face authentication start button 72, an IC card authentication only button 73, and a keyboard that allows the authentication subject T to input information such as a password and an ID number instead of IC card authentication when the authentication subject T loses his / her IC card, etc.

[0028] Next, a description will be given of the functional configuration of the control unit 20 of the image processing device 10. The control unit 20 realizes the functions of an image acquisition unit 21, a feature point detection unit 22, an object detection unit 23, a discrimination unit 24, a guidance unit 25, a registration information authentication unit 26, a face authentication unit 27, and a correction unit 28, and performs face authentication entry / exit processing, which will be described later.

[0029] The image acquisition unit 21 causes the imaging unit 40 to capture an imaging range including the background B under conditions set in advance in the image processing device 10 or under conditions set by the user, and captures a first image P1, a second image P2, etc. The image acquisition unit 21 transmits the acquired first image P1 and second image P2 to the feature point detection unit 22.

[0030] The feature point detection unit 22 detects feature points K1 and K2 in the first image P1 and the second image P2 transmitted from the image acquisition unit 21. The feature point detection unit 22 stores the detected feature points K1 and K2 in the feature point storage unit 32.

[0031] The object detection unit 23 detects a person with a face equal to or larger than the detection size (80 pixels) within the detection area of ​​the captured image P2. The object detection unit 23 stores the detected face image in the object storage unit .

[0032] The discrimination unit 24 discriminates whether or not the face image detected by the object detection unit 23 is equal to or larger than the detection size for a set authentication confirmation time or more. Furthermore, the discrimination unit 24 compares the feature point K1 of the first image P1 with the feature point K2 of the second image P2.

[0033] The guidance unit 25 guides the authentication target T to move. As shown in FIG. 5(a), the guidance image Y including the center line L1 indicating the center line of the current face of the authentication target T, the face upper and lower lines L2 indicating the upper and lower ends of the face, and a frame E to be displayed at a position to which the face and body of the authentication target T are to be moved is displayed on the display unit 60 to guide the authentication target T. The guidance image Y is displayed by superimposing it on the second image P2 or by itself. For example, when the authentication target T is to be moved backward, as shown in FIG. 5(b), the authentication target T is guided by displaying the face upper and lower lines L2 at the current upper and lower ends of the face of the authentication target T and displaying the frame E at a position to which the authentication target T is to be moved. When the authentication target T is to be moved to the side, as shown in FIG. 5(c), the guidance is performed by displaying the center line L1 at the current center position of the face of the authentication target T and displaying the frame E at a position to which the authentication target T is to be moved. By guiding the authentication target T, the position, number, etc. of the feature points K2 in the second image P2 change from the state shown in FIG. 6(a) to that shown in FIG. 6(b) or (c). By comparing the feature points before and after the change and determining whether there is an unnatural change in the feature points (discontinuity in the change in the feature points of the authentication target T, no change in the feature points of the authentication target T regardless of whether the authentication target T approaches or moves away from the imaging unit 40, appearance of the feature points K2S shown in FIG. 3, etc.), it is possible to check with high accuracy whether an unauthorized authentication breakthrough is being attempted. Guidance by voice may be performed. The position to be moved may be indicated by the center line L1 and the upper and lower lines of the face L2, and the frame E may indicate the current position of the authentication target T.

[0034] In this embodiment, the registration information authentication unit 26 authenticates the IC card.

[0035] The face authentication unit 27 performs face authentication by comparing feature points in the face image detected from the second image P2 to determine whether or not there is a face image that can be regarded as the same as the face image stored in the personal information storage unit 36. The face authentication unit 27 can perform highly accurate AI (Artificial Intelligence) face authentication by machine learning.

[0036] The correction unit 28 compares the feature points K1 of the first image P1 with the feature points K2 of the second image P2, and corrects the names of the feature points K1 in the first image P1 that are not detected at positions corresponding to the feature points K1 in the second image P2 to feature points K1H, thereby creating a third image P3 (Figure 2(c)) and storing the result in the feature point memory unit 32.

[0037] This concludes the description of the functional configuration of the control unit 20. Hereinafter, the face authentication entry / exit process performed in the image processing device 10 will be described with reference to a flowchart.

[0038] (Facial recognition entry / exit processing) The flow of the face authentication entry / exit process will be described with reference to FIG. 7. The face authentication entry / exit process can prevent unauthorized breakthrough of face authentication. The image processing device 10 performs entry / exit management by combining face authentication and IC card authentication. There are combinations of authentication using either face authentication or IC card authentication, and authentication using both face authentication and IC card authentication. If authentication using both face authentication and IC card authentication is cleared, permission level 3 authority that allows entry to all managed areas is obtained. If authentication using only face authentication is cleared, permission level 2 authority that allows entry to areas except for some areas is obtained. If authentication using only IC card authentication is cleared, permission level 1 authority that has a wider range of inaccessible areas than permission level 2 is obtained. That is, the authentication subject T is given authority according to the type of authentication that has succeeded. While it is possible for an IC card to be used by someone other than the person himself / herself, in the face authentication of the present application, it is difficult to imagine a breakthrough of authentication by someone other than the person himself / herself, so permission level 2 using only face authentication is more reliable and is set higher than permission level 1 using only IC card authentication.

[0039] First, the control unit 20 determines whether or not the IC card has been read by the input device 71 (step S1). If the IC card has been read (step S1: Yes), the process proceeds to step S5.

[0040] If the IC card has not been read (step S1: No), the process proceeds to step S2.

[0041] In step S2, the control unit 20 determines whether or not the face authentication start button 72 has been pressed. If the face authentication start button 72 has been pressed (step S2: Yes), the control unit 20 proceeds to step 6. If the face authentication start button 72 has not been pressed (step S2: No), the control unit 20 proceeds to step S3.

[0042] In step S3, the control unit 20 executes a human detection process to determine whether or not the authentication object T is captured by the imaging unit 40 for a period of time equal to or greater than the detection size and equal to or greater than the authentication confirmation time. The human detection process will be described in detail later. After the human detection process is executed, the process proceeds to step S4.

[0043] In step S4, the control unit 20 determines whether or not a person has been detected in the human detection process in step S3. If a person has been detected in the human detection process (step S4: Yes), the process proceeds to step S6, and if a person has not been detected (step S4: No), the process returns to step S1.

[0044] In step S5, the registration information authentication unit 26 determines whether or not the information on the IC card matches the information stored in the personal information storage unit 36. If the information on the IC card matches the information stored in the personal information storage unit 36 ​​(step S5: Yes), the process proceeds to step S6, and if the information on the IC card does not match the information stored in the personal information storage unit 36 ​​(step S5: No), face authentication is not permitted (step S14), and the face authentication entry / exit process ends.

[0045] In step S6, the control unit 20 executes a face authentication process to prevent unauthorized access to face authentication. The face authentication process will be described in detail later. After executing the face authentication process, the process proceeds to step S7.

[0046] In step S7, the control unit 20 determines whether person recognition has been performed in the face recognition process in step S6 (comparison of feature points has been performed in person recognition in step S33). If person recognition has been performed in step S6 (step S7: Yes), the process proceeds to step S8, and if person recognition has not been performed in step S6 (step S7: No), the process proceeds to step S10.

[0047] In step S8, the control unit 20 determines whether the face authentication process in step S6 ended with authentication (step S38) or non-authentication (step S39). If the face authentication process ended in step S6 (step S8: Yes), the process proceeds to step S9. If the face authentication process ended with non-authentication (step S8: No), the face authentication is not permitted (step S14), and the face authentication entry / exit process ends.

[0048] In step S9, the control unit 20 determines whether or not an IC card was used in step S1. If an IC card was used (step S9: Yes), permission of permission level 3 is issued (step S11) and the process ends. If an IC card was not used (step S9: No), permission of permission level 2 is issued (step S12) and the process ends.

[0049] In step S10, it is determined whether or not an IC card has been used. If an IC card has been used (step S10: Yes), permission of permission level 1 is issued (step S13) and the process ends. If an IC card has not been used (step S10: No), permission is denied (step S14) and the process ends.

[0050] The face authentication entry / exit process has been described above. Next, the human detection process performed in step S3 of the face authentication entry / exit process will be described with reference to FIG.

[0051] First, the image acquisition unit 21 acquires the first image P1 or the second image P2 captured by the imaging unit 40 (step S21).

[0052] Next, the discrimination unit 24 determines whether or not a person with a face larger than the detection size has been detected within the detection area of ​​the image P2 captured by the object detection unit 23 (step S22). If a person with a face larger than the detection size has been detected (step S22: Yes), the discrimination unit 24 proceeds to step S23, and if a person with a face larger than the detection size has not been detected (step S22: No), the discrimination unit 24 ends the person detection process.

[0053] In step S23, the discrimination unit 24 discriminates whether the person detected in step S22 was equal to or larger than the detection size for the authentication confirmation time or longer. If the person detected in step S22 was equal to or larger than the detection size for the authentication confirmation time or longer (step S23: Yes), it is determined that a person has been detected (step S24) and the process ends. If the person detected in step S22 was equal to or larger than the detection size for less than the authentication confirmation time (step S23: No), the process returns to step S21.

[0054] The human detection process has been described above. Next, the face authentication process performed in step S6 of the face authentication entry / exit process will be described with reference to FIG.

[0055] First, the image acquisition unit 21 acquires the first image P1 or the second image P2 captured by the imaging unit 40 (step S31).

[0056] Proceeding to step S32, if the authentication subject T wishes to perform only IC card authentication, he / she presses the IC card authentication only button 73 provided on the input unit 70 (step S32: Yes), and the face authentication process ends. If face authentication is to be performed, the authentication subject waits without pressing the IC card authentication only button 73 provided on the input unit 70 (step S32: No), proceeds to step S33, and person recognition is performed.

[0057] Proceeding to step S33, the discrimination unit 24 performs human recognition on the acquired first image P1 or second image P2. Specifically, the feature point detection unit 22 detects and compares feature points of the first image P1 or the second image P2, so that the discrimination unit 24 determines, for example, whether the authentication target T exists in the background B, and whether the feature point K2S constituting a rectangle surrounding the authentication target T as illustrated in FIG. 3 exists in the second image P2. If the authentication target T exists and the feature point K2S does not exist (step S33: Yes), the discrimination unit 24 determines that the authentication target T is a person, and proceeds to steps S34 and S35. If the authentication target T does not exist, or if the authentication target T exists and the feature point K2S exists (step S33: No), the discrimination unit 24 determines that the authentication target T does not exist, or that the authentication target T is not a person and an illegal authentication breakthrough has been attempted, and does not perform face authentication judgment, but determines an abnormality (step S41), and ends the face authentication process. When it is difficult to detect the characteristic point K2 of the authentication object T, the guiding unit 25 performs guidance to encourage the authentication object T to move or the like.

[0058] Steps S34 and S35 are processed in parallel as shown in Fig. 10. In step S34, facial expression recognition is performed, and in step S35, AI face authentication is performed.

[0059] In the facial expression recognition, after the object detection unit 23 detects a face image in the first image P1 or the second image P2, the feature point detection unit 22 detects the feature points KF in the face image during the facial expression recognition time. In this embodiment, the facial expression recognition time is set to 5 seconds. The facial expression recognition time is set to a time sufficient for blinking, movement of the mouth, etc. to occur. The discrimination unit 24 observes the feature points KF during the facial expression recognition time, and if there is a change in the position, number, etc. of the feature points KF, it determines that there is a change in the facial expression in the face image (the face image is a human face captured by the imaging unit 40), and ends the facial expression recognition as OK. If there is no change in the position, number, etc. of the feature points KF during the facial expression recognition time, it determines that there is no change in the facial expression in the face image (the face image is an image prepared in advance, etc.), and ends the facial expression recognition as NG. Note that if a face image is not detected, the facial expression recognition is also ended as NG.

[0060] In AI face recognition, the face recognition unit 27 performs face recognition by comparing feature points in the face image detected by the object detection unit 23 in the first image P1 or the second image P2 with face images stored in the personal information storage unit 36 ​​to determine whether there is any face image that can be considered to be the same as the face image detected by the object detection unit 23 in the first image P1 or the second image P2, among the face images stored in the personal information storage unit 36. In particular, since there may be differences in the eyes, mouth, etc., even if it is the same person, the face image in the captured image and the face image stored in the personal information storage unit 36 ​​are considered to be the same if there are a certain number (or a certain ratio) or more of the feature points KF in the face image, and the face image in the captured image and the face image stored in the personal information storage unit 36 ​​are considered to be the same, and the face recognition is determined to be OK, and if there are less than a certain number (or a certain ratio) of the same points, the face recognition is determined to be NG.

[0061] After the facial expression recognition and AI facial recognition are processed in parallel, the process proceeds to step S36. If the facial expression recognition is OK (step S36: Yes), the discrimination unit 24 proceeds to step S37. If the facial expression recognition is not OK (step S36: No), the discrimination unit 24 judges that there is an abnormality (step S41) and terminates the facial recognition process.

[0062] In step S37, if there is no error in the facial authentication (step S37: No), the discrimination unit 24 proceeds to step S38. If there is an error in the facial authentication (step S37: Yes), an error is displayed on the display unit 60 (step S42), and the process returns to step S31. The error display includes the reason why the facial authentication could not be performed, such as wearing sunglasses, a hat, a mask, or insufficient illuminance, and prompts the authentication subject T to take action. Furthermore, if level 1 permission is sufficient, the authentication subject T can also change to IC card authentication only in step S32.

[0063] In step S38, if face authentication is successful (step S38: Yes), the discrimination unit 24 displays the result that the face authentication was successful (step S39) on the display unit 60 and ends the face authentication process. If face authentication is unsuccessful (step S38: No), the discrimination unit 24 displays the result that the face authentication was unsuccessful (step S40) on the display unit 60 and ends the face authentication process.

[0064] As described above, by the facial recognition entry / exit process, the image processing device 10 can detect the feature points K2S constituting the corners of the rectangle surrounding the authentication target T in the second image P2 that can also be used for facial recognition when a fraudulent attempt is made using a video or the like on a tablet, thereby preventing fraudulent authentication breakthrough. In addition, by performing facial recognition, the image processing device 10 can prevent fraudulent authentication breakthrough using a still image, poster photo, doll, or the like that has been prepared in advance by observing the feature points KF during the facial recognition time. Furthermore, by processing facial recognition and AI facial recognition in parallel, an increase in processing time can be prevented.

[0065] (Modification) In the above embodiment, the image processing device 10 determines that an unauthorized authentication breakthrough has been attempted when the feature point K2S constituting the corner of a rectangle is detected in the person recognition in step S32. However, instead of the feature point K2S, for example, as shown by the broken line in FIG. 11, the image processing device 10 may determine that an unauthorized authentication breakthrough has been attempted by confirming the presence of a feature point disappearance zone V of a certain range around the authentication target T. In other words, the image processing device 10 may determine that an unauthorized authentication breakthrough has been attempted when a feature point is not detected in the second image P2 at a position corresponding to the feature point K1 of the first image P1 within a range surrounded by the rectangular feature point K2S. This is effective when the feature point K2S cannot be detected due to the amount of light, the color of the background, etc. Alternatively, the image processing device 10 may determine that an unauthorized authentication breakthrough has been attempted by identifying a feature point KT near the top of the head (identifying a feature point KN indicating the nose and identifying a feature point above it), and determining that an unauthorized authentication breakthrough has been attempted when, for example, a feature point corresponding to the feature point K1 5 pixels or more above the feature point KT is not found in the second image P2. Alternatively, the authentication target T may be determined to be a human when the feature point K1 of the first image P1 and the feature point K2 of the second image P2 shown in Figures 2(a) and 2(b) are compared and only the feature point at a position corresponding to the feature point K1H shown in Figure 2(c) among the feature points K1 of the first image P1 cannot be confirmed in Figure 2(b). Also, for example, when the feature point K2S constituting the corner of a rectangle is detected, a condition may be set to exclude feature points within a range surrounded by the feature points K2S constituting the corner of the rectangle, thereby preventing the feature points detected on the screen of a smartphone, tablet, etc. from accidentally matching the feature point K1 of the first image P1.

[0066] In the above embodiment, the image processing device 10 detects people with faces of 80 pixels or more, but the detection target may be determined by, for example, an eye distance of 32 pixels or more, or by other criteria.

[0067] In the above embodiment, the facial expression recognition time in the image processing device 10 is set to 5 seconds, but it may be set to 3 seconds, 10 seconds, or another time.

[0068] In the above embodiment, the image processing device 10 includes the imaging unit 40, the communication unit 50, the display unit 60, the input unit 70, etc. However, instead of including these, the image processing device 10 may be substituted by connecting to an external device such as a camera, a tablet, an IC card reader, etc. Alternatively, for example, the display unit and the input unit may be integrally configured.

[0069] In the above embodiment, the image processing device 10 processes facial expression recognition and face authentication in parallel, but person recognition and face authentication may be processed in parallel, or face authentication may be performed by an external device connected to the image processing device 10. Alternatively, as shown in Fig. 12, after the facial expression recognition and face authentication are processed in parallel (even if face authentication has not yet ended), person recognition may be performed again to improve the accuracy of fraud prevention.

[0070] In the above embodiment, the image processing device 10 moves the authentication target T using a guidance image Y formed by a center line L1, upper and lower lines of the face L2, and a frame E, but it may also be possible to guide the target T using a guidance image such as an arrow, a video, etc., or an image that combines an arrow, a video, etc. with the guidance image Y.

[0071] In addition, in the above embodiment, the image processing device 10 changes the position and number of feature points by moving the authentication object T using the guiding image Y, but the position and number of feature points may be changed by changing the angle of view of the imaging device 41 using the driving device 42, or the guiding image Y and the driving device 42 may be used in combination.

[0072] In the above embodiment, the image processing device 10 performs facial expression recognition, but may also perform pulse rate recognition using an image, for example.

[0073] In the above embodiment, the image processing device 10 also uses IC card authentication. However, instead of IC card authentication, authentication using a smartphone, a token, or the like, or authentication using information such as a PIN number, a password, or a security question may also be used.

[0074] In the above embodiment, the image processing device 10 performs AI face recognition, but may perform other face recognition such as face recognition using an infrared sensor.

[0075] Each function of the image processing device 10 of the present invention can be implemented by a computer such as a normal PC (Personal Computer) or tablet. Specifically, in the above embodiment, the programs for the face recognition entry / exit processing, human detection processing, and face recognition processing performed by the image processing device 10 have been described as being stored in advance in the ROM of the storage unit 30. However, the programs may be stored in a computer-readable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), a DVD (Digital Versatile Disc), or an MO (Magneto-Optical Disc) and distributed, and the programs may be read and installed in a computer to configure a computer capable of implementing each of the above-mentioned functions.

[0076] Although the preferred embodiment of the present invention has been described above, the present invention is not limited to the specific embodiment, and the present invention includes the inventions described in the claims and their equivalents. The inventions described in the original claims of this application are listed below.

[0077] (Additional Note) (Appendix 1) a feature point detection unit that detects feature points of a first image showing a background and feature points of a second image showing an object to be authenticated against the background; a discrimination unit that discriminates whether or not an attempt has been made to break through the authentication by comparing a feature point of the first image with a feature point of the second image; An image processing device comprising:

[0078] (Appendix 2) a guiding unit that guides the authentication target to move to a position where the authentication target overlaps with at least one of the feature points of the first image, 2. An image processing device as described in claim 1.

[0079] (Appendix 3) A display unit for displaying the second image is further provided, The guidance is performed by displaying, on the display unit, a guidance image indicating a direction in which the authentication target is to be moved together with the second image. 3. An image processing device as described in claim 2.

[0080] (Appendix 4) if rectangular feature points surrounding the authentication target are detected in the second image, or if feature points are not detected in the second image at positions corresponding to the feature points of the first image within the range surrounded by the rectangular feature points, it is determined that there has been an attempt to fraudulently break through the authentication. 2. An image processing device as described in claim 1.

[0081] (Appendix 5) the discrimination unit compares feature points of each of the two or more second images acquired, and performs facial expression recognition of the authentication target. 5. An image processing device according to any one of claims 1 to 4.

[0082] (Appendix 6) A face authentication unit that performs face authentication simultaneously with the discrimination or the facial expression recognition, 6. The image processing device according to claim 5.

[0083] (Appendix 7) An imaging unit that captures the first image and the second image, changing the number or positions of feature points in the second image by changing the angle of view of the imaging unit; 2. An image processing device as described in claim 1.

[0084] (Appendix 8) combining authentication with something or information possessed by the subject, thereby authorizing the subject according to the type of successful authentication; 2. An image processing device as described in claim 1.

[0085] (Appendix 9) If it is determined that there is no attempt to break through the authentication, and the authentication target is successfully authenticated by the thing or information it possesses, the authentication target is given the highest authority; If it is determined that there is no attempt to break through the authentication, and the authentication subject has not performed authentication by possession or information, the authentication subject is given the next highest authority; If the presence or absence of an attempt to break through the authentication is not determined and the authentication subject is successfully authenticated based on the possession or information of the authentication subject, the authentication subject is given the lowest privilege. 9. The image processing device according to claim 8.

[0086] (Appendix 10) Detecting feature points of a first image showing a background and feature points of a second image showing a target object against the background; determining whether or not there is an attempt to break through the authentication by comparing the feature points of the first image with the feature points of the second image; Image processing methods.

[0087] (Appendix 11) On the computer, Detecting feature points of a first image showing a background and feature points of a second image showing a target object against the background; determining whether or not there is an attempt to break through the authentication by comparing the feature points of the first image with the feature points of the second image; A program to make it function like this. [Explanation of symbols]

[0088] 10...image processing device, 20...control unit, 21...image acquisition unit, 22...feature point detection unit, 23...object detection unit, 24...discrimination unit, 25...guidance unit, 26...registration information authentication unit, 27...face authentication unit, 28...correction unit, 30...memory unit, 31...image memory unit, 32...feature point memory unit, 33...detection area memory unit, 34...object memory unit, 35...incorrect condition memory unit, 36...personal information memory unit, 40...imaging unit, 41...imaging device, 42... Drive device, 50... communication unit, 51... communication device, 60... display unit, 61... display device, 70... input unit, 71... input device, 72... face recognition start button, 73... IC card recognition only button, B... background, C... body, E... frame, F... face, L1... center line, L2... upper and lower lines of face, K1, K2, KF... feature points, P1... first image, P2... second image, P3... third image, T... authentication target, V... feature point disappearance zone, Y... guidance image

Claims

1. A feature point detection unit that detects feature points of a first image reflecting a background and feature points of a second image in which an authentication target is reflected in the background; A determination unit that determines whether there is an attempt to illegally break through authentication by comparing the feature points of the first image and the feature points of the second image; An image processing apparatus comprising the above.

2. The image processing apparatus according to claim 1, further comprising a guiding unit that guides the authentication target to move to a position overlapping at least one of the feature points of the first image. The image processing apparatus according to claim 1.

3. The image processing apparatus according to claim 2, further comprising a display unit that displays the second image, and performs the guiding by displaying a guiding image indicating the direction in which the authentication target moves together with the second image on the display unit. The image processing apparatus according to claim 2.

4. The determination unit determines whether there is an attempt to illegally break through authentication by comparing the feature points in the first image and the feature points of the face region of the authentication target in the second image obtained after the movement of the authentication target is performed by the guidance of the guiding unit. The image processing apparatus according to claim 3.

5. When rectangular feature points surrounding the authentication target are detected in the second image, or when no feature points are detected at positions corresponding to the feature points of the first image within the range surrounded by the rectangular feature points in the second image, it is determined that there is an attempt to illegally break through authentication. The image processing apparatus according to claim 1.

6. The determination unit compares the feature points of each of the second images obtained from two or more of the second images, and performs expression recognition of the authentication target. The image processing apparatus according to any one of claims 1 to 5.

7. The image processing apparatus according to claim 6, further comprising a face authentication unit that performs face authentication simultaneously with the determination or the expression recognition. The image processing apparatus according to claim 6.

8. The image processing apparatus according to claim 1, further comprising an imaging unit that images the first image and the second image, and changes the number or position of the feature points of the second image by changing the imaging angle of the imaging unit. The image processing apparatus according to claim 1.

9. Permissions are given to the authentication target according to the type of successful authentication by combining authentication using something or information owned by the authentication target. The image processing apparatus according to claim 1.

10. The image processing apparatus according to claim 1.

10. When it is determined that there is no attempt to break through the unauthorized authentication and the authentication of the authentication target by what or information the authentication target owns is successful, the highest authority is given to the authentication target. When it is determined that there is no attempt to break through the unauthorized authentication and the authentication of the authentication target by what or information the authentication target owns is not performed, the next highest authority is given to the authentication target. When the presence or absence of an attempt to break through the unauthorized authentication is not determined and the authentication of the authentication target by what or information the authentication target owns is successful, the lowest authority is given to the authentication target. The image processing apparatus according to claim 9.

11. Detect a feature point of a first image showing a background and a feature point of a second image showing an authentication target in the background. By comparing the feature point of the first image and the feature point of the second image, determine whether there is an attempt to break through the unauthorized authentication. Image processing method.

12. On a computer, Detect a feature point of a first image showing a background and a feature point of a second image showing an authentication target in the background. By comparing the feature point of the first image and the feature point of the second image, determine whether there is an attempt to break through the unauthorized authentication. A program for causing the computer to function as described above.

Images