Terminal, method, and program

Abstract

To provide a terminal, method and program, which enable more accurate determination as to whether a target object is an organism or not.SOLUTION: A terminal disclosed herein comprises a display unit capable of displaying a first guidance (screen 100-1) for guiding a target person to be imaged by an image capturing unit to maintain his / her line-of-sight, a second guidance (screen 100-2) for guiding the target person to maintain the orientation of the face, and a third guidance (screen 100-3) for guiding the target person to move the terminal equipped with image capturing means.SELECTED DRAWING: Figure 22

Description

[Technical field]

[0001] The present disclosure relates to liveness detection using images. [Background technology]

[0002] Technology has been developed to detect liveness using cameras. Liveness detection here means determining whether an object captured by a camera is a living body or not. For example, liveness detection is used to prevent spoofing using photographs.

[0003] An example of a prior art document related to live body detection using a camera is Patent Document 1. The system in Patent Document 1 discloses a technology for controlling lighting to change the lighting environment, acquiring images of a person captured in each of different lighting environments, and using these images to determine whether or not the captured object is a person. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2003-178306 A Summary of the Invention [Problem to be solved by the invention]

[0005] This disclosure aims to improve upon the technology disclosed in Patent Document 1. [Means for solving the problem]

[0006] The disclosed terminal is equipped with a display capable of displaying a first guide that guides the user to maintain a line of sight with respect to a subject imaged by an imaging unit, a second guide that guides the user to maintain the orientation of the subject's face with respect to the subject, and a third guide that guides the user to move the terminal equipped with the imaging means with respect to the subject.

[0007] The disclosed method is executed by a computer and includes displaying on a display of a terminal a first guide to maintain a line of sight with respect to a subject captured by an imaging unit, displaying on the display a second guide to maintain a facial orientation of the subject with respect to the subject, and displaying on the display a third guide to move the terminal equipped with the imaging means with respect to the subject.

[0008] The program disclosed herein causes a computer to display on a display of a terminal a first guide that guides the user to maintain a line of sight toward a subject captured by an imaging unit, to display on the display a second guide that guides the user to maintain the orientation of the subject's face relative to the subject, and to display on the display a third guide that guides the user to move the terminal equipped with the imaging means relative to the subject. [Brief description of the drawings]

[0009] [Figure 1] 4 is a diagram illustrating an example of an outline of the operation of the biological detection device of the first embodiment. FIG. [Diagram 2] 2 is a block diagram illustrating a functional configuration of the biological detection device according to the first embodiment. FIG. [Diagram 3] FIG. 2 is a block diagram illustrating a hardware configuration of a computer that realizes the biological detection device of the first embodiment. [Figure 4] 4 is a flowchart illustrating a flow of processing executed by the biological detection device of the first embodiment. [Diagram 5] FIG. 13 is a diagram illustrating an example of a screen composed of two areas. [Figure 6] FIG. 13 is a diagram illustrating a screen having three regions. [Figure 7] FIG. 13 is a diagram illustrating a case where the screen is divided both horizontally and vertically. [Figure 8] FIG. 13 is a diagram illustrating a case where a boundary line on a screen is determined based on the position of a camera. [Figure 9]1 is a first diagram illustrating a plurality of screens displayed on the display device by a display control unit. FIG. [Figure 10] 11 is a second diagram illustrating a plurality of screens displayed on the display device by the display control unit. FIG. [Figure 11] FIG. 13 illustrates a guide showing appropriate face sizes. [Figure 12] 13 is a diagram illustrating an example in which the face direction is moved left and right while the line of sight is fixed; FIG. [Figure 13] 11A and 11B are diagrams illustrating examples of changes over time in face direction and gaze direction when the face is turned from side to side while the gaze direction is fixed. [Figure 14] FIG. 11 is a first diagram illustrating an example of guide information. [Figure 15] FIG. 11 is a block diagram illustrating a functional configuration of a biological detection device according to a second embodiment. [Figure 16] 10 is a flowchart illustrating a flow of processing executed by a biological detection device according to a second embodiment. [Figure 17] 11 is a diagram illustrating an example of guide information indicating a next action that a target object should take. [Figure 18] 13A and 13B are diagrams illustrating an example of an animation of a guide indicating a face direction. [Figure 19] 11 is a diagram illustrating an example of guide information including the gaze position and face direction of a target object. [Figure 20] 13 is a second diagram illustrating an example of guide information including the gaze position and face direction of a target object. FIG. [Figure 21] 13A to 13C are diagrams illustrating an example of the movement of a target object in a living body detection device according to the first modification of the second embodiment. [Figure 22] 13 is a diagram illustrating an example of guide information in a biological detection device according to a first modified example of the second embodiment. FIG. [Diagram 23] 13A to 13C are diagrams illustrating an example of the movement of a target object in a living body detection device according to Modification 2 of Embodiment 2. [Figure 24] 13 is a diagram illustrating an example of guide information in a biological detection device according to a second modification of the second embodiment. FIG. [Diagram 25]10 is a flowchart illustrating a process flow when live body detection by the live body detection device of the first embodiment and live body detection by the live body detection device of the second embodiment are performed in sequence. [Figure 26] 11 is a diagram for explaining the direction of a face of a target object on a vertical plane. FIG. [Figure 27] 13 is a diagram illustrating an example of a guide for ensuring that the terminal is in an appropriate position. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are given the same reference numerals, and duplicated explanations will be omitted as necessary for clarity of explanation. In addition, in the drawings, filling in colors other than white is expressed using patterns such as dot patterns, unless otherwise specified. In addition, various predetermined values ​​(threshold values, etc.) are assumed to be stored in advance in a storage device, unless otherwise specified.

[0011] [Embodiment 1] <Summary> Fig. 1 is a diagram illustrating an example of an outline of the operation of the living body detection device 2000 according to the first embodiment. Here, Fig. 1 is a diagram for facilitating understanding of the outline of the living body detection device 2000, and the operation of the living body detection device 2000 is not limited to that shown in Fig. 1.

[0012] The live object detection device 2000 determines whether or not the target object 50 captured by the camera 10 is a living organism (i.e., live object detection). For example, the live object detection device 2000 distinguishes between a case where the target object 50 is a real person and a case where the target object 50 is a non-living organism such as a photograph of a person (i.e., a case where a photograph of a person is captured instead of a real person). However, the target of live object detection is not limited to a person, and may be other animals such as a dog or a snake, or an inanimate object such as a robot. In the following disclosure, a case where a person is the target of live object detection is exemplified, but a similar method can also be applied to a case where an inanimate object such as a non-human animal or a robot is the target of live object detection.

[0013] For this purpose, the living body detection device 2000 controls the output of a screen by the display device 20. For example, the display device 20 is provided in a terminal 60 (such as a smartphone) used by a user. Specifically, the living body detection device 2000 causes the display device 20 to display a plurality of screens 30 at different times. For example, in the example of Fig. 1, the living body detection device 2000 causes the display device 20 to display a screen 30-1, and then changes the screen displayed on the display device 20 from the screen 30-1 to the screen 30-2.

[0014] Here, at least one screen 30 has a plurality of regions 32. Among the plurality of regions 32, there are at least two regions 32 that include different displays. For example, the screen 30-1 in Fig. 1 has a region 32-1 filled with white and a region 32-2 filled with black. For convenience of illustration, the black is expressed by a dot pattern.

[0015] The camera 10 captures an image of the target object 50 for each of at least two screens 30 while that screen 30 is being displayed on the display device 20, and generates a captured image 40 representing the result. For example, in Fig. 1, a captured image 40-1 is generated by capturing an image while the screen 30-1 is being displayed, and a captured image 40-2 is generated by capturing an image while the screen 30-2 is being displayed.

[0016] The living body detection device 2000 acquires a plurality of captured images 40 thus generated, and uses the acquired plurality of captured images 40 to determine whether or not the target object 50 is a living body.

[0017] <Example of action and effect> According to the living body detection device 2000 of this embodiment, an image is captured by the camera 10 while the screen 30 is displayed on the display device 20. Here, when the target object 50 is a paper photograph showing a face, a display device displaying an image of a face, or the like, the photograph is irradiated with light from the display device 20, and the reflected light is imaged by the camera 10. On the other hand, when the target object 50 is a real person, the real face is irradiated with light from the display device 20, and the reflected light is imaged by the camera 10.

[0018] The shapes of paper, display devices, etc. are approximately flat and are significantly different from the shapes of real faces. Therefore, it can be said that the characteristics of the reflected light are significantly different when the light emitted from the display device 20 is irradiated onto a paper photograph or the like and when the light is irradiated onto a real face. Such differences in the characteristics of the reflected light appear in the captured image 40 obtained by capturing the reflected light with the camera 10.

[0019] According to the living body detection device 2000, by analyzing the captured image 40 obtained by capturing the reflected light, it is possible to determine whether or not the target object 50 is a living body based on such differences in characteristics.

[0020] In addition, in the system of Patent Document 1, light is irradiated from one illuminating lamp to the target of the live body detection. Therefore, only limited changes in the lighting environment can be realized, such as "illuminating the entire screen brightly" or "illuminating the entire screen dimly". In this regard, in the live body detection device 2000 of the present embodiment, the screen 30 includes a plurality of areas 32 whose displays are different from each other. Therefore, the pattern of light irradiated from the display device 20 to the target object 50 can be flexibly set. For example, as shown in FIG. 1, the target object 50 can be irradiated with various patterns of light, such as "the left half is bright and the right half is dark" or "the right half is bright and the left half is dark".

[0021] According to the living body detection device 2000 of this embodiment, the target object 50 is irradiated with such various patterns of light, and the reflected light is captured to determine whether or not the target object 50 is a living body by using the captured image 40. Therefore, it is possible to more accurately determine whether or not the target object 50 is a living body, compared to a case where only light with a relatively simple pattern, such as "illuminating the entire screen brightly" or "illuminating the entire screen darkly," can be irradiated with the target object 50.

[0022] The biological detection device 2000 of this embodiment will be described in more detail below.

[0023] <Example of functional configuration> 2 is a block diagram illustrating a functional configuration of the living body detection device 2000 of the first embodiment. The living body detection device 2000 includes a display control unit 2020, an acquisition unit 2040, and a living body detection unit 2060. The display control unit 2020 causes the display device 20 to display a plurality of screens 30 at different times. Here, at least one screen 30 has a plurality of areas 32. Also, at least two areas 32 include different displays. The acquisition unit 2040 acquires a plurality of captured images 40 generated by the camera 10 while each of the plurality of screens 30 is being displayed. The living body detection unit 2060 uses the plurality of captured images 40 to determine whether or not the target object 50 is a living body.

[0024] <Example of hardware configuration> Each functional component of the living body detection device 2000 may be realized by hardware that realizes each functional component (e.g., a hardwired electronic circuit, etc.), or may be realized by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it, etc.). Below, a further description will be given of the case where each functional component of the living body detection device 2000 is realized by a combination of hardware and software.

[0025] 3 is a block diagram illustrating a hardware configuration of a computer 500 that realizes the biological detection device 2000. The computer 500 is any computer. For example, the computer 500 is a stationary computer such as a PC (Personal Computer) or a server machine. In addition, for example, the computer 500 is a portable computer such as a smartphone, a tablet terminal, or a notebook PC. In addition, when a terminal 60 is used, the biological detection device 2000 may be provided integrally with the terminal 60 or may be provided separately.

[0026] The computer 500 may be a dedicated computer designed to realize the living body detection device 2000, or may be a general-purpose computer. For example, by installing a predetermined application on the computer 500, each function of the living body detection device 2000 is realized by the computer 500. The application is configured by a program for realizing the functional components of the living body detection device 2000.

[0027] The computer 500 has a bus 502, a processor 504, a memory 506, a storage device 508, an input / output interface 510, and a network interface 512. The bus 502 is a data transmission path for the processor 504, the memory 506, the storage device 508, the input / output interface 510, and the network interface 512 to transmit and receive data to and from each other. However, the method of connecting the processor 504 and other components to each other is not limited to a bus connection.

[0028] The processor 504 is a processor such as a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a digital signal processor (DSP), or an application specific integrated circuit (ASIC). The memory 506 is a main storage device realized using a random access memory (RAM) or the like. The storage device 508 is an auxiliary storage device realized using a hard disk, a solid state drive (SSD), a memory card, or a read only memory (ROM) or the like.

[0029] The input / output interface 510 is an interface for connecting the computer 500 to an input / output device. For example, the input / output interface 510 is connected to the camera 10 and the display device 20.

[0030] The network interface 512 is an interface for connecting the computer 500 to a network. This network may be a LAN (Local Area Network) or a WAN (Wide Area Network).

[0031] The storage device 508 stores a program (a program that realizes the above-mentioned application) that realizes each functional component of the living body detection device 2000. The processor 504 realizes each functional component of the living body detection device 2000 by reading the program into the memory 506 and executing it.

[0032] The biological detection device 2000 may be realized by one computer 500, or may be realized by multiple computers 500. In the latter case, the configurations of the computers 500 do not need to be the same, and may be different from each other.

[0033] <Regarding the display device 20> The display device 20 is any display device capable of displaying the screen 30. For example, the display device 20 is communicably connected to a terminal 60 operated by a user, and is configured to display a screen output from the terminal 60. The terminal 60 is any computer used by a user. When the target object 50 is a real person and the terminal 60 is used by the target object 50, the user of the terminal 60 is the target object 50. On the other hand, when impersonation is attempted using a photograph or the like as the target object 50, the user of the terminal 60 is a person who has the camera 10 capture such a photograph or the like. The terminal 60 may be realized by a portable computer such as a smartphone, a tablet terminal, or a notebook PC, or may be realized by a stationary computer such as a desktop PC.

[0034] When the terminal 60 is realized by a portable computer, the display device 20 is a display device provided in the terminal 60. On the other hand, when the terminal 60 is realized by a stationary computer, the display device 20 is a display device that is communicatively connected to the terminal 60 and configured to display a screen output from the terminal 60.

[0035] The living body detection device 2000 may be provided integrally with or separately from the terminal 60. When the terminal 60 is provided separately from the living body detection device 2000, the hardware configuration of the computer that realizes the terminal 60 is, for example, as shown in FIG. 3, similar to the computer 500.

[0036] Also, some of the functions of the living body detection device 2000 may be provided in the terminal 60, and the remaining functions may be provided in a computer other than the terminal 60. For example, the display control unit 2020 and the acquisition unit 2040 are provided in the terminal 60, and the living body detection unit 2060 is provided in a server device. In this case, for example, a plurality of captured images 40 generated by the camera 10 are transmitted from the terminal 60 to the server device. Then, the server device performs living body detection by analyzing the received captured images 40, and transmits information indicating the result to the terminal 60.

[0037] The display device 20 is not limited to one that is communicably connected to a terminal 60 operated by a user. For example, the display device 20 can be realized as a display device that can be viewed by a person passing through a gate at the entrance of a facility where the opening and closing of the gate is controlled by a computer. In this case, for example, face authentication and liveness detection using the camera 10 are performed on a person who is about to pass through the gate. Specifically, a captured image of the target object 50 obtained from the camera 10 is used to determine whether the target object 50 is registered as a person who can pass through the gate. Furthermore, the liveness detection device 2000 controls the display device 20 to perform liveness detection on the target object 50. Then, if the target object 50 is registered as a person who can pass through the gate and is determined to be a live body, the gate is opened by a gate control device that controls the opening and closing of the gate. On the other hand, if the target object 50 is not registered as a person who can pass through the gate or if the target object 50 is determined to be a non-live body, the gate is not opened (if the gate is open, the gate is closed by the gate control device). By performing liveness detection, it is possible to prevent impersonation using a facial photo of a registered person.

[0038] In addition, in a case where the opening and closing of the gate is controlled based on the results of face authentication and liveness detection, face authentication and liveness detection may be performed using a terminal 60 such as a smartphone used by a user. Specifically, first, the user operates the terminal 60 to perform face authentication and liveness detection by the liveness detection device 2000. The gate control device controls the opening and closing of the gate by obtaining information representing the results of face authentication and liveness detection from the terminal 60. The information representing the results of face authentication and liveness detection is transmitted, for example, from the terminal 60 to the gate control device by wireless communication. In another example, the terminal 60 displays information representing the results of face authentication and liveness detection (for example, a two-dimensional code, etc.) on the display device 20, and causes a reader connected to the gate control device so as to be able to communicate with the gate control device to read the information. The gate control device controls the opening and closing of the gate based on the information read by the reader.

[0039] <About Camera 10> The camera 10 is used in a position and orientation that allows it to capture an image of the face of the target object 50 in a state in which light output from the display area of ​​the display device 20 is irradiated onto the face of the target object 50. For example, the camera 10 is a camera built in near the display area of ​​the display device 20. Such a camera is called an in-camera. However, the camera 10 is not limited to a camera built in the terminal 60 like an in-camera. For example, a portable camera (such as a Web camera) attached to the edge of the display device 20 can be used as the camera 10. In addition, for example, the camera 10 may be installed at a position away from the display device 20. For example, in the above-mentioned example of controlling the opening and closing of a gate according to the result of the live body detection by the live body detection device 2000, a surveillance camera installed so that a person trying to pass through the gate is within the imaging range can be used as the camera 10.

[0040] The camera 10 may be a still camera that generates still images, or a video camera that generates moving images. When the camera 10 is a still camera, the camera 10 generates the captured image 40 in a state in which each of the multiple screens 30 is displayed on the display device 20. On the other hand, when the camera 10 is a video camera, for example, the camera 10 generates video data by capturing an image during a period including the period from when the first screen 30 is displayed to when the last screen 30 is displayed. In this case, the captured image 40 is a video frame that constitutes the video data.

[0041] <Processing flow> 4 is a flowchart illustrating the flow of processing executed by the biological detection device 2000 of the embodiment 1. S102 to S106 configure loop processing A. The loop processing A is executed for each screen 30 to be displayed on the display device 20.

[0042] In S102, the display control unit 2020 determines whether or not loop process A has been executed for all screens 30. If loop process A has been executed for all screens 30, the process in FIG. 4 proceeds to S108. On the other hand, if there are screens 30 that have not yet been the target of loop process A, the display control unit 2020 selects one of them. The screen 30 selected here is referred to as screen i. After the screen 30 is selected, the process in FIG. 4 proceeds to S104.

[0043] The screens 30 may be selected in any order. For example, the display control unit 2020 selects the screens 30 in a predetermined order. Alternatively, for example, the display control unit 2020 may select the screens 30 in a random order.

[0044] The display control unit 2020 causes the display device 20 to display the screen i (S104). Since S106 is the end of the loop process A, the process in Fig. 4 proceeds to S102. After the screen i is displayed on the display device 20 by executing S104, the camera 10 captures an image, and the captured image 40 is generated.

[0045] After the loop process A ends, the acquisition unit 2040 acquires a plurality of captured images 40 (S108). The living body detection unit 2060 performs living body detection on the target object 50 using the acquired plurality of captured images 40 (S110).

[0046] <About Screen 30> The screen 30 has a plurality of regions 32. At least one of the screens 30 displayed on the display device 20 has a plurality of regions 32. At least two of the plurality of regions 32 include displays that are different from each other. The regions 32 that include displays that are different from each other are, for example, regions 32 that are different in color or brightness, or regions 32 that include different characters, symbols, figures, or the like. By including two such regions 32 in the screen 30, it is possible to irradiate the target object 50 with light of non-uniform brightness from the display device 20 on which the screen 30 is displayed.

[0047] FIG. 5 is a diagram illustrating a screen 30 composed of two regions 32. The screen 30-1 is divided into two regions 32-1 and 32-2 by a vertical boundary line. The region 32-1 is the left half of the screen 30-1, and the region 32-2 is the right half of the screen 30-1. The regions 32-1 and 32-2 are filled with different colors (all pixels are the same color). Note that instead of setting a color to all pixels in the region 32 (completely filling the region 32), a color may be set to only some of the pixels in the region 32. For example, in the region 32, pixels to which a color is set (pixels to be illuminated) and pixels to which no color is set (pixels to be not illuminated) may be arranged alternately.

[0048] On the other hand, screen 30-2 is divided into two areas 32-3 and 32-4 by a horizontal boundary line. Area 32-3 is the upper half of screen 30-2, and area 32-4 is the lower half of screen 30-2. Areas 32-3 and 32-4 are filled with different colors.

[0049] The color of the area 32 can be any color such as white, black, red, or blue. In addition, if invisible light such as infrared light can be output from the display device 20, invisible light may be used. For example, after a process of "irradiating visible light from the left half of the display device 20 and irradiating infrared light from the right half of the display device 20," a process of "irradiating infrared light from the left half of the display device 20 and irradiating visible light from the right half of the display device 20" can be performed. When invisible light is used, it is necessary to use a sensor that can detect the invisible light. For example, when visible light and infrared light are used, a camera that can capture visible light and a camera that can capture infrared light are used.

[0050] 5, the sizes of the two regions 32 are the same in both screens 30-1 and 30-2. However, the sizes of the regions 32 may be different.

[0051] 5, the boundary line of the region 32 is a straight vertical or horizontal line. However, the boundary line of the region 32 is not limited to a straight vertical or horizontal line and may be a diagonal line. Furthermore, the boundary line is not limited to a straight line and may be a curved line or a zigzag line.

[0052] The number of regions 32 included in the screen 30 may be three or more. For example, the screen 30 is divided into n (n>=2) regions in the horizontal and / or vertical directions. FIG. 6 is a diagram illustrating a screen 30 having three regions 32. In FIG. 6, the screen 30-1 is divided into three in the horizontal direction (there are three regions 32 aligned in the horizontal direction). Meanwhile, the screen 30-2 is divided into three in the vertical direction (there are three regions 32 aligned in the vertical direction).

[0053] Here, when the screen 30 has three or more areas 32, the display of the multiple areas 32 that are not adjacent to each other may be the same. For example, in the screen 30-1 of Fig. 6, the areas 32-1 and 32-3 are filled with the same color, and the area 32-2 is filled with a different color. On the other hand, in the screen 30-2 of Fig. 6, the three areas 32 are each filled with a different color.

[0054] Moreover, the screen 30 may be divided both horizontally and vertically. Fig. 7 is a diagram illustrating a case where the screen 30 is divided both horizontally and vertically. In Fig. 7, the screen 30-1 is divided into four by horizontal and vertical boundaries. On the other hand, the screen 30-2 is divided into four by two diagonal lines.

[0055] There are various methods for making the displays of the multiple regions 32 different from one another. For example, as described above, the multiple regions 32 are filled with colors different from one another. When filling the regions 32 with colors different from one another, it is preferable to use colors with different brightness so as to make the brightness of the light irradiated from the display device 20 different when these regions 32 are displayed.

[0056] Alternatively, for example, the area 32 may be filled with a pattern such as a lattice pattern or a dot pattern. In this case, it is preferable to vary the brightness of the light emitted from the display device 20 by changing the characteristics of the pattern used. For example, in a lattice pattern, the brightness of the light emitted from the display device 20 can be changed by changing the size of the lattice or the thickness of the lines. Also, in a dot pattern, the brightness of the light emitted from the display device 20 can be changed by changing the size of the dots or the spacing between the dots.

[0057] Alternatively, for example, the region 32 may include characters, symbols, figures, etc. In this case, too, the brightness of the light emitted from the display device 20 can be changed by changing the size or spacing of the characters, symbols, or figures included in the region 32.

[0058] Furthermore, when the brightness of the display area of ​​the display device 20 can be changed for each part, the multiple areas 32 may display the same color at different brightnesses. For example, in the screen 30-1 of FIG. 5, the area 32-1 is a relatively bright white area, and the area 32-2 is a relatively dark white area. In this way, even if the multiple areas 32 are filled with the same color, the brightness of the light irradiated from the display device 20 can be made different for each area 32. Furthermore, the relatively dark area 32 can also be realized by making it an area where light is not irradiated.

[0059] <<Relationship between camera 10 position and division method>> The position of the boundary line dividing the screen 30 into the multiple regions 32 may be determined based on the relative position of the camera 10 with respect to the display device 20. When a person takes an image of himself / herself with the camera 10, there is a high possibility that the terminal 60 is held so that the position of the camera 10 is the center of the face. Therefore, by determining the position of the boundary line of the screen 30 based on the position of the camera 10, the boundary line of the screen 30 can be positioned at the center of the target object 50. Therefore, it becomes possible to change the characteristics of the light irradiated to the target object 50 with the center of the target object 50 as the boundary, such as "irradiating the left half of the face with bright light and the right half of the face with dark light." Since the shape of the face is roughly symmetrical, it is possible to make the comparison easier by irradiating the left and right sides of the face with light of different characteristics.

[0060] Fig. 8 is a diagram illustrating a case in which a boundary line on the screen 30 is determined based on the position of the camera 10. The camera 10 in Fig. 8 is provided to the left of the center of the display area of ​​the display device 20 in the horizontal direction.

[0061] In the case of screen 30-1, the boundary line 34-1 is a straight line extending vertically downward from camera 10. That is, screen 30 is divided into left and right halves based on the straight line extending vertically downward from camera 10. Here, because camera 10 is provided on the left side, area 32-1 is smaller than area 32-2.

[0062] On the other hand, in the case of screen 30-2, the screen 30 is divided into two equal parts while a boundary line 34 is defined based on the position of the camera 10. Specifically, the upper end of boundary line 34-2 is defined as the intersection of a straight line drawn vertically downward from the camera 10 and the upper side of the display area of ​​the display device 20. Then, area 32-4 is a straight line drawn diagonally downward to the right from the upper end so that the screen 30-2 is divided into two equal parts by boundary line 34-2.

[0063] When the screen 30 is divided either horizontally or vertically, which direction it is divided into may be determined by the position of the camera 10. Specifically, when the camera 10 is provided above or below the display area of ​​the display device 20, the display control unit 2020 divides the screen 30 in the horizontal direction. On the other hand, when the camera 10 is provided to the left or right of the display area of ​​the display device 20, the display control unit 2020 divides the screen 30 in the vertical direction.

[0064] In order to determine the boundary line 34 based on the position of the camera 10, the live body detection device 2000 needs to be able to grasp the positional relationship between the camera 10 and the display area of ​​the display device 20. For example, the position of one end of the boundary line 34 determined as described above is set in advance in the live body detection device 2000. Alternatively, for example, the position of one end of the boundary line 34 may be set by the user of the terminal 60. For example, the live body detection device 2000 allows the user of the terminal 60 to specify a position on the display area of ​​the display device 20. Then, the live body detection device 2000 uses the x coordinate of the specified position as the x coordinate of the upper end of the boundary line 34.

[0065] Note that not only one end of the boundary line 34 but also the boundary line 34 itself may be set by the user of the terminal 60. For example, the user may be made to draw a line on the display device 20, and the line may be used as the boundary line 34. Note that, when it is desired to determine the boundary line 34 based on the user's face, for example, an image of the user's face captured by the camera 10 (captured image 40) is displayed on the display device 20. Then, for example, a guide message such as "Draw a line so that it passes through the center of your face" is displayed on the display device 20, thereby allowing the boundary line 34 based on the user's face to be set.

[0066] Furthermore, the living body detection device 2000 may detect a face from the captured image 40, and automatically determine the boundary line 34 based on the position of the face on the captured image 40 (for example, a vertical line passing through the center of the face).

[0067] <Display control of screen 30: S102-S106> The display control unit 2020 causes the display device 20 to display the multiple screens 30 at different times (S102-S106). The multiple screens 30 can be the various screens 30 described above. Note that two or more types of screens 30 are displayed on the display device 20, and at least one of the screens 30 has a plurality of different regions 32. The screens 30 displayed on the display device 20 may include a screen 30 that is not divided into a plurality of regions 32.

[0068] Fig. 9 is a first diagram illustrating a plurality of screens 30 displayed on the display device 20 by the display control unit 2020. In the example of Fig. 9, the screens 30 are displayed in the following order: screen 30-1, screen 30-2, screen 30-3, and screen 30-4.

[0069] The left half of screen 30-1 is filled with white, and the right half is filled with black. The left half of screen 30-2 is filled with black, and the right half is filled with white. Screen 30-3 is filled with white entirely. Screen 30-4 is filled with black entirely.

[0070] Fig. 10 is a second diagram illustrating a plurality of screens 30 displayed on the display device 20 by the display control unit 2020. In Fig. 10, m screens 30 are displayed in sequence. These m screens 30 are generated by gradually shifting to the right the boundary between the left region 32 filled with white and the right region 32 filled with black.

[0071] In addition, when the boundary line is moved as shown in FIG. 10, a plurality of patterns of the screen 30 may be displayed for one position of the boundary line. For example, in the example of FIG. 10, the boundary line is moved every three seconds, and the display of the area 32 is changed every second. In this case, three patterns of the screen 30 are displayed with the boundary line position fixed. For example, by displaying three types of screens 30, namely, a screen 30 in which the left area 32 is white, a screen 30 in which the left area 32 is relatively light gray, and a screen 30 in which the left area 32 is relatively dark gray, a plurality of patterns of the screen 30 can be displayed with the boundary line fixed. However, the area 32 whose display is changed with the boundary line fixed is not limited to only one area 32, and the display of each of the plurality of areas 32 may be changed.

[0072] The time for which each screen 30 is displayed on the display device 20 may be the same or different.

[0073] The camera 10 needs to be able to capture images of two or more screens 30 at the timing when the screens 30 are displayed on the display device 20. To achieve this, for example, the display control unit 2020 controls the capture of images by the camera 10. Specifically, the living body detection device 2000 repeats a process of "displaying the screen 30 on the display device 20, and then causing the camera 10 to capture an image" while changing the screen 30. In this way, for each screen 30, a captured image 40 is obtained for the state in which the screen 30 is displayed on the display device 20.

[0074] However, the timing of capturing images by the camera 10 does not have to be controlled by the display control unit 2020. For example, the camera 10 is configured to capture images repeatedly. By switching the screens 30 displayed on the display device 20 while the camera 10 is capturing images repeatedly, a captured image 40 can be obtained for each screen 30 being displayed. In this case, for example, it is preferable that the length of time during which one screen 30 is displayed on the display device 20 is set to be longer than the imaging cycle of the camera 10. In this way, for each of all screens 30, one or more captured images 40 generated while the screen 30 is displayed on the display device 20 can be obtained. However, the length of time during which the screen 30 is displayed on the display device 20 does not necessarily have to be set to be longer than the imaging cycle of the camera 10.

[0075] When three or more screens 30 are displayed on the display device 20, it is not necessary to generate captured images 40 for all of the screens 30. For example, in the example of FIG. 10, the camera 10 is a video camera, and the capturing period of the camera 10 is longer than the movement period of the boundary line. In this case, for any one or more of the m screens 30, there is a possibility that the camera 10 will not capture an image while the screen 30 is displayed. However, if two or more captured images 40 can be obtained, the live body detection unit 2060 can realize live body detection.

[0076] <Acquisition of captured image 40: S108> The acquiring unit 2040 acquires the multiple captured images 40 generated by the camera 10 (S108). Here, various methods can be used to acquire the images generated by the camera. For example, the acquiring unit 2040 acquires the captured images 40 by accessing a storage device in which the captured images 40 are stored. Here, the storage device may be provided either inside or outside the camera 10. Alternatively, for example, the acquiring unit 2040 may acquire the captured images 40 by receiving the captured images 40 transmitted from the camera 10 to the biological detection device 2000.

[0077] <Performing live body detection: S110> The living body detection unit 2060 performs living body detection using the acquired multiple captured images 40. Specifically, the living body detection unit 2060 determines whether or not the target object 50 captured by the camera 10 is a living body.

[0078] As described above, the shape of the target object 50 is significantly different between a case where the target object 50 is a paper photograph or the like and a case where the target object 50 is a real person or the like, and such a difference in shape appears as a difference in the characteristics of the reflected light in the captured image 40. Therefore, for example, the living body detection unit 2060 determines whether the target object 50 is a living body or not based on the characteristics of the color and brightness distribution of the target object 50 on the captured image 40.

[0079] There are various specific methods for realizing the live body detection by the live body detection unit 2060. For example, the live body detection is realized by using a trained discrimination model. This discrimination model outputs data indicating a determination result as to whether or not the object captured by the camera 10 is a live body in response to input of a plurality of captured images 40. The data indicating the determination result is, for example, a label indicating 1 when the object captured by the camera 10 is a live body and indicating 0 when the object captured by the camera 10 is not a live body. Such a discrimination model can be realized by, for example, a neural network such as a CNN (Convolutional Neural Network). However, the type of discrimination model is not limited to a neural network. According to the method of performing live body detection using such a trained discrimination model, an algorithm for determining whether or not the target object 50 is a live body can be easily constructed in a data-driven manner.

[0080] The learning of the discrimination model can be realized by using training data consisting of a combination of "plural captured images 40 and a correct answer label." For example, it is assumed that the display control unit 2020 displays three types of screens 30, namely, a first pattern screen 30 to a third pattern screen 30, on the display device 20. In this case, the captured images 40 of the first pattern to the third pattern are obtained corresponding to the first pattern screen 30 to the third pattern screen 30, respectively. Here, the captured image 40 of the nth pattern (n is an integer between 1 and 3) is the captured image 40 generated by the camera 10 while the screen 30 of the nth pattern is displayed on the display device 20. When the captured images 40 of three patterns are obtained in this way, the training data consisting of a combination of "a captured image 40 of the first pattern, a captured image 40 of the second pattern, a captured image 40 of the third pattern, and a correct answer label" is used for learning the discrimination model.

[0081] It is preferable to use both positive example training data and negative example training data to train the discrimination model. The positive example training data indicates a plurality of captured images 40 obtained when a living body is imaged by the camera 10, and a correct answer label (e.g., 1) indicating that a living body has been imaged by the camera 10. On the other hand, the negative example training data indicates a plurality of captured images 40 obtained when an object other than a living body is imaged by the camera 10, and a correct answer label (e.g., 0) indicating that an object other than a living body has been imaged by the camera 10. By using both the positive example training data and the negative example training data, it is possible to learn the difference between the captured image 40 obtained when a living body is imaged and the captured image 40 obtained when an object other than a living body is imaged.

[0082] <Result output> The living body detection device 2000 outputs information indicating the result of living body detection (hereinafter, detection result information). The detection result information indicates the determination result of whether the target object 50 is a living body or not. It is preferable that the detection result information is displayed on a display device (e.g., the display device 20) that can be viewed by the user of the terminal 60. However, the detection result information may be output to a device other than the display device of the terminal 60.

[0083] <Other processing> In addition to the light irradiated from the display device 20, the target object 50 is also irradiated with ambient light (such as sunlight or lighting) around the display device 20. If the ambient light is extremely stronger than the light irradiated from the display device 20, the ambient light becomes dominant, and therefore even if the intensity of the light irradiated from the display device 20 is changed, the change is unlikely to appear in the captured image 40. For this reason, it is preferable that the detection of a living body using the living body detection device 2000 is performed in a place where the ambient light is not too strong.

[0084] Therefore, for example, the living body detection device 2000 may specify the intensity of the ambient light before controlling the display device 20 by the display control unit 2020, and start a series of processes for living body detection (S102 to S110 in FIG. 4) only when it is determined that the ambient light is equal to or less than a threshold. In this way, the living body detection process can be performed when the intensity of the ambient light is appropriate.

[0085] The intensity of the ambient light is specified, for example, by using an illuminance sensor. The illuminance sensor may be built into the display device 20 or the terminal 60 in advance, or may be provided separately. When the ambient light is greater than a threshold, for example, the living body detection device 2000 displays a message on the display device 20 urging the user to move to a slightly darker place (a place where the ambient light is weaker than the current environment). In this way, when the terminal 60 is not being used in an appropriate environment, the user is notified of this, and the terminal 60 can be used in an appropriate environment.

[0086] The method of determining whether the intensity of the ambient light is appropriate is not limited to the method of measuring the intensity of the ambient light by a sensor. For example, the living body detection device 2000 determines whether the intensity of the ambient light is appropriate based on the luminance of the captured image 40. The captured image 40 may be generated while the screen 30 is displayed on the display device 20, or may be generated while the screen 30 is not displayed on the display device 20. For example, the living body detection device 2000 determines that the intensity of the ambient light is appropriate when a statistical value (e.g., an average value) of the luminance of all or a part of the pixels included in the captured image 40 is equal to or less than a threshold value. On the other hand, when the statistical value is greater than the threshold value, the living body detection device 2000 determines that the intensity of the ambient light is not appropriate.

[0087] Alternatively, for example, the biological detection device 2000 may compare a captured image 40 generated by the camera 10 while a relatively bright screen 30 (e.g., an entirely white screen 30) is displayed on the display device 20 with a captured image 40 generated by the camera 10 while a relatively dark screen 30 (e.g., an entirely black screen 30) is displayed on the display device 20, and determine whether there is a sufficient difference in luminance (e.g., the difference in luminance is equal to or greater than a threshold). The difference in luminance can be represented, for example, by the difference between a statistical value (e.g., average value) of the luminance of all pixels included in the former captured image 40 and a statistical value of the luminance of all pixels included in the latter captured image 40.

[0088] However, in calculating the luminance statistics, the luminance of the area including the target object 50 may be prioritized over the luminance of other areas. For example, the living body detection device 2000 detects an area including the target object 50 from the captured image 40, and calculates the luminance statistics using only the luminance of the area. As another example, the living body detection device 2000 may assign a larger weight to the luminance of the area including the target object 50 than to the other areas, and then calculate a weighted average of the luminance of the entire image.

[0089] If it is determined that there is not a sufficient difference in luminance, it is considered that the environmental light is too strong. Therefore, if it is determined that there is not a sufficient difference in luminance, for example, the living body detection device 2000 does not determine whether the target object 50 is a living body or not, and displays a message on the display device 20 prompting the user to move to a slightly darker place and then try the living body detection again. This method of determining whether the environmental light is too strong by comparing the captured images 40 has the advantage that it does not require hardware (such as an illuminance sensor) for specifying the intensity of the environmental light.

[0090] The display control unit 2020 may change the screen 30 displayed on the display device 20 according to the intensity of the ambient light. For example, it is preferable to increase the contrast of the screen 30 as the ambient light increases. This can reduce the likelihood that the ambient light will become dominant. Also, by doing so, the contrast of the screen 30 decreases as the ambient light decreases, so that the power consumption of the display device 20 can be reduced. The contrast of the screen 30 can be achieved by either one or both of the two measures of "making the relatively bright area 32 brighter" and "making the relatively dark area 32 darker."

[0091] For example, a plurality of screens 30 to be displayed on the display device 20 are predefined for each level of the intensity of ambient light (hereinafter, ambient light level). A higher ambient light level is associated with a screen 30 having a higher contrast. The display control unit 2020 identifies which ambient light level the current ambient light intensity corresponds to, and uses the screen 30 associated with the identified ambient light level.

[0092] Alternatively, for example, the display control unit 2020 may generate the screen 30 according to the intensity of the ambient light by changing (correcting) the color of pixels included in one or more regions 32 included in the screen 30 according to the intensity of the ambient light. For example, a correction coefficient α is prepared whose value increases as the ambient light becomes stronger. The value range of the correction coefficient α is, for example, 0<α≦1. By multiplying the brightness of the color included in the region 32 by this correction coefficient, the region 32 can be made brighter as the ambient light becomes stronger.

[0093] The live body detection device 2000 may acquire a captured image 40 and perform face detection processing before displaying the screen 30 on the display device 20, and start a series of processes for live body detection in response to detection of a face from the captured image 40. This is because live body detection cannot be performed if a face is not detected from the captured image 40. For example, the live body detection device 2000 displays a message on the display device 20 to request the user to capture a face, and then repeatedly acquires the captured image 40 and performs face detection processing. Then, when a face is detected from the captured image 40, the live body detection device 20 starts a series of processes for live body detection. On the other hand, if a face is not detected from the captured image 40 for a certain period of time, the live body detection device 20 displays a message on the display device 20 prompting the user to capture a face.

[0094] Furthermore, when a face is detected from the captured image 40, the live body detection device 2000 may determine whether or not the size of the face is sufficiently large. For example, the live body detection device 2000 starts a series of processes for live body detection only when the size of the face is sufficiently large. On the other hand, when the size of the face is not sufficiently large, the live body detection device 2000 displays a message on the display device 20 prompting the user to capture a larger image of the face.

[0095] Whether or not the face size is sufficient is determined, for example, by comparing the area of ​​the face region of the target object 50 in the captured image 40 with a threshold. Alternatively, whether or not the face size is sufficient may be determined by comparing the distance between the left eye and the right eye with a threshold, instead of the face size itself. In this case, the live body detection device 2000 detects the left eye and the right eye of the target object 50 from the captured image 40 and calculates the distance between them. Then, the live body detection device 2000 determines whether or not the calculated distance is equal to or smaller than a threshold. If the distance between the eyes is equal to or smaller than the threshold, the live body detection device 2000 determines that the face size is sufficient. On the other hand, if the distance between the eyes is greater than the threshold, the live body detection device 2000 determines that the face size is not sufficient.

[0096] Alternatively, for example, the living body detection device 2000 may determine whether the face size is sufficient based on the distance between the display device 20 and the target object 50. For example, when the distance between the display device 20 and the target object 50 is equal to or less than a threshold, the living body detection device 2000 determines that the face size is sufficient. On the other hand, when the distance between the display device 20 and the target object 50 is greater than the threshold, the living body detection device 2000 determines that the face size is not sufficient. Note that, various methods such as a method using a depth sensor can be used to calculate the distance between the display device 20 and the target object 50. Also, the distance between the camera 10 and the target object 50 may be used instead of the distance between the display device 20 and the target object 50. In this case, the distance between the camera 10 and the target object 50 can be grasped by using a stereo camera or a camera with a depth sensor as the camera 10.

[0097] In order to facilitate the user to capture an image of the face at an appropriate size, the liveness detection device 2000 may display a guide indicating an appropriate face size on the display device 20. Fig. 11 is a diagram illustrating an example of a guide indicating an appropriate face size. The elliptical frame indicates the position where the face should be positioned. Also, the dotted line indicates the position where the center line of the face (the line passing through the center) should be positioned. By displaying such a guide, the user can capture an image more easily.

[0098] The live body detection device 2000 may detect that the center line of the user's face in the captured image 40 overlaps with the center line of the face indicated by the above-mentioned guide (dotted line in FIG. 11), and may start a series of processes for live body detection in response to the detection. This allows live body detection to be performed at the timing when the face is in an appropriate posture, improving the accuracy of live body detection. Note that existing technology can be used as a technology for identifying the center line of the face by image analysis.

[0099] Alternatively, for example, the living body detection device 2000 may detect that the target object 50 has blinked using the captured image 40, and may start a series of processes for living body detection in response to the detection. Note that existing technology can be used to detect blinking from time-series images.

[0100] [Embodiment 2] The living body detection device 2000 of the second embodiment performs living body detection on the target object 50, similar to the living body detection device 2000 of the first embodiment. However, the living body detection device 2000 of the second embodiment achieves living body detection in a different method from that of the living body detection device 2000 of the first embodiment.

[0101] As a premise, in this embodiment, the camera 10 captures an image of a person (target object 50) to be subjected to liveness detection moving the face direction from side to side with the line of sight fixed. Fig. 12 is a diagram illustrating a state in which the face direction is moved from side to side with the line of sight fixed (while continuing to look at the same place). The line of sight is fixed at one point on the terminal 60.

[0102] Here, if the face captured by the camera 10 is a real face, the target person can move their eyes independently of the movement of their face, so they can change the direction of their face while keeping their gaze direction fixed. If the person performs an action that changes the direction of their face over time while keeping their gaze direction fixed, the difference between the direction of their face and the direction of their gaze changes over time. For example, when both the face and the gaze are facing forward, the difference between the direction of their face and the direction of their gaze is small, whereas when the face is facing left or right while the gaze is facing forward, the difference between the direction of their face and the direction of their gaze is large.

[0103] On the other hand, suppose that the target person wears an article of clothing (such as a so-called 3D mask) on which another person's face is drawn for the purpose of impersonation. In this case, since the eyes are drawn on the mask, the eyes cannot be moved independently of the movement of the face. In other words, the direction of gaze changes together with the direction of the face. Therefore, even if an attempt is made to change the direction of the face over time while keeping the direction of gaze fixed, the difference between the direction of the face and the direction of gaze does not actually change over time, or the amount of change becomes small.

[0104] FIG. 13 is a diagram illustrating the change over time in face direction and gaze direction when the face is turned left and right while the gaze direction is fixed. In both graphs, the solid line represents the face direction, and the dotted line represents the gaze direction. The graph on the left represents the case where the face captured by the camera 10 is a real face. On the other hand, the graph on the right represents the case where the face captured by the camera 10 is a 3D mask. As can be seen by comparing these graphs, when a 3D mask is worn, the difference between the face direction and gaze direction becomes relatively small.

[0105] Therefore, the living body detection device 2000 judges whether the target object 50 is a living body or not based on the difference between the face direction and the gaze direction. Specifically, the living body detection device 2000 acquires a plurality of captured images 40 generated by the camera 10 at different times, identifies the face direction and gaze direction of the target person's face captured in each captured image 40, and calculates the difference between them. Here, existing technology can be used to identify the face direction and gaze direction of the face included in the image. Then, the living body detection device 2000 judges whether the target object 50 captured in the captured image 40 is a living body or not based on the difference between the face direction and the gaze direction. Note that when it is necessary to distinguish the captured image 40 of the second embodiment from the captured image 40 of the first embodiment, the captured image 40 of the second embodiment is also referred to as a "second captured image."

[0106] Furthermore, in order to realize the above-mentioned live body detection, the live body detection device 2000 displays information (hereinafter, guide information) indicating a guide for making the target object 50 perform a desired operation on the display device 20 of the terminal 60. The live body detection device 2000 outputs at least both guide information indicating a guide regarding the line of sight direction and guide information indicating a guide regarding the face direction. Note that these pieces of guide information may be output simultaneously or at different times.

[0107] Fig. 14 is a first diagram illustrating an example of guide information. In Fig. 14, a screen 80 including guide information is displayed on the display device 20. The screen 80 includes, as guides regarding the gaze direction, a mark indicating the position to direct the gaze and a message such as "Direct your gaze here." The screen 80 also includes, as guides regarding the facial direction, a message such as "Move your face left and right."

[0108] In the following explanation, unless otherwise specified, the face and gaze direction of the target object 50 is represented by a reference angle of 0° in the forward direction (the direction in which the display device 20 is viewed from the front), a positive angle to the right, and a negative angle to the left.

[0109] <Example of action and effect> As mentioned above, one method of impersonating another person is to use a 3D mask or the like on which another person's face is drawn. Here, as mentioned above, when a 3D mask or the like is worn, the gaze moves along with the face, whereas when a 3D mask is not worn, the gaze can be moved independently of the face. Therefore, there is a difference between the direction of the face and the direction of the gaze when a 3D mask or the like is worn and when it is not.

[0110] In this regard, according to the live body detection device 2000, the camera 10 captures an image of a situation in which a person is moving his or her face from side to side while fixing the line of sight. The live body detection device 2000 then analyzes the captured image 40 obtained from the camera 10 to calculate the difference between the face direction and the line of sight, and performs live body detection based on the calculated difference. Therefore, the live body detection device 2000 can prevent spoofing using a 3D mask or the like.

[0111] <Example of functional configuration> 15 is a block diagram illustrating a functional configuration of the living body detection device 2000 of the second embodiment. The living body detection device 2000 of the second embodiment includes a second display control unit 2080, a second acquisition unit 2100, and a second living body detection unit 2120. The second display control unit 2080 displays guide information on the display device 20 of the terminal 60. The second acquisition unit 2100 acquires a plurality of captured images 40 generated by the camera 10 after displaying the guide information. The second living body detection unit 2120 specifies the face direction and the line of sight direction of the target object 50 for each of the plurality of captured images 40. Then, the second living body detection unit 2120 determines whether the target object 50 is a living body or not based on the difference between the face direction and the line of sight direction specified for each captured image 40.

[0112] <Example of hardware configuration> The hardware configuration of the biological detection device 2000 of the second embodiment is shown in, for example, Fig. 3, similar to the hardware configuration of the biological detection device 2000 of the first embodiment. However, the storage device 508 of the second embodiment stores programs for realizing each function of the biological detection device 2000 of the second embodiment.

[0113] <Processing flow> 16 is a flowchart illustrating a process executed by the living body detection device 2000 of the second embodiment. The second display control unit 2080 causes the display device 20 to display guide information (S202). The second acquisition unit 2100 acquires a plurality of captured images 40 (S204). The second living body detection unit 2120 calculates a difference between the gaze direction and the face direction of the target object 50 for each of the plurality of captured images 40 (S206). Note that the gaze direction and the face direction do not necessarily need to be calculated for all captured images 40. The second living body detection unit 2120 determines whether the target object 50 is a living body or not based on the difference between the face direction and the gaze direction calculated for each of the plurality of captured images 40 (S208).

[0114] Note that the flow of the process executed by the living body detection device 2000 of the second embodiment is not limited to the flow shown in Fig. 16. For example, the captured images 40 used by the second living body detection unit 2120 do not need to be acquired all at once. For example, the living body detection device 2000 calculates the difference between the face direction and the gaze direction for each of the multiple captured images 40 by repeating the process of "1) acquiring one new captured image 40, and 2) calculating the difference between the face direction and the gaze direction for the acquired captured image 40".

[0115] Alternatively, for example, the timing for displaying the guide information is not limited to before acquisition of the captured image 40. For example, as described later, the living body detection device 2000 may determine whether or not the action being performed by the target object 50 is an appropriate action, and when the appropriate action is not being performed, display guide information for causing the appropriate action to be performed.

[0116] <About guide information> There are various kinds of guide information displayed on the display device 20 by the second display control unit 2080. For example, as shown in the example of Fig. 14, the guide for the line of sight direction indicates the position where the line of sight should be fixed, and the guide for the face direction indicates the direction in which the face should be moved. By looking at this guide, the user can see that the target object 50 should move the face from side to side while fixing the line of sight direction.

[0117] Alternatively, for example, the second display control unit 2080 displays guide information indicating the next action that the target object 50 should take, according to the movement of the face of the target object 50. FIG. 17 is a diagram illustrating an example of guide information indicating the next action that the target object 50 should take. In FIG. 17, guide information including a message for the target object 50, a guide indicating a position to direct the gaze, and a guide indicating a direction to direct the face are displayed on all of the screens 90-1 to 90-3 displayed on the display device 20. By displaying such guides on the display device 20, the target object 50 can easily understand which direction to face.

[0118] First, the second display control unit 2080 causes the display device 20 to display a screen 90-1. The guide information displayed on the screen 90-1 includes a message encouraging the user to face forward and an image of a face facing forward as a guide regarding the direction of the face. In addition, a cross mark and an image of an eye are displayed as a guide regarding the line of sight.

[0119] The second display control unit 2080 determines whether or not the face of the target object 50 faces forward. The direction of the face of the target object 50 is calculated by the second living body detection unit 2120 using the captured image 40.

[0120] Here, it is preferable that the screen 90-1 continues to be displayed until the face of the target object 50 faces forward. In this way, it is possible to prevent live body detection from starting until the face of the target object 50 faces the front direction as a reference. However, if the face of the target object 50 does not face forward even after a predetermined time has elapsed, the target object 50 may be determined to be a non-live body due to a timeout.

[0121] On the other hand, if it is determined that the face of the target object 50 faces forward, the second display control unit 2080 causes the display device 20 to display a screen 90-2. On the screen 90-2, a message urging the user to turn left, an arrow pointing left, and an image of a face turning left are displayed as guides for the face direction. Also, a cross mark and an image of an eye are displayed as guides for the line of sight.

[0122] The second display control unit 2080 determines whether or not the face of the target object 50 faces left. For example, when the direction of the face of the target object 50 is equal to or smaller than a threshold, it is determined that the face of the target object 50 faces left.

[0123] The display device 20 continues to display the screen 90-2 until the face of the target object 50 turns to the left. However, if the face of the target object 50 does not turn to the front even after a predetermined time has elapsed, a timeout may occur and the target object 50 may be determined to be a non-living body.

[0124] On the other hand, if it is determined that the face of the target object 50 faces left, the second display control unit 2080 causes the display device 20 to display a screen 90-3. On the screen 90-3, a message urging the user to face right, a rightward arrow, and an image of a face facing right are displayed as guides for the face direction. Also, a cross mark and an image of an eye are displayed as guides for the line of sight direction.

[0125] In this way, by displaying the screens 90-1, 90-2, and 90-3 in order, it is possible to encourage the subject to change the direction of the face to the front, left, and right in that order while keeping the line of sight fixed on the target object 50. Furthermore, by subsequently displaying the screens 90-2 and 90-3 in an alternating manner, it is possible to encourage the subject to repeatedly change the direction of the face to the left and right with respect to the target object 50.

[0126] Note that a guide showing the face direction may be displayed as an animation so that the target object 50 can more easily understand how to move its face. Fig. 18 is a diagram showing an example of an animation of a guide showing the face direction.

[0127] Here, it is preferable that the guide information is displayed on the display device 20 together with the captured image 40 generated by the camera 10. For example, in a mobile terminal such as a smartphone, when the camera is used, the result of imaging by the camera can be displayed on the display device in real time. Therefore, in the terminal 60 as well, the captured image 40 generated by the camera 10 is displayed on the display device 20 in real time, and the guide information is displayed on the display device 20 together with the captured image 40 (for example, superimposed on the captured image 40). In this way, the target object 50 can perform the action of shaking its face from side to side while fixing its gaze, while checking the state of its own face and gaze.

[0128] For example, in the case of animation guide information shown in FIG. 18, it is preferable to display the captured image 40 generated by the camera 10 on the display device 20 in real time together with the animation. This allows the target object 50 to move its own face so that it overlaps with the animation, thereby allowing the target object 50 to appropriately shake its face from side to side. This allows the target object 50 to more intuitively perform the actions required for live body detection. However, the animation representing the facial movement may be displayed in a position (e.g., a corner of the screen) that does not overlap with the face of the target object 50.

[0129] Alternatively, for example, when the action performed by the target object 50 is not appropriate, the second display control unit 2080 may display guide information for enabling the target object 50 to perform an appropriate action. For example, the second display control unit 2080 determines whether the magnitude of the amplitude of the face of the target object 50 is sufficiently large (whether it is equal to or greater than a threshold value). The amplitude of the face of the target object 50 can be calculated as the difference between the maximum value and the minimum value of the direction of the face of the target object 50.

[0130] When the magnitude of the shaking width of the target object 50's face is smaller than the threshold value, the second display control unit 2080 causes the display device 20 to display guide information requesting an increase in the shaking width of the face. For example, guide information including a message such as "Please move your face more" is displayed. By displaying such guide information, the target object 50 can understand that it should shake its face more. Then, when the shaking width of the face is increased by such a guide, the difference between the face direction and the gaze direction becomes larger when the target object 50 is a real face. Therefore, it becomes possible to perform liveness detection with higher accuracy.

[0131] Another case where the action performed by the target object 50 is inappropriate is when the line of sight of the target object 50 is not fixed. For example, the second display control unit 2080 determines whether the line of sight of the target object 50 is fixed, and when the line of sight of the target object 50 is not fixed, causes the display device 20 to display guide information including a message encouraging the user to fix the line of sight. For example, guide information including a message such as "Do not move your line of sight" is displayed. By displaying such guide information, it is possible to know that the target object 50 is not able to fix the line of sight direction.

[0132] Whether the gaze is fixed or not can be calculated based on, for example, a change in the gaze direction. For example, if the absolute value of the difference between the gaze direction calculated when the target object 50 faces forward and the current gaze direction is equal to or greater than a threshold, it is determined that the gaze is not fixed.

[0133] The second display control unit 2080 may cause the display device 20 to display guide information indicating the gaze position and facial direction of the target object 50 so that the target object 50 can understand the gaze position and facial direction of the target object 50. Fig. 19 is a diagram illustrating an example of guide information including the gaze position and facial direction of the target object 50. In Fig. 19, the current gaze position is represented by a white circle. The target object 50 can intuitively fix the gaze by adjusting the gaze position so that the white circle overlaps with the cross.

[0134] In addition, in Fig. 19, the current face direction is represented by a black circle. By looking at the black circle, the target object 50 can intuitively know in which direction its face is facing. Note that Fig. 19 shows a case in which the target object 50 is required to face forward. Therefore, the target object 50 can intuitively face the face forward by adjusting the face direction so that the black circle overlaps with the cross.

[0135] In addition, when a series of processes for live body detection are started when the target object 50 faces forward, the live body detection device 2000 may determine whether or not both the white circle and the black circle overlap with the cross in the example of Figure 19, and if it is determined that both overlap with the cross, start the series of processes for live body detection.

[0136] The guide indicating the face direction is not limited to that shown in Fig. 19. Fig. 20 is a second diagram illustrating an example of guide information including the gaze position and face direction of the target object 50. In Fig. 20, the marks indicating the current gaze position and the current face direction are the same as those in Fig. 19.

[0137] In Fig. 20, a black circle representing the face direction is displayed in a horizontally long rectangular frame. A black star is also displayed as a mark representing the target face direction. Furthermore, a message is also displayed encouraging the user to move the face direction to the direction of the star. The target object 50 can intuitively change the face direction to an appropriate direction by changing the face direction so that the black circle overlaps with the black star.

[0138] In addition, when the face direction moves to the maximum direction (black star mark in Figure 20), the living body detection device 2000 may display guide information on the display device 20 urging the user to keep the face direction in that direction for a certain period of time (e.g., 2 seconds).

[0139] Another case where the action performed by the target object 50 is inappropriate may be when the facial movement is too fast. In this case, the second display control unit 2080 may cause the display device 20 to display a guide to encourage the user to slow down the facial movement when the facial movement is too fast. The speed of the facial movement may be represented, for example, by the difference in the position of the face in the captured images 40 adjacent in time series. Alternatively, for example, the difference in the facial movement may be represented by the amount of change in the direction of the face over time.

[0140] For example, the second display control unit 2080 determines whether the speed of the facial movement is equal to or greater than a threshold. If the speed of the facial movement is equal to or greater than the threshold, the second display control unit 2080 causes the display device 20 to display guide information that prompts the user to slow down the facial movement. This guide information includes, for example, a message such as "Please move your face more slowly." Alternatively, for example, the facial movement animation shown in FIG. 17 may be included in the guide information. In this case, the animation included in the guide information may cause the face to move slower than normal.

[0141] The second display control unit 2080 may display a guide on the posture of the torso (for example, shoulders, etc.) in addition to the face. For example, the second display control unit 2080 calculates the direction of the torso of the target object 50 using the captured image 40, and determines whether the torso faces forward (for example, whether the absolute value of the difference between the direction of the torso and the forward direction is equal to or less than a threshold) based on the calculated direction of the torso. Then, when it is determined that the torso does not face forward, the second display control unit 2080 displays guide information on the display device 20 to prompt the user to face the torso forward. For example, this guide information includes a message such as "Please face your torso forward." In addition, for example, the guide information may include an image of the torso facing forward.

[0142] In the above example, the guide information is displayed on the display device 20. However, in addition to or instead of displaying the guide information on the display device 20, the guide information may be output as a voice. For example, various messages such as "Please face forward" or "Please look left while keeping your gaze fixed" are output as voice messages. Also, as described above, when it is detected that the movement of the target object 50 is not appropriate, a voice message according to the detected state may be output. For example, a voice message such as "Please turn your gaze a little more to the upper right" or "Please move your face a little more slowly" is output. Also, when a direction can be expressed by voice using stereo sound or the like, the direction in which the gaze or face should be directed may be expressed by the direction of the voice. For example, a message "Please turn your face in the direction of the sound" is output as a voice heard from the right direction.

[0143] <Living body detection: S206, S208> The second living body detection unit 2120 calculates the face direction and gaze direction of the target object 50 for each captured image 40 using the image of the face of the target object 50 included in the captured image 40, and calculates the difference therebetween (S206). Then, based on the difference between the face direction and gaze direction calculated for each captured image 40, it determines whether the target object 50 is a living body or not (S208). For example, when there is a sufficient difference between the face direction and gaze direction of the target object 50, the second living body detection unit 2120 determines that the target object 50 is a living body. On the other hand, when there is not a sufficient difference between the face direction and gaze direction of the target object 50, the second living body detection unit 2120 determines that the target object 50 is not a living body.

[0144] Whether or not there is a sufficient difference between the face direction and the gaze direction of the target object 50 is determined by, for example, comparing the difference between the face direction and the gaze direction with a threshold. As a specific example, the second living body detection unit 2120 calculates the difference between the face direction and the gaze direction for each captured image 40, and determines whether or not the maximum value of the difference between the face direction and the gaze direction is equal to or greater than a first threshold. If the maximum value of the difference between the face direction and the gaze direction is equal to or greater than the first threshold, it is determined that there is a sufficient difference between the face direction and the gaze direction. On the other hand, if the maximum value of the difference between the face direction and the gaze direction is less than the first threshold, it is determined that there is no sufficient difference between the face direction and the gaze direction. Here, since the right direction is defined as the positive direction, the fact that the maximum value of the difference between the face direction and the gaze direction is equal to or greater than the first threshold means that there is a sufficient difference between the face direction and the gaze direction when the face is turned to the right (in other words, the face is turned to the right to a sufficient extent while keeping the gaze fixed).

[0145] Alternatively, for example, whether or not there is a sufficient difference between the face direction and the gaze direction of the target object 50 may be determined by comparing the minimum value of the difference between the face direction and the gaze direction with the second threshold. If the minimum value of the difference between the face direction and the gaze direction is equal to or less than the second threshold, it is determined that there is a sufficient difference between the face direction and the gaze direction. On the other hand, if the minimum value of the difference between the face direction and the gaze direction is greater than the second threshold, it is determined that there is not a sufficient difference between the face direction and the gaze direction. Here, since the left direction is defined as a negative direction, the fact that the minimum value of the difference between the face direction and the gaze direction is equal to or less than the second threshold means that there is a sufficient difference between the face direction and the gaze direction when the face is turned to the left (in other words, the face is turned to the left sufficiently far while keeping the gaze fixed).

[0146] The second living body detection unit 2120 may determine that there is a sufficient difference between the face direction and the gaze direction when the face is sufficiently moved left and right while keeping the gaze fixed. In this case, for example, the second living body detection unit 2120 determines whether or not "the maximum value of the difference between the face direction and the gaze direction is equal to or greater than a first threshold value, and the minimum value of the difference between the face direction and the gaze direction is equal to or less than a second threshold value." If the maximum value of the difference between the face direction and the gaze direction is equal to or greater than the first threshold value, and the minimum value of the difference between the face direction and the gaze direction is equal to or less than a second threshold value, the second living body detection unit 2120 determines that there is a sufficient difference between the face direction and the gaze direction. On the other hand, if the maximum value of the difference between the face direction and the gaze direction is not equal to or greater than the first threshold value, or the minimum value of the difference between the face direction and the gaze direction is not equal to or less than the second threshold value, the second living body detection unit 2120 determines that there is not a sufficient difference between the face direction and the gaze direction.

[0147] By making such a determination as to both the left and right directions, it is possible to prevent spoofing using a 3D mask or the like with higher accuracy. For example, suppose that the gaze direction depicted in a certain 3D mask is significantly biased to the right. Then, suppose that the target object 50 wears this 3D mask and attempts to spoof. In this case, when the target object 50 turns its face to the left, the face turns to the left while the gaze turns to the right, so the difference between the face direction and the gaze direction becomes large. Therefore, when the user turns to the left, it may be difficult to detect spoofing based on the difference between the face direction and the gaze direction. On the other hand, in this case, when the user turns to the right, both the face and the gaze turn to the right, so the difference between the face direction and the gaze direction becomes small. Therefore, when the user turns to the right, it is possible to detect spoofing based on the difference between the face direction and the gaze direction. From the above, it is preferable to have the target object 50 turn to both the left and right.

[0148] The living body detection device 2000 may make the target object 50 perform a series of movements of shaking its face from side to side a plurality of times, and determine whether or not there is a sufficient difference between the face direction and the gaze direction for each of the plurality of movements. The series of movements of shaking the face from side to side is a series of movements of "turning the face left and then turning the face right" or a series of movements of "turning the face right and then turning the face left". For example, the second living body detection unit 2120 treats a series of movements of shaking the face from side to side as one set, and detects a plurality of sets of movements by analyzing a plurality of captured images 40. The second living body detection unit 2120 calculates either one or both of the maximum and minimum values ​​of the difference between the face direction and the gaze direction for each set, and determines whether or not the target object 50 is a living body according to the calculation result.

[0149] For example, the second living body detection unit 2120 determines whether or not there is a sufficient difference between the face direction and the gaze direction for each of a predetermined number n (n is an integer) of sets. Then, the second living body detection unit 2120 determines whether or not the number of sets determined to have a sufficient difference between the face direction and the gaze direction is equal to or greater than a predetermined number m (m is an integer between 1 and n). If the number of sets determined to have a sufficient difference between the face direction and the gaze direction is equal to or greater than m, the second living body detection unit 2120 determines that the target object 50 is a living body. On the other hand, if the number of sets determined to have a sufficient difference between the face direction and the gaze direction is less than the predetermined number m, the second living body detection unit 2120 determines that the target object 50 is not a living body.

[0150] As another example, the second living body detection unit 2120 may determine whether or not there is a sufficient difference between the face direction and the gaze direction for each set detected within a specified time, and determine that the target object 50 is a living body if the number of sets determined to have a sufficient difference between the face direction and the gaze direction is m or more.

[0151] Alternatively, for example, the second living body detection unit 2120 may have the target object 50 repeat the motion of shaking its face from side to side until the number of sets determined to have a sufficient difference between the face direction and the gaze direction reaches m or more. In this case, the second living body detection unit 2120 determines whether or not there is a sufficient difference between the face direction and the gaze direction for each series of motions in which the target object 50 shakes its face from side to side. If it is determined that there is a sufficient difference between the face direction and the gaze direction, 1 is added to the counter. If the counter reaches m or more, the second living body detection unit 2120 determines that the target object 50 is a living body. On the other hand, if the counter does not reach m or more (for example, if a predetermined time has passed before the counter reaches m or more), the second living body detection unit 2120 determines that the target object 50 is not a living body.

[0152] In addition, if the face shaking action is not performed correctly, the counter may be cleared to 0. Cases in which the face shaking action is not performed correctly include 1) a case in which the face is not turned sufficiently left or right (for example, in the example of FIG. 25, the face is turned to the right before the black circle overlaps with the black star), and 2) a case in which it is determined that there is not enough difference between the face direction and the line of sight direction. When the counter is cleared in this way, the target object 50 is determined to be a living organism only when the face shaking action is performed correctly m times in succession. On the other hand, the target object 50 is not determined to be a living organism until the face shaking action is performed correctly m times in succession. In addition, an upper limit may be set for the number of times the counter is cleared, and when the number of times the counter is cleared reaches the upper limit, the target object 50 may be determined to be a non-living organism. Also, a time limit may be set for the time to perform the action, and when the face shaking action is not performed correctly m times in succession within the time limit, the target object 50 may be determined to be a non-living organism.

[0153] Instead of treating the action of turning the face to the right and the action of turning the face to the left as a set, these actions may be counted individually. In this case, the number of times to perform the action of turning the face to the right and the action of turning the face to the left (the above-mentioned predetermined number m) may be the same or different. In the latter case, for example, it is possible to "make the subject turn the face to the left three times and the action of turning the face to the right twice."

[0154] When the user is caused to perform the action of shaking the face from side to side multiple times, the display control unit 2020 may cause the display device 20 to display the number of times the action has been performed correctly. Furthermore, each time the action is performed, the display device 20 may display an indication (such as a circle or OK) indicating the determination result as to whether the action has been performed correctly or not.

[0155] <Guide to the posture of the target object 50> When performing liveness detection using the difference between the face direction and the gaze direction, it is preferable that the posture of the target object 50 is suitable for liveness detection. A posture suitable for liveness detection is, for example, a posture that makes it easy to swing the face from side to side. When swinging the face from side to side in a posture that makes it easy to swing the face from side to side, the amplitude of the face movement is likely to be larger than when swinging the face from side to side in a posture that does not make it easy to swing the face from side to side. For example, compared to a posture of looking up or down, a posture in which the face is facing horizontally makes it easier to swing the face from side to side, so the amplitude of the face movement when swinging the face from side to side is likely to be larger. And, when the amplitude of the face movement is larger, the difference between the face direction and the gaze direction becomes larger when the target object 50 is a living organism. For this reason, when liveness detection is performed in a situation in which the target object 50 is in a posture that makes it easy to swing the face from side to side, the difference between the face direction and the gaze direction becomes clearer between a case in which the target object 50 is a living organism and a case in which it is not, and the accuracy of liveness detection is improved.

[0156] Therefore, the living body detection device 2000 may further include a third display control unit (not shown) that performs the following process. The third display control unit determines whether or not the posture of the target object 50 is such that it is easy to turn the head from side to side by analyzing the captured image 40. Then, if the posture is not such that it is easy to turn the head from side to side, the third display control unit causes the display device 20 to display a guide for guiding the target object to take a posture that makes it easy to turn the head from side to side.

[0157] For example, the third display control unit determines whether or not the face direction of the target object 50 in the vertical plane is included within a predetermined range for the target object 50 in each captured image 40. If the face direction of the target object 50 in the vertical plane is not included within the predetermined range, the third display control unit displays a guide so that the face direction in the vertical plane is an appropriate direction (a direction within the predetermined range). The face direction in the vertical plane can be expressed, for example, with the horizontal direction as the reference 0 degrees, with a direction above the horizontal direction as a positive angle, and a direction below the horizontal direction as a negative angle.

[0158] FIG. 26 is a diagram for explaining the direction of the face of the target object 50 on a vertical plane. In FIG. 26, the vertical plane is the xz plane. The predetermined range is ±β. In the case in the upper part of FIG. 26, the face of the target object 50 faces horizontally. Therefore, the direction of the face on the vertical plane is within the predetermined range. Therefore, the posture of the face of the target object 50 is such that it is easy to turn the face from side to side.

[0159] On the other hand, in the case in the lower part of Fig. 26, the face of the target object 50 is in a pose looking up, and the direction of the face in the vertical plane is larger than β. Therefore, in the case in the lower part of Fig. 26, the third display control unit displays a guide so that the direction of the face in the vertical plane is an appropriate direction (a direction within a predetermined range).

[0160] The guide may be the same or different before and after the target object 50 starts to shake its head from side to side. In the former case, for example, a guide such as "Keep your face horizontal" may be displayed.

[0161] In the latter case, before the target object 50 starts to shake its face from side to side, it is preferable that the face of the target object 50 is not moving from side to side or up and down, that is, facing forward (looking straight at the camera). Therefore, for example, before the face starts to move from side to side (for example, when the screen 90-1 is displayed in the example of FIG. 17), the third display control unit causes the display device 20 to display a guide such as "Please face forward" or "Please face the camera" to encourage the user to face forward.

[0162] On the other hand, while the user is shaking his / her face from side to side, it is preferable to encourage the user to move the face horizontally. Therefore, after the user starts to move the face from side to side (for example, after the screen 90-2 is displayed in the example of FIG. 17), the third display control unit causes the display device 20 to display a guide such as "Shake your face horizontally" to encourage the user to move the face horizontally.

[0163] The living body detection device 2000 may take into consideration not only the face of the target object 50 but also the posture of the torso. For example, when the back is straightened, the face is more likely to turn from side to side than when the back is hunched. For example, the third display control unit uses the captured image 40 to determine whether the relationship between the posture of the face and the posture of the torso of the target object 50 satisfies a predetermined condition, and when the predetermined condition is not satisfied, causes the display device 20 to display a guide encouraging the target object 50 to adopt a posture that satisfies the predetermined condition. For example, the predetermined condition is a condition that is satisfied when the back is straightened. It should be noted that existing technology can be used for the technology for determining whether the person's back is straightened using an image in which the person is photographed.

[0164] More specifically, the third display control unit detects the joint points of the head, neck, and waist of the target object 50 by analyzing the image of the body of the target object 50 shown in the captured image 40. Then, the determination is performed using the condition that "the magnitude of the angle formed by the line connecting the joint points of the face and the joint points of the neck (face posture) and the line connecting the joint points of the neck and the joint points of the waist (torso posture) is equal to or less than a predetermined threshold value" as the above-mentioned predetermined condition. That is, the third display control unit calculates the magnitude of the angle formed by the line connecting the joint points of the head and the joint points of the neck and the line connecting the joint points of the neck and the joint points of the waist. Then, when the magnitude of the angle is equal to or less than a predetermined threshold value, the third display control unit determines that the relationship between the posture of the face and the posture of the torso of the target object 50 satisfies the predetermined condition. On the other hand, when the magnitude of the angle is greater than the threshold value, the third display control unit determines that the relationship between the posture of the face and the posture of the torso of the target object 50 does not satisfy the predetermined condition. In addition, existing technology can be used to detect the joint points of a person by analyzing an image in which the person appears.

[0165] When the posture of the torso of the target object 50 is taken into consideration, it is necessary that the captured image 40 includes not only the face of the target object 50 but also the neck, torso, etc. Therefore, it is preferable that the third display control unit further outputs a guide for obtaining the captured image 40 including the neck, torso, etc. For example, the third display control unit causes the display device 20 to display a guide for capturing an image of the entire body, such as "Please move the camera away from the subject and capture the entire body" or "Please move the camera up and down to capture the entire body."

[0166] In the above example, it is determined whether the relationship between the facial posture and the torso posture is appropriate when the target object 50 is viewed from the front. In addition to or instead of this determination, the third display control unit may determine whether the relationship between the facial posture and the torso posture is appropriate when the target object 50 is viewed from the side. In this case, the third display control unit causes the display device 20 to display a guide such as "Please take a picture of the side of the body", thereby causing the target object 50 to capture the side of the body. As a result, a captured image 40 in which the side of the target object 50 is captured is obtained. Using this captured image 40, the third display control unit determines whether the relationship between the facial posture and the torso posture when the target object 50 is viewed from the side satisfies a predetermined condition. The above-mentioned joint points can also be used for this determination.

[0167] <Terminal 60 Posture Guide> In order to capture an image of the face of the target object 50 from the front, it is preferable to have the terminal 60 in an upright position (a position in which the front direction of the camera is horizontal). Therefore, for example, the living body detection device 2000 may further include a fourth display control unit (not shown) that performs the following process. The fourth display control unit calculates the magnitude of the tilt of the terminal 60 in the vertical plane, and determines whether the magnitude is equal to or less than a predetermined threshold. If the calculated magnitude of the tilt is not equal to or less than the threshold, the fourth display control unit causes the display device 20 to display a guide for correcting the attitude of the terminal 60. The attitude of the terminal 60 can be calculated, for example, by using an acceleration sensor or a gyro sensor provided in the terminal 60.

[0168] Here, when the terminal 60 is used in a portrait orientation, the magnitude of the inclination of the terminal 60 in the vertical plane can be calculated as the angle that the long side direction of the terminal 60 (the long side direction of the display surface of the display device 20) forms with the vertical direction. On the other hand, when the terminal 60 is used in a landscape orientation, the magnitude of the inclination of the terminal 60 in the vertical plane can be calculated as the angle that the short side direction of the terminal 60 (the short side direction of the display surface of the display device 20) forms with the vertical direction.

[0169] 27 is a diagram illustrating a guide for ensuring that the terminal 60 is in an appropriate position. In this example, the position of the terminal 60 is represented by the angle between the longitudinal direction of the terminal 60 and the vertical direction in a vertical plane (xz plane). This angle is positive when the display surface of the display device 20 faces upward, and negative when the display surface faces downward. Also, a straight line 120 representing the position of the terminal 60 is shown on the display device 20. Furthermore, straight lines 130 and 140 representing the range of target positions are also shown.

[0170] The smaller the angle between the longitudinal direction and the vertical direction of the terminal 60, the lower the straight line 120 is displayed on the display surface of the display device 20. Therefore, when the straight line 120 is above the straight line 130 or below the straight line 140, the target person moves the position of the terminal 60 closer to an upright position so that the position of the straight line 120 is between the straight lines 130 and 140.

[0171] Furthermore, the liveness detection device 2000 may detect that the attitude of the terminal 60 has changed significantly while liveness detection is being performed (the target object 50 is shaking its head from side to side), and may redo the liveness detection in response to the detection. At this time, it is preferable for the liveness detection device 2000 to display on the display device 20 a notification that liveness detection should be redoed, such as "Authentication will be performed again. Please do not move the terminal 60," and a guide indicating that the terminal 60 should not be moved.

[0172] <Modification 1 of the second embodiment> In the living body detection device 2000 of the second embodiment, the display device 20 may be moved left and right while the gaze is fixed at a specific position on the display device 20 (for example, a cross mark in FIG. 14, etc.) without moving the face. FIG. 21 is a diagram illustrating the movement of the target object 50 in the living body detection device of the first modification of the second embodiment. In this case, too, if the front direction of the display device 20 is taken as the reference direction (0°), the face direction and gaze direction will change in the same manner as the graph shown in FIG. 13. Therefore, in the living body detection device 2000 of this modification, the difference between the face direction and gaze direction of the target object 50 is calculated for each of the multiple captured images 40, and it is possible to determine whether or not the target object 50 is a living body based on the difference.

[0173] The specific method of determining whether the target object 50 is a living organism is as described above. However, instead of detecting the motion of shaking the face from side to side, the living organism detection device 2000 detects the motion of shaking the display device 20 from side to side. The motion of shaking the display device 20 from side to side can be detected based on a change in the direction of the face relative to the display device 20. For example, whether the display device 20 has been moved sufficiently to the right can be determined by determining whether the direction of the face relative to the display device 20 has moved sufficiently to the left. Similarly, whether the display device 20 has been moved sufficiently to the left can be determined by determining whether the direction of the face relative to the display device 20 has moved sufficiently to the right.

[0174] Here, in the living body detection device 2000 of this modification, the guide information may include information different from that in the case of the embodiment 2. Specifically, the living body detection device 2000 displays on the display device 20 guide information that prompts the user to shake the terminal 60 from side to side, rather than prompting the user to shake the face from side to side. It is also preferable to display guide information that prompts the user not to move the face.

[0175] 22 is a diagram illustrating an example of guide information in the living body detection device 2000 of the first modified example of the second embodiment. In this example, first, a screen 100-1 is displayed. On the screen 100-1, a message is displayed encouraging the user to hold the terminal 60 (display device 20) in front of the user without moving the face and with the user's gaze fixed on a cross mark on the display device 20.

[0176] After that, screen 100-2 is displayed. A message is displayed on screen 100-2, prompting the user to move terminal 60 to the left without moving the face and with the gaze fixed on the cross mark on display device 20. Screen 100-2 is displayed in response to, for example, detection that both the gaze direction and the face direction of target object 50 are facing forward with respect to display device 20.

[0177] After that, screen 100-3 is displayed. A message is displayed on screen 100-3, prompting the user to move terminal 60 to the right without moving the face and with the line of sight fixed on the cross mark on display device 20. Screen 100-3 is displayed, for example, in response to detection that terminal 60 has been moved sufficiently to the left while the face is not moved and the line of sight is fixed on the cross mark on display device 20. Determination of whether terminal 60 has been moved sufficiently to the left can be realized, for example, by determining whether the face direction has been turned sufficiently to the right (determination of whether the face direction has become equal to or greater than a positive threshold).

[0178] When the terminal 60 is repeatedly moved left and right, the screen 100-2 is displayed again. The screen 100-2 is displayed, for example, in response to detection that the terminal 60 has been moved sufficiently to the right while the face is not moved and the gaze is fixed on the cross mark on the display device 20. The determination of whether the terminal 60 has been moved sufficiently to the right can be realized, for example, by determining whether the face direction has been turned sufficiently to the left (determining whether the face direction is equal to or less than a negative threshold).

[0179] <Modification 2 of the second embodiment> In the living body detection device 2000 of the second embodiment, the gaze may be moved from side to side while the display device 20 and the face are kept fixed (without moving the display device 20 and with the face facing in front of the display device 20). Fig. 23 is a diagram illustrating an example of the movement of the target object 50 in the living body detection device 2000 of the second modification of the second embodiment.

[0180] This case corresponds to the reverse of the handling of the gaze direction and the face direction in the live body detection device 2000 of the second embodiment. That is, instead of fixing the gaze direction, the face direction is fixed, and instead of shaking the face left and right, the gaze is shaken left and right. Also, in the guide information, the same guide as that performed for the gaze direction in the live body detection device 2000 of the second modification of the second embodiment is performed for the face direction in the live body detection device 2000 of the second modification of the second embodiment. That is, instead of a guide to fix the gaze direction, a guide to fix the face direction is displayed. Furthermore, the same guide as that performed for the face direction in the live body detection device 2000 of the second modification of the second embodiment is performed for the gaze direction in the live body detection device 2000 of the second modification of the second embodiment. That is, instead of a guide to turn the face direction to the front or left or right, a guide to turn the gaze to the front or left or right is displayed.

[0181] 24 is a diagram illustrating an example of guide information in the living body detection device 2000 according to the second modification of the second embodiment. In this example, first, a screen 110-1 is displayed. A message is displayed on the screen 110-1 to prompt the user to direct the line of sight to the front of the display device 20 without moving the face.

[0182] After that, screen 110-2 is displayed. A message is displayed on screen 110-2, prompting the user to move the line of sight to the left without moving the face. Screen 110-2 is displayed in response to, for example, detection that both the line of sight direction and the face direction of target object 50 face forward with respect to display device 20.

[0183] After that, screen 110-3 is displayed. A message is displayed on screen 110-3, prompting the user to move the line of sight to the right without moving the face. Screen 110-3 is displayed, for example, in response to detection that the line of sight has been moved sufficiently to the left (the line of sight direction has become equal to or less than a negative threshold) with the face direction fixed forward. Note that screen 110-3 is viewed with the line of sight directed to the left, so the rightward arrow is displayed in a position closer to the left.

[0184] If the gaze is repeatedly moved left and right, screen 110-2 is displayed again. Screen 110-2 is displayed, for example, in response to detection that the gaze has been moved sufficiently to the right (the gaze direction has reached a positive threshold value or more) while the face direction is fixed forward. However, since the image is viewed with the gaze directed to the right, it is preferable that the leftward arrow be displayed in a position closer to the right.

[0185] <Combination with embodiment 1> The living body detection device 2000 of the first embodiment and the living body detection device 2000 of the second embodiment perform living body detection by different methods. Therefore, it is preferable that both the living body detection by the living body detection device 2000 of the first embodiment and the living body detection by the living body detection device 2000 of the second embodiment (which may be the first or second modification) are performed for the same target object 50. FIG. 25 is a flowchart illustrating a process flow in the case where the living body detection by the living body detection device 2000 of the first embodiment and the living body detection by the living body detection device 2000 of the second embodiment are performed in sequence. In the example of FIG. 25, the living body detection by the living body detection device 2000 of the first embodiment (a series of processes shown in the flowchart of FIG. 4) is performed, and then the living body detection by the living body detection device 2000 of the second embodiment (a series of processes shown in the flowchart of FIG. 16) is performed. However, the living body detection by the living body detection device 2000 of the first embodiment may be performed after the living body detection by the living body detection device 2000 of the second embodiment.

[0186] For example, these liveness detections may be performed as part of user authentication. For example, one method of user authentication is to capture a user's face with a camera and compare the captured face with a face registered in advance to determine whether the user is a registered user. In this case, in order to prevent another person from impersonating a registered user by using a photo of the registered user, it is preferable to confirm that the object captured by the camera is a live body, not a photo. Therefore, in addition to confirming that the target object 50 is a registered user by comparing the image of the target object 50 with the image of the registered user, it is preferable to confirm that the target object 50 is a live body (i.e., that impersonation using a photo, a 3D mask, or the like has not been performed) using the liveness detection device 2000. Note that the process of confirming that the target object 50 is a registered user and the process of confirming that the target object 50 is a live body may be performed in either order, or may be performed in parallel.

[0187] <How to prevent replacement of target object 50> When liveness detection is performed using a plurality of methods, it is preferable to ensure that the processing is performed on the same target object 50 from start to finish. For example, when liveness detection is performed by the liveness detection device 2000 of the first embodiment and then by the liveness detection device 2000 of the second embodiment, the same target object 50 is set as the target of liveness detection from the start of liveness detection by the liveness detection device 2000 of the first embodiment to the completion of liveness detection by the liveness detection device 2000 of the second embodiment. In this way, the target of liveness detection is switched for each type of liveness detection method, thereby making it possible to prevent the use of spoofing means that are difficult to prevent by that method.

[0188] For example, the living body detection device 2000 performs the following process. First, as a premise, the living body detection device 2000 executes two or more types of living body detection processes. For example, it is assumed that the living body detection process performed by the living body detection device 2000 of the first embodiment (hereinafter, the first living body detection process) and the living body detection process performed by the living body detection device 2000 of the second embodiment (hereinafter, the second living body detection process) are performed. In this case, the living body detection device 2000 repeatedly determines whether or not the same target object 50 is continuously captured by the camera 10 in parallel with the process for living body detection from the start of the first living body detection process (from the first start of S102 in FIG. 25) until the second living body detection process is completed (until S208 in FIG. 25 is completed). Then, when it is determined that there is a possibility that the same target object 50 is not continuously captured by the camera 10, the living body detection device 2000 ends the living body detection process.

[0189] The determination of whether the same target object 50 is continuously captured by the camera 10 can be realized, for example, by tracking the target object 50. Specifically, the living body detection device 2000 tracks the target object 50 by analyzing the captured image 40 generated by the camera 10 in a time series. The living body detection device 2000 repeatedly determines whether the target object 50 is included in the captured image 40 based on the tracking. Then, if the target object 50 is no longer included in the captured image 40 before a series of live body detection processes are completed, the living body detection device 2000 determines that the same target object 50 is not continuously captured by the camera 10. On the other hand, while the target object 50 is continuously included in the captured image 40, it is determined that the same target object 50 is continuously captured by the camera 10. In the tracking, the area in the captured image 40 to be searched for detecting the target object 50 is limited to the vicinity of the area detected previously. Therefore, there is an advantage that the processing load required for detecting the target object 50 is reduced.

[0190] Alternatively, for example, the live body detection device 2000 may register an image feature of the target object 50 at the start of the live body detection process, and determine whether the target object 50 continues to be captured by the camera 10 by checking that the image feature continues to be included in the captured image 40. In this case, the live body detection device 2000 detects the target object 50 from the captured image 40 before a series of live body detection processes is started (for example, before S102 in FIG. 25 is started for the first time), and registers (stores in a storage device) the image feature of the target object 50. The live body detection device 2000 detects the image feature of the registered target object 50 from the captured image 40 obtained thereafter.

[0191] The process of detecting the image feature of the target object 50 from the captured images 40 may be performed on all the captured images 40, or may be performed on some of the captured images 40. In the latter case, for example, the living body detection device 2000 detects the image feature of the target object 50 from the captured images 40 at a predetermined time interval (predetermined number of times). In addition, for example, the living body detection device 2000 detects the image feature of the target object 50 from the captured images 40 obtained at that time in response to a specific condition being satisfied. In the case of the living body detection device 2000 of the first embodiment, the specific condition is, for example, that the screen 30 displayed on the display device 20 has been changed. In addition, in the case of the living body detection device 2000 of the second embodiment, the specific condition is, for example, any one or more of the following conditions: the face faces forward (the face direction becomes 0°), the face faces left (the face direction becomes equal to or less than a negative threshold), the face faces right (the face direction becomes equal to or more than a positive threshold), and the face is moving.

[0192] If the image feature amount of the target object 50 is no longer detected from the captured image 40 before the series of live body detection processes are completed, the live body detection device 2000 determines that the same target object 50 is not continuing to be imaged by the camera 10. On the other hand, if the image feature amount of the target object 50 continues to be detected from the captured image 40 until the series of live body detection processes are completed, it is determined that the same target object 50 is continuing to be imaged by the camera 10. According to the method of detecting the image feature amount of the target object 50 from the captured image 40, even if an object other than the target object 50 can be imaged by the camera 10, it is possible to accurately determine whether or not the target object 50 is continuing to be imaged by the camera 10.

[0193] When it is determined that the camera 10 is not continuously capturing the same target object 50, it is preferable that the live body detection device 2000 takes some kind of action. For example, the live body detection device 2000 restarts the live body detection of the target object 50 from the beginning (for example, from S102 in FIG. 25). At this time, it is preferable to make the display device 20 display a message such as "Please continue to capture yourself with the camera 10" to make the user aware that he or she must continue to capture himself or herself with the camera 10. As another example, the live body detection device 2000 may display on the display device 20 information indicating which of multiple types of live body detection was performed when it was determined that the camera 10 was not continuously capturing the same target object 50.

[0194] In addition, when the liveness detection process is redone a predetermined number of times or more (even once), the liveness detection process may be prevented from being performed any more. In addition, a process for protecting the account of the user who is the target of impersonation may be performed. For example, as described above, liveness detection is performed as a part of user authentication. In this case, for example, it is assumed that person A attempts to impersonate registered user B. Specifically, it is assumed that person A causes the camera 10 to capture a photo of user B, and the person to be authenticated is determined to be user B, and then liveness detection is performed by the liveness detection device 2000. Then, it is assumed that the liveness detection is redone a predetermined number of times or more due to tracking of the target object 50 described above. In this case, for example, the liveness detection device 2000 protects the account of user B by locking the account of user B or sending a warning notification to the mobile terminal of user B. In this way, it is possible to more appropriately prevent damage caused by impersonation.

[0195] When a plurality of human faces are detected from the captured image 40, the living body detection device 2000 may perform the same process as that performed when the same target object 50 is not continuously imaged.

[0196] Although the present invention has been described above with reference to the embodiment, the present invention is not limited to the above embodiment. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0197] In the above example, the program can be stored and provided to the computer using various types of non-transitory computer readable media. The non-transitory computer readable medium includes various types of tangible storage media. Examples of the non-transitory computer readable medium include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs, CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs). The program may also be provided to the computer by various types of transitory computer readable media. Examples of the transitory computer readable medium include electrical signals, optical signals, and electromagnetic waves. The transitory computer readable medium can supply the program to the computer via a wired communication path such as an electric wire or optical fiber, or via a wireless communication path.

[0198] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes. (Appendix 1) A computer-readable medium having a program executed by a computer stored thereon, The program causes the computer to a display control step of displaying on a display device a first screen including a display in each of a plurality of regions and a second screen including a display different from the first screen; an acquisition step of acquiring a plurality of captured images generated by capturing an image of an object with a camera while the first screen and the second screen are being displayed, respectively; a live body detection step of determining whether or not the target is a live body using a plurality of the captured images; The first screen includes at least two regions that are displayed differently from each other. (Appendix 2) A computer-readable medium as described in Appendix 1, wherein at least two of the multiple regions included in the first screen display different colors, different brightnesses, or different letters, symbols, or figures. (Appendix 3) 3. The computer-readable medium of claim 1, wherein in the display control step, boundaries of the multiple areas included in the first screen are determined based on a positional relationship between a camera that generates the captured image and the display device. (Appendix 4) 3. The computer-readable medium of claim 1, wherein the boundaries of the multiple regions included in the first screen are determined by user input. (Appendix 5) the program has a discrimination model trained to output a label indicating whether the object shown in the captured image is a living body or not in response to input of a plurality of the captured images; A computer-readable medium described in any one of Appendix 1 to 4, wherein in the display control step, the multiple captured images acquired in the acquisition step are input into the identification model, and the label is obtained from the identification model to determine whether the object is a living body or not. (Appendix 6) The discriminative model is trained using both positive and negative training data; the positive example training data includes a plurality of captured images obtained by capturing an image of a face of a real person while the first screen and the second screen are being displayed, and data indicating that the captured object is a living body; The computer-readable medium of claim 5, wherein the negative example training data includes a plurality of captured images obtained by capturing images of a human face while the first screen and the second screen are each displayed, and data indicating that the captured subject is not a living body. (Appendix 7) A computer-readable medium according to any one of appendixes 1 to 6, wherein in the display control step, the intensity of ambient light surrounding the display device is identified, and when the intensity of the ambient light is below a threshold, the first screen and the second screen are displayed on the display device. (Appendix 8) The computer-readable medium of claim 7, wherein, in the display control step, if the ambient light intensity is greater than a threshold, a message is output encouraging the user to move to a location where the ambient light is weaker than the current intensity. (Appendix 9) A computer-readable medium according to any one of appendixes 1 to 8, wherein in the display control step, an intensity of ambient light around the display device is identified, and a display to be included on at least one of the first screen and the second screen is determined according to the intensity of the ambient light. (Appendix 10) a second display control step of causing the display device to display a guide regarding a line of sight direction and a guide regarding a face direction; A second acquisition step of acquiring captured images generated by capturing images of the object multiple times with the camera; a second live body detection step of calculating a difference between a face direction and a gaze direction for each of the captured images using an image of the face of the target appearing in each of the captured images, and determining whether or not the target is a live body based on the calculated difference; A computer-readable medium according to any one of appendix 1 to 9, wherein in the second display control step, a guide is displayed on the display device encouraging the user to shake their head left and right while continuing to look at the same place. (Appendix 11) In the second display control step, displaying on the display device a guide to direct the face in a first direction without moving the line of sight; determining whether the direction of the face of the subject is the first direction; 11. The computer-readable medium of claim 10, further comprising: displaying on the display device a guide encouraging the subject to turn the face in a second direction without moving the line of sight if the subject's face direction is in the first direction. (Appendix 12) The computer-readable medium of claim 10 or 11, wherein in the second display control step, it is determined whether the magnitude of the target's facial movement is equal to or greater than a threshold, and if the magnitude of the facial movement is not equal to or greater than the threshold, a guide encouraging the user to move the face more widely is displayed on the display device. (Appendix 13) A computer-readable medium according to any one of appendices 10 to 12, wherein in the second display control step, it is determined whether the speed at which the target's face moves is greater than or equal to a threshold, and if the speed at which the face moves is less than or equal to the threshold, a guide encouraging the target to move the face more slowly is displayed on the display device. (Appendix 14) A computer-readable medium according to any one of appendices 10 to 13, wherein in the second display control step, it is determined whether the gaze direction of the subject is fixed, and if the gaze direction is not fixed, a guide urging the subject not to move the gaze direction is displayed on the display device. (Appendix 15) A computer-readable medium according to any one of appendices 10 to 14, wherein in the second display control step, the display device is caused to display one or more of an indication indicating the direction of the subject's face, an indication indicating the direction the subject should face, an indication indicating the direction of the subject's gaze, and an indication indicating the direction the subject should gaze. (Appendix 16) determining whether or not the same object is continuously imaged by the camera using captured images generated by the camera during the period from when display of the screen is started by the display control step to when liveness detection is performed by the second liveness detection step; 16. The computer-readable medium of any one of claims 10 to 15, wherein if the same object is not continuing to be imaged, processing is restarted from the display control step. (Appendix 17) By tracking the object, it is determined whether the object is captured in each of the captured images; 17. The computer-readable medium of claim 16, wherein if an image is detected in which the object is not captured, it is determined that the same object is not being continuously captured. (Appendix 18) Detecting image features of the object from each of the captured images; 17. The computer-readable medium of claim 16, wherein if an image capture image is detected that does not include image features of the object, it is determined that the same object is not being continuously captured. (Appendix 19) A computer-readable medium according to any one of appendices 10 to 18, which causes the computer to execute a third display control step of determining, using the captured image, whether or not the posture of the subject is suitable for liveness detection, and, if the posture of the subject is not suitable for liveness detection, displaying on the display device a guide encouraging the subject to assume an appropriate posture. (Appendix 20) In the third display control step, determining whether a direction of the face of the subject in a vertical plane is within a predetermined range using an image of the face of the subject captured in the captured image; 20. The computer-readable medium of claim 19, further comprising causing the display device to display a guide encouraging the subject to adopt an appropriate posture if the direction of the subject's face in a vertical plane is not within the predetermined range. (Appendix 21) The computer-readable medium of claim 20, wherein in the third display control step, different guides are displayed depending on whether the subject is making a head-waving motion or not. (Appendix 22) In the third display control step, determining whether or not a facial posture and a torso posture of the subject satisfy a predetermined condition using an image of the face and torso of the subject captured in the captured image; 20. The computer-readable medium of claim 19, further comprising: displaying, on the display device, a guide prompting the user to adopt an appropriate posture if the predetermined condition is not met. (Appendix 23) A computer-readable medium according to any one of appendices 10 to 22, which causes the computer to execute a fourth display control step of determining whether or not a magnitude of tilt in a vertical plane of a terminal on which the display device is installed is equal to or less than a predetermined threshold, and, if the magnitude of the tilt is not equal to or less than the predetermined threshold, displaying on the display device a guide encouraging the user to adjust the terminal to an appropriate position. (Appendix 24) a display control unit that causes a display device to display a first screen including a display in each of a plurality of regions and a second screen including a display different from the first screen; an acquisition unit that acquires a plurality of captured images generated by capturing an image of an object with a camera while the first screen and the second screen are being displayed; a living body detection unit that determines whether the target is a living body by using a plurality of the captured images, The first screen includes at least two areas that are displayed differently from each other. (Appendix 25) A biological detection device as described in Appendix 24, wherein at least two of the multiple areas included in the first screen display different colors, different brightnesses, or different letters, symbols, or figures. (Appendix 26) The biological detection device according to claim 24 or 25, wherein the display control unit determines the boundaries of the multiple areas included in the first screen based on a positional relationship between a camera that generates the captured image and the display device. (Appendix 27) 26. The biosensing device according to claim 24, wherein boundaries of the multiple regions included in the first screen are determined by user input. (Appendix 28) a classification model that is trained to output a label indicating whether the object shown in the captured image is a living body or not in response to input of a plurality of the captured images; The display control unit is configured to input the acquired multiple captured images into the identification model and obtain the label from the identification model, thereby determining whether the target is a living body or not. (Appendix 29) The discriminative model is trained using both positive and negative training data; the positive example training data includes a plurality of captured images obtained by capturing an image of a face of a real person while the first screen and the second screen are being displayed, and data indicating that the captured object is a living body; The negative example training data of the live body detection device described in Appendix 28 includes a plurality of captured images obtained by capturing images of a human face while the first screen and the second screen are each displayed, and data indicating that the captured subject is not a live body. (Appendix 30) The display control unit determines the intensity of ambient light surrounding the display device, and when the intensity of the ambient light is below a threshold, causes the display device to display the first screen and the second screen. (Appendix 31) The biological detection device of claim 30, wherein the display control unit outputs a message encouraging the user to move to a location where the ambient light is weaker than the current intensity when the ambient light intensity is greater than a threshold value. (Appendix 32) 32. The biosensing device of any one of appendices 24 to 31, wherein the display control unit identifies an intensity of ambient light around the display device, and determines a display to be included in at least one of the first screen and the second screen depending on the intensity of the ambient light. (Appendix 33) a second display control unit that causes the display device to display a guide regarding a line of sight direction and a guide regarding a face direction; A second acquisition unit that acquires captured images generated by capturing an image of the object multiple times with the camera; a second living body detection unit that calculates a difference between a face direction and a gaze direction for each of the captured images using an image of the face of the target appearing in each of the captured images, and determines whether or not the target is a living body based on the calculated difference; The biosensing device according to any one of appendices 24 to 32, wherein the second display control unit causes the display device to display a guide encouraging the user to shake their head from side to side while continuing to look at the same spot. (Appendix 34) The second display control unit is displaying on the display device a guide to direct the face in a first direction without moving the line of sight; determining whether the direction of the face of the subject is the first direction; The biosensing device of claim 33, further comprising: a display device that displays a guide encouraging the subject to turn their face in a second direction without moving their line of sight when the subject's face is in the first direction. (Appendix 35) The second display control unit determines whether the magnitude of the target's facial movement is greater than or equal to a threshold, and if the magnitude of the facial movement is not greater than or equal to the threshold, displays on the display device a guide encouraging the target to move the face more widely. (Appendix 36) The second display control unit determines whether the speed at which the target's face moves is greater than or equal to a threshold, and if the speed at which the target's face moves is less than or equal to the threshold, causes the display device to display a guide encouraging the target to move the face more slowly. (Appendix 37) The second display control unit determines whether the gaze direction of the subject is fixed or not, and if the gaze direction is not fixed, causes the display device to display a guide urging the subject not to move the gaze direction. (Appendix 38) The biometric detection device of any one of appendices 33 to 37, wherein the second display control unit causes the display device to display one or more of the following: a display indicating the direction of the subject's face, a display indicating the direction the subject should face, a display indicating the direction of the subject's gaze, and a display indicating the direction the subject should gaze. (Appendix 39) determining whether or not the same object is continuously imaged by the camera using captured images generated by the camera during the period from when the display control unit starts displaying a screen to when the second live body detection unit performs live body detection; 39. The biological detection device according to any one of appendices 33 to 38, wherein if the same object is not continuously imaged, the display control unit restarts processing. (Appendix 40) By tracking the object, it is determined whether the object is captured in each of the captured images; 40. The biological detection device of claim 39, wherein when an image in which the object is not captured is detected, it is determined that the same object is not being continuously captured. (Appendix 41) Detecting image features of the object from each of the captured images; 40. The biological detection device of claim 39, wherein when an image that does not include image features of the object is detected, it is determined that the same object is not being continuously imaged. (Appendix 42) A live body detection device as described in any one of Appendices 33 to 41, which causes the computer to execute a third display control unit that uses the captured image to determine whether the posture of the subject is suitable for live body detection, and if the posture of the subject is not suitable for live body detection, displays on the display device a guide encouraging the subject to assume an appropriate posture. (Appendix 43) The third display control unit is determining whether a direction of the face of the subject in a vertical plane is within a predetermined range using an image of the face of the subject captured in the captured image; 43. The biosensing device of claim 42, wherein, if the direction of the subject's face in a vertical plane is not within the specified range, a guide is displayed on the display device encouraging the subject to assume an appropriate posture. (Appendix 44) The liveness detection device of claim 43, wherein the third display control displays different guides when the subject is making a head-shaking motion and when the subject is not making a head-shaking motion. (Appendix 45) The third display control unit is determining whether or not a facial posture and a torso posture of the subject satisfy a predetermined condition using an image of the face and torso of the subject captured in the captured image; 43. The biosensing device of claim 42, wherein, if the specified condition is not satisfied, a guide is displayed on the display device encouraging the user to assume an appropriate posture. (Appendix 46) A biological detection device as described in any one of Appendices 33 to 45, further comprising a fourth display control unit that determines whether or not a magnitude of inclination in a vertical plane of a terminal on which the display device is provided is equal to or less than a predetermined threshold, and, if the magnitude of the inclination is not equal to or less than the predetermined threshold, causes the display device to display a guide encouraging the user to adjust the terminal to an appropriate position. (Appendix 47) 1. A computer-implemented control method comprising: a display control step of displaying on a display device a first screen including a display in each of a plurality of regions and a second screen including a display different from the first screen; an acquisition step of acquiring a plurality of captured images generated by capturing an image of an object with a camera while the first screen and the second screen are being displayed, respectively; a live body detection step of determining whether or not the target is a live body by using a plurality of the captured images, A control method, wherein the first screen includes at least two areas having different displays. (Appendix 48) The control method described in Appendix 47, wherein at least two of the multiple areas included in the first screen display different colors, different brightnesses, or different letters, symbols, or figures. (Appendix 49) The control method according to claim 47 or 48, wherein in the display control step, boundaries of the multiple areas included in the first screen are determined based on a positional relationship between a camera that generates the captured image and the display device. (Appendix 50) 49. The control method according to claim 47 or 48, wherein boundaries of the multiple areas included in the first screen are determined by user input. (Appendix 51) the computer has a discrimination model trained to output a label indicating whether the object shown in the captured image is a living body or not in response to input of a plurality of the captured images; A control method described in any one of Appendices 47 to 50, wherein in the display control step, the multiple captured images acquired in the acquisition step are input into the identification model, and the label is obtained from the identification model to determine whether the object is a living body or not. (Appendix 52) The discriminative model is trained using both positive and negative training data; the positive example training data includes a plurality of captured images obtained by capturing an image of a face of a real person while the first screen and the second screen are being displayed, and data indicating that the captured object is a living body; The control method described in Appendix 51, wherein the negative example training data includes a plurality of captured images obtained by capturing images of a human face while the first screen and the second screen are each displayed, and data indicating that the captured subject is not a living body. (Appendix 53) A control method according to any one of appendices 47 to 52, wherein in the display control step, the intensity of ambient light around the display device is identified, and when the intensity of the ambient light is equal to or lower than a threshold value, the first screen and the second screen are displayed on the display device. (Appendix 54) The control method described in Appendix 53, wherein, in the display control step, if the ambient light intensity is greater than a threshold, a message is output encouraging the user to move to a location where the ambient light is weaker than the current intensity. (Appendix 55) A control method described in any one of Appendix 47 to 54, wherein, in the display control step, the intensity of ambient light around the display device is identified, and a display to be included in at least one of the first screen and the second screen is determined according to the intensity of the ambient light. (Appendix 56) a second display control step of causing the display device to display a guide regarding a line of sight direction and a guide regarding a face direction; A second acquisition step of acquiring captured images generated by capturing images of the object multiple times with the camera; a second living body detection step of calculating a difference between a face direction and a gaze direction for each of the captured images using an image of the face of the subject appearing in each of the captured images, and determining whether or not the subject is a living body based on the calculated difference; A control method according to any one of appendices 47 to 55, wherein in the second display control step, a guide is displayed on the display device encouraging the user to shake their head from side to side while continuing to look at the same place. (Appendix 57) In the second display control step, displaying on the display device a guide to direct the face in a first direction without moving the line of sight; determining whether the direction of the face of the subject is the first direction; The control method described in Appendix 56, wherein, if the direction of the target's face is the first direction, a guide is displayed on the display device encouraging the target to turn their face in a second direction without moving their line of sight. (Appendix 58) The control method described in Appendix 56 or 57, wherein in the second display control step, it is determined whether the magnitude of the target's facial movement is equal to or greater than a threshold, and if the magnitude of the facial movement is not equal to or greater than the threshold, a guide encouraging the user to move the face more widely is displayed on the display device. (Appendix 59) A control method described in any one of Appendices 56 to 58, in which in the second display control step, it is determined whether the speed at which the target's face moves is above a threshold, and if the speed at which the face moves is below the threshold, a guide encouraging the target to move the face more slowly is displayed on the display device. (Appendix 60) A control method described in any one of Appendices 56 to 59, wherein in the second display control step, it is determined whether the gaze direction of the target is fixed, and if the gaze direction is not fixed, a guide urging the target not to move the gaze direction is displayed on the display device. (Appendix 61) A control method described in any one of Appendix 56 to 60, wherein in the second display control step, the display device is caused to display one or more of an indication indicating the direction of the subject's face, an indication indicating the direction the subject should face, an indication indicating the subject's gaze direction, and an indication indicating the direction the subject should gaze. (Appendix 62) determining whether or not the same object is continuously imaged by the camera using captured images generated by the camera during the period from when display of the screen is started by the display control step to when liveness detection is performed by the second liveness detection step; 62. The control method according to any one of appendices 56 to 61, wherein if the same object is not continuously imaged, processing is restarted from the display control step. (Appendix 63) By tracking the object, it is determined whether the object is captured in each of the captured images; The control method of claim 62, wherein when an image in which the object is not captured is detected, it is determined that the same object is not being continuously captured. (Appendix 64) Detecting image features of the object from each of the captured images; The control method described in Appendix 62, wherein when a captured image is detected that does not include image features of the object, it is determined that the same object is not being continuously captured. (Appendix 65) A control method as described in any one of Appendices 56 to 64, comprising a third display control step of using the captured image to determine whether the posture of the subject is suitable for live body detection, and, if the posture of the subject is not suitable for live body detection, displaying on the display device a guide encouraging the subject to assume an appropriate posture. (Appendix 66) In the third display control step, determining whether a direction of the face of the subject in a vertical plane is within a predetermined range using an image of the face of the subject captured in the captured image; The control method described in Appendix 65, further comprising displaying on the display device a guide encouraging the subject to assume an appropriate posture when the direction of the subject's face in the vertical plane is not within the specified range. (Appendix 67) The control method described in Appendix 66, wherein in the third display control step, different guides are displayed depending on whether the subject is making a head-waving motion or not. (Appendix 68) In the third display control step, determining whether or not a facial posture and a torso posture of the subject satisfy a predetermined condition using an image of the face and torso of the subject captured in the captured image; The control method of claim 67, further comprising causing the display device to display a guide encouraging the user to adopt an appropriate posture if the specified condition is not satisfied. (Appendix 69) A control method described in any one of Appendices 56 to 68, comprising a fourth display control step of determining whether or not a magnitude of tilt in a vertical plane of a terminal on which the display device is provided is equal to or less than a predetermined threshold, and, if the magnitude of the tilt is not equal to or less than the predetermined threshold, displaying on the display device a guide encouraging the user to adjust the terminal to an appropriate position.

[0199] This application claims priority from International Patent Cooperation Treaty Application PCT / JP2020 / 038329, filed on October 9, 2020, the disclosure of which is incorporated herein in its entirety. [Explanation of symbols]

[0200] 10. Camera 20 Display device 30 screens 32 areas 34 Borderline 40 Captured images 50 Target Object 60 terminals 80, 90, 100, 110 screens 120, 130, 140 straight line 500 Computers 502 Bus 504 Processor 506 Memory 508 Storage Devices 510 Input / Output Interface 512 Network Interface 2000 Biological detection device 2020 Display control unit 2040 Acquisition Department 2060 Biometric detector 2080 Second display control unit 2100 2nd Acquisition Department 2120 Second living body detection unit

Claims

1. A display capable of displaying a first guide to guide a subject captured by an imaging unit provided in a terminal to maintain a line of sight directed toward the terminal, a second guide to guide the subject to maintain the orientation of the subject's face relative to the subject, and a third guide to guide the user to move the terminal relative to the subject; a detection unit that calculates a difference between the direction of the subject's gaze and the direction of the subject's face from an image generated by the imaging unit by imaging the subject following the first guidance, the second guidance, and the third guidance, and determines whether the subject is a living body based on the difference.

2. The terminal according to claim 1 , wherein the display is capable of displaying the first guide, the second guide, and the third guide together.

3. The terminal according to claim 2 , wherein the third guidance is guidance for instructing the subject to move the terminal to the left or right.

4. The terminal according to claim 1, characterized in that, when it is determined that the subject has performed an action based on the third guidance, the display displays content of the third guidance that is different from the content of the third guidance that was displayed when the subject was performing the action based on the third guidance.

5. Displaying a first guide on a display of the terminal, the first guide instructing a user to maintain a line of sight toward the terminal with respect to a subject imaged by an imaging unit provided in the terminal; displaying a second guidance on the display to guide the subject to maintain a facial orientation of the subject; displaying a third guidance on the display, the third guidance instructing the user to move the terminal relative to the subject; A method executed by a computer, comprising: calculating a difference between the direction of the subject's gaze and the direction of the subject's face from an image generated by the imaging unit by imaging the subject following the first guidance, the second guidance, and the third guidance, and determining whether the subject is a living body based on the difference.

6. Displaying a first guide on a display of the terminal, the first guide instructing the user to maintain a line of sight toward the terminal with respect to a subject imaged by an imaging unit provided in the terminal; displaying a second guidance on the display to guide the subject to maintain a facial orientation of the subject; displaying a third guidance on the display, the third guidance instructing the user to move the terminal relative to the subject; A program that causes a computer to execute the following steps: calculate a difference between the direction of the subject's gaze and the direction of the subject's face from an image generated by the imaging unit by imaging the subject following the first guidance, the second guidance, and the third guidance; and determine whether the subject is a living body based on the difference.

7. A display capable of displaying a first guide that guides a subject captured by an imaging unit provided in a terminal to maintain a line of sight directed toward the terminal, a second guide that guides the subject to maintain the orientation of the subject's face relative to the subject, and a third guide that guides the user to move the terminal relative to the subject; a detection unit that determines whether the subject is a living body based on a difference between the direction of gaze and the direction of the face of the subject following the first guide while following the second guide and the third guide, the image of which is captured by the imaging unit.

8. The terminal described in Claim 7, characterized in that the display is capable of displaying the first guidance, the second guidance, and the third guidance together.

9. The terminal described in Claim 8, characterized in that the third guidance is guidance instructing the subject to move the terminal to the left or right.

10. The terminal described in Claim 7, characterized in that when it is determined that the subject has performed an action based on the third guidance, the display displays the content of the third guidance that is different from the third guidance that was displayed when the subject was performing the action based on the third guidance.

Images