Information processing system, information processing method, and recording medium

Abstract

This information processing system (10) comprises: an acquisition means (110) that acquires an image of interest including the face of a subject; a generation means (120) that, on the basis of the image of interest, generates first guide information indicating the current position of the face of the subject and second guide information indicating the current angle of the face of the subject; and a display means (130) that displays, together with the first guide information and second guide information, third guide information indicating a target position for the face of the subject and fourth guide information indicating a target angle for the face of the subject. This information processing system makes it possible to capture an image after suitably adjusting the position and angle of a face.

Description

Information processing system, information processing method, and recording medium

[0001] The present disclosure relates to the technical fields of an information processing system, an information processing method, and a recording medium.

[0002] Known systems of this type output predetermined guide information when capturing a facial image. For example, Patent Document 1 discloses displaying an image of a user's face together with a facial position guide. Patent Document 2 discloses displaying an instruction image for guiding the orientation of the user's face in the target direction after moving the user to a target position. Patent Document 3 discloses displaying a registration progress meter extending radially outward from the user's facial image.

[0003] JP 2020-091876 A JP 2019-212156 A JP 2019-204494 A

[0004] This disclosure aims to improve upon the techniques disclosed in the prior art documents.

[0005] One aspect of the information processing system of this disclosure includes an acquisition means for acquiring a target image including a face of a subject, a generation means for generating, based on the target image, first guide information indicating the current position of the subject's face and second guide information indicating the current angle of the subject's face, and a display means for displaying, together with the first guide information and the second guide information, third guide information indicating a target position of the subject's face and fourth guide information indicating a target angle of the subject's face.

[0006] One aspect of the information processing method of this disclosure is to obtain a target image including a target person's face by at least one computer, generate first guide information indicating the current position of the target person's face and second guide information indicating the current angle of the target person's face based on the target image, and display, together with the first guide information and the second guide information, third guide information indicating a target position of the target person's face and fourth guide information indicating a target angle of the target person's face.

[0007] One aspect of the recording medium of this disclosure is a recording medium having recorded thereon a computer program that causes at least one computer to execute an information processing method, which acquires a target image including a face of a subject, generates first guide information indicating the current position of the face of the subject and second guide information indicating the current angle of the face of the subject based on the target image, and displays, together with the first guide information and the second guide information, third guide information indicating a target position of the face of the subject, and fourth guide information indicating a target angle of the face of the subject.

[0008] 1. A block diagram showing a hardware configuration of an information processing system according to a first embodiment. 2. A block diagram showing a functional configuration of an information processing system according to a first embodiment. 3. A flowchart showing an operation flow of an information processing system according to a second embodiment. 4. A block diagram showing a functional configuration of an information processing system according to a third embodiment. 5. A flowchart showing an operation flow of an information processing system according to a fifth embodiment. 6. A plan view (part 1) showing an example of guide information in an information processing system according to a sixth embodiment. 7. A plan view (part 2) showing an example of guide information in an information processing system according to a sixth embodiment. 8. A flowchart showing an operation flow of an information processing system according to a seventh embodiment. 9. A plan view (part 1) showing an example of display of guide information in an information processing system according to a ninth embodiment. 10. A plan view (part 2) showing an example of display of guide information in an information processing system according to a ninth embodiment. 11. A plan view (part 1) showing an example of display of guide information in an information processing system according to a tenth embodiment. 12. A plan view (part 2) showing an example of display of guide information in an information processing system according to a eleventh embodiment.

[0009] Hereinafter, embodiments of an information processing system, an information processing method, and a recording medium will be described with reference to the drawings.

[0010] First Embodiment An information processing system according to a first embodiment will be described with reference to FIGS. 1 to 3. FIG.

[0011] (Hardware Configuration) First, the hardware configuration of the information processing system according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of the information processing system according to the first embodiment.

[0012] 1 , an information processing system 10 according to the first embodiment includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, and a storage device 14. The information processing system 10 may further include an input device 15 and an output device 16. The processor 11, RAM 12, ROM 13, storage device 14, input device 15, and output device 16 are connected via a data bus 17.

[0013] The processor 11 loads a computer program. For example, the processor 11 is configured to load a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may load a computer program stored in a computer-readable storage medium using a storage medium reading device (not shown). The processor 11 may acquire (i.e., load) the computer program from a device (not shown) located outside the information processing system 10 via a network interface. The processor 11 controls the RAM 12, the storage device 14, the input device 15, and the output device 16 by executing the loaded computer program. In particular, in this embodiment, when the processor 11 executes the loaded computer program, a functional block for displaying guide information when capturing a facial image is realized within the processor 11. In other words, the processor 11 may function as a controller that executes each control in the information processing system 10.

[0014] The processor 11 may be configured as, for example, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a demand-side platform (DSP), an application-specific integrated circuit (ASIC), or a quantum processor. The processor 11 may be configured as one of these, or may be configured to use multiple processors in parallel.

[0015] The RAM 12 temporarily stores computer programs executed by the processor 11. The RAM 12 temporarily stores data that the processor 11 temporarily uses while it is executing the computer programs. The RAM 12 may be, for example, a dynamic random access memory (D-RAM) or a static random access memory (SRAM). Alternatively, other types of volatile memory may be used instead of the RAM 12.

[0016] The ROM 13 stores computer programs executed by the processor 11. The ROM 13 may also store fixed data. The ROM 13 may be, for example, a programmable read-only memory (PROM) or an erasable read-only memory (EPROM). Alternatively, other types of non-volatile memory may be used instead of the ROM 13.

[0017] The storage device 14 stores data that is to be saved long-term by the information processing system 10. The storage device 14 may operate as a temporary storage device for the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0018] The input device 15 is a device that receives input instructions from a user of the information processing system 10. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel. The input device 15 may be configured as a mobile terminal such as a smartphone or a tablet. The input device 15 may also be, for example, a device that includes a microphone and is capable of voice input.

[0019] The output device 16 is a device that outputs information related to the information processing system 10 to the outside. For example, the output device 16 may be a display device (e.g., a display) that can display information related to the information processing system 10. The output device 16 may also be a speaker or the like that can output information related to the information processing system 10 as audio. The output device 16 may be configured as a mobile terminal such as a smartphone or a tablet. The output device 16 may also be a device that outputs information in a format other than an image. For example, the output device 16 may be a speaker that outputs information related to the information processing system 10 as audio.

[0020] 1 illustrates an example of an information processing system 10 including multiple devices, but all or some of the functions may be realized by a single device (information processing device). In this case, the information processing device may be configured to include only the processor 11, RAM 12, and ROM 13 described above, and the other components (i.e., the storage device 14, input device 15, and output device 16) may be provided by an external device connected to the information processing device. Furthermore, some of the calculation functions of the information processing device may be realized by an external device (e.g., an external server, a cloud, etc.).

[0021] (Functional Configuration) Next, the functional configuration of the information processing system 10 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the information processing system according to the first embodiment.

[0022] 2, the information processing system 10 according to the first embodiment is configured to include, as components for realizing its functions, an image acquisition unit 110, a guide information generation unit 120, and a display unit 130. Each of the image acquisition unit 110, the guide information generation unit 120, and the display unit 130 may be a processing block realized by, for example, the above-mentioned processor 11 (see FIG. 1).

[0023] The image acquisition unit 110 is configured to be able to acquire an image including the face of the subject (hereinafter referred to as "target image" as appropriate). This target image may be, for example, an image used for biometric authentication. The image acquisition unit 110 acquires the target image from, for example, a camera that captures an image of the subject. Note that the camera that captures the target image may be a camera external to the information processing system 10 (for example, a camera mounted on a smartphone owned by the user), or a camera included in the information processing system 10 (for example, a camera installed at a predetermined shooting location). The target image acquired by the image acquisition unit 110 is configured to be output to the guide information generation unit 120.

[0024] The guide information generation unit 120 is configured to be able to generate first guide information and second guide information based on the target image acquired by the image acquisition unit 110. The first guide information indicates the current face position of the target person (i.e., when the target image was captured). The second guide information indicates the current face angle of the target person (i.e., the direction the face is facing). The first guide information is generated by, for example, detecting the face position from the target image. The second guide information is generated by estimating the face direction detected from the target image. Specific examples of guide information will be described in detail in other embodiments described later. The first and second guide information generated by the guide information generation unit 120 are configured to be output to the display unit 130.

[0025] The display unit 130 is configured to be able to display first guide information (i.e., guide information indicating the current face position of the subject) and second guide information (i.e., guide information indicating the current face angle of the subject) generated by the guide generation unit 120. Furthermore, the display unit 130 is configured to be able to display third guide information and fourth guide information along with the first guide information and second guide information. The third guide information indicates a target position of the subject's face. The fourth guide information indicates a target angle of the subject's face. The target position and target angle indicated by the third guide information and fourth guide information are set based on a position and angle at which the subject's face can be appropriately photographed. The third guide information and fourth guide information may be set in advance based on, for example, the specifications of the camera. In this case, the third guide information and fourth guide information may be stored in the storage device 14 (see FIG. 1 ) described above. Alternatively, the third and fourth guide information may be generated each time according to the shooting environment, etc. (see the seventh embodiment described below). The display unit 130 may display each piece of guide information on a display included in the output device 16 (see FIG. 1 ), for example. Alternatively, the display unit 130 may display each piece of guide information on a display external to the information processing system 10 .

[0026] (Operation Flow) Next, the operation flow of the information processing system 10 according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the operation flow of the information processing system according to the first embodiment.

[0027] As shown in FIG. 3, when the operation of the information processing system 10 according to the first embodiment is started, the image acquisition unit 110 first acquires a target image including the face of a target person (step S101).

[0028] Next, the guide information generating unit 120 detects the face of the target person from the target image (step S102). For example, the guide information generating unit 120 detects an area in the target image where the face of the target person is present. Then, the guide information generating unit 120 estimates the face direction (angle) from the detected face of the target person (step S103). Note that, since existing technologies can be appropriately adopted as methods for detecting faces and methods for estimating face direction, detailed explanations thereof will be omitted here.

[0029] Next, the guide information generating unit 120 generates first guide information and second guide information based on the detected face position of the target person and the estimated face direction of the target person (step S104).

[0030] Next, the display unit 130 reads out the third guide information and the fourth guide information (step S105). Then, the display unit 130 displays the first guide information and the second guide information generated by the guide information generating unit 120, and the read out third guide information and the fourth guide information (step S106). Each piece of guide information may continue to be displayed until adjustment of the position and angle of the target person's face is completed (for example, until the position and angle of the target face become the target position and the target angle).

[0031] The information processing system 10 may have a function of photographing the subject again after displaying each piece of guide information described above. That is, the information processing system 10 may be configured to photograph the face of the subject at the target position and the target angle by displaying each piece of guide information. The information processing system 10 may be configured to end the display of each piece of guide information when the subject's face has been photographed at the target position and the target angle.

[0032] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the first embodiment will be described.

[0033] 1 to 3, the information processing system 10 according to the first embodiment displays first guide information and second guide information indicating the current position and angle of the face of the subject, as well as third guide information and fourth guide information indicating the target position and angle of the face. In this way, appropriate guidance can be provided by comparing the current position and angle of the subject with the target position and angle of the face, making it possible to appropriately capture an image of the subject's face.

[0034] Second Embodiment An information processing system 10 according to a second embodiment will be described with reference to Figures 4 and 5. Note that the second embodiment differs from the first embodiment described above only in some configurations and operations, and other parts may be the same as the first embodiment. Therefore, the following will describe in detail the parts that differ from the first embodiment already described, and will omit explanations of other overlapping parts as appropriate.

[0035] (Functional Configuration) First, the functional configuration of the information processing system 10 according to the second embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the functional configuration of the information processing system according to the second embodiment. Note that in Fig. 4, the same elements as those shown in Fig. 2 are denoted by the same reference numerals.

[0036] As shown in Fig. 4, the information processing system 10 according to the second embodiment is configured to include, as components for realizing its functions, an image acquisition unit 110, a guide information generation unit 120, a display unit 130, and a guidance unit 140. That is, the information processing system 10 according to the second embodiment further includes the guidance unit 140 in addition to the configuration of the first embodiment (see Fig. 2). The guidance unit 140 may be a processing block realized by, for example, the above-mentioned processor 11 (see Fig. 1).

[0037] The guidance unit 140 is configured to be able to output guidance information that guides the facial movement of the target person. The guidance information is information that guides the facial movement of the target person so that the first guide information and the third guide information overlap, and so that the second guide information and the fourth guide information overlap. The guidance information may be generated, for example, based on the degree of deviation between the first guide information and the third guide information (i.e., the degree of deviation between the current facial position of the target person and the target position). Furthermore, the guidance information may be generated, for example, based on the degree of deviation between the second guide information and the fourth guide information (i.e., the degree of deviation between the current facial angle of the target person and the target angle).

[0038] The guiding unit 140 may display the guidance information on the same display as each piece of guide information (i.e., the first to fourth guide information). For example, the guiding unit 140 may display messages such as "Please overlap the first guide information with the third guide information" and "Please overlap the second guide information with the fourth guide information" on the display on which each piece of guide information is displayed. Alternatively, the guiding unit 140 may display messages or arrows indicating the direction in which the user should move their face to overlap the first guide information with the third guide information and the direction in which the user should move their face to overlap the second guide information with the fourth guide information on the display on which each piece of guide information is displayed. Alternatively, the guiding unit 140 may output the guidance information as audio. For example, the guiding unit 140 may output the above-mentioned various messages as audio from a speaker installed near the display on which each piece of guide information is displayed.

[0039] (Operation Flow) Next, the operation flow of the information processing system 10 according to the second embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the operation flow of the information processing system according to the second embodiment. Note that in Fig. 5, the same processes as those shown in Fig. 3 are denoted by the same reference numerals.

[0040] As shown in FIG. 5 , when the operation of the information processing system 10 according to the second embodiment starts, first, the same processes as steps S101 to S106 described in the first embodiment are executed. That is, the image acquisition unit 110 acquires a target image including the face of a target person (step S101). The guide information generation unit 120 detects the face of the target person from the target image (step S102). The guide information generation unit 120 estimates the facial orientation from the detected face of the target person (step S103). The guide information generation unit 120 generates first guide information and second guide information based on the detected position of the target person's face and the estimated facial orientation of the target person (step S104). The display unit 130 reads out third guide information and fourth guide information (step S105). Then, the display unit 130 displays the first guide information and second guide information generated by the guide information generation unit 120 and the read third guide information and fourth guide information (step S106).

[0041] Thereafter, particularly in the second embodiment, the guiding unit 140 outputs the guidance information (step S201). The guiding unit 140 may continue to output the guidance information until the adjustment of the position and angle of the target person's face is completed (for example, until the position and angle of the target person's face become the target position and target angle).

[0042] The information processing system 10 may have a function of photographing the target person again after outputting the above-described guidance information. That is, the information processing system 10 may photograph the target person's face at the target position and the target angle by displaying the guidance information. The information processing system 10 may end output of the guidance information when the target person's face is photographed at the target position and the target angle.

[0043] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the second embodiment will be described.

[0044] 4 and 5, in the information processing system 10 according to the second embodiment, guidance information is output that guides the facial movement of the target person so that the first guide information and the third guide information overlap, and so that the second guide information and the fourth guide information overlap. In this way, the facial movement of the target can be guided, and the position and angle of the target face can be encouraged to become the target position and the target angle.

[0045] Third Embodiment An information processing system 10 according to a third embodiment will be described with reference to Fig. 6. The third embodiment is an embodiment that describes an example of displaying the first guide information and third guide information described above, and the system configuration and operation may be the same as those of the first and second embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0046] (Display example of guide information) First, a specific example of guide information in the information processing system 10 according to the third embodiment will be described with reference to Fig. 6. Fig. 6 is a plan view showing an example of guide information in the information processing system according to the third embodiment.

[0047] As shown in Fig. 6, the guide information output by the information processing system 10 according to the third embodiment is displayed as a frame line surrounding the face of the target person. The first guide information indicating the current face position of the target person is displayed as a frame line that follows the face of the target person. The third guide information indicating the target position is also displayed as a frame line of the same shape as the first guide information. Furthermore, the width of the frame line of the third guide information is displayed thicker than the width of the frame line of the first guide information.

[0048] As described above, the first guide information and third guide information are displayed together with the second guide information and fourth guide information, but for the sake of convenience, the second guide information and fourth guide information are not shown here. Display examples of the second guide information and fourth guide information will be described in detail in other embodiments described later.

[0049] In the example shown in FIG. 6 , a message saying "Please overlap the face frame" is displayed on the screen as an example of guidance information output by the guidance unit 140. As a result, the subject moves their face in an attempt to overlap the frame line of the first guide information with the frame line of the third guide information. Specifically, in the example shown in FIG. 6 , the subject attempts to overlap the first guide information and the third guide information by bringing their face closer to the camera (i.e., by moving so that the frame line of the first guide information becomes larger). As a result, the state in which the first guide information and the third guide information overlap is a face position suitable for capturing an image of the subject.

[0050] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the third embodiment will be described.

[0051] As described in Fig. 6, in the information processing system 10 according to the third embodiment, the first guide information and the third guide information are displayed in frame shapes of different thicknesses. In this way, by moving the face so that the first guide information overlaps the third guide information, the position of the face can easily approach the target position. Furthermore, by displaying the third guide information with a width wider than the width of the first guide information, it becomes easier to overlap the first guide information with the third guide information.

[0052] Fourth Embodiment An information processing system 10 according to a fourth embodiment will be described with reference to Fig. 7. The fourth embodiment is an embodiment that describes a method for setting the width of a border in the third embodiment described above, and other aspects may be the same as those of the third embodiment. Therefore, the following will describe in detail the aspects that differ from the embodiments already described, and will omit a description of other overlapping aspects as appropriate.

[0053] (Setting the width of guide information) First, a method for setting the width of guide information in the information processing system 10 according to the fourth embodiment will be specifically described with reference to Fig. 7. Fig. 7 is a plan view showing an example of guide information in the information processing system according to the fourth embodiment.

[0054] As shown in FIG. 7 , in the information processing system 10 according to the fourth embodiment, the width of the frame lines of the first guide information and the third guide information changes depending on the allowable range for the target position of the face. Specifically, when the allowable range for the face position is wide (i.e., when an appropriate image can be captured even if the position is slightly off), the width of the frame lines of the third guide information is displayed thick. On the other hand, when the allowable range for the face position is narrow (i.e., when even a slight deviation in the face position makes it impossible to capture an appropriate image), the width of the frame lines of the third guide information is displayed thin. Note that, although two examples with different frame line thicknesses have been given here, the width of the frame line may change more finely depending on the allowable range. In other words, the width of the frame line of the third guide information may change in three or more stages depending on the allowable range, or may change linearly.

[0055] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the fourth embodiment will be described.

[0056] 7, in the information processing system 10 according to the fourth embodiment, the widths of the first guide information and the third guide information are determined according to the allowable range for the target position of the face. In this way, it becomes easy to superimpose the first guide information on the third guide information within the allowable range. In other words, it is possible to prevent the width of the target position indicated by the third guide from being too narrow, making it difficult to superimpose the first guide information on the third guide information.

[0057] Fifth Embodiment An information processing system 10 according to a fifth embodiment will be described with reference to Fig. 8. The fifth embodiment is an embodiment that describes a display example of the second guide information and fourth guide information described above, and the system configuration and operation may be the same as those of the first to fourth embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0058] (Display example of guide information) First, a specific example of guide information in the information processing system 10 according to the fifth embodiment will be described with reference to Fig. 8. Fig. 8 is a plan view showing an example of guide information in the information processing system according to the fifth embodiment.

[0059] As shown in FIG. 8 , in the information processing system 10 according to the fifth embodiment, the first guide information and the third guide information are displayed as frame lines surrounding the face of the target person. Meanwhile, the second guide information and the fourth guide information are displayed as cross lines within the frame lines of the first guide information and the third guide information, respectively. More specifically, the second guide information indicating the current face angle of the target person is displayed as cross lines extending vertically and horizontally within the frame lines of the first guide information. The fourth guide information indicating the target face angle is displayed as cross lines extending vertically and horizontally within the frame lines of the third guide information. Furthermore, the width of the frame lines of the fourth guide information is displayed thicker than the width of the frame lines of the second guide information.

[0060] The second guide information is displayed as an arc along a spherical surface corresponding to the face. Specifically, the second guide information is displayed as two arcs connecting the tip of the face's normal vector with the horizontal and vertical rotation axes. Therefore, the shape of the arcs changes depending on the direction of the face, making it possible to indicate the angle of the target person's face. Furthermore, the width of the fourth guide information becomes thicker as it approaches the center (i.e., the part where the cross intersects). By changing the width of the crosshairs in this way, it is possible to make it easier to overlap the second guide information and the fourth guide information.

[0061] In the example shown in Fig. 8, the subject moves their face in an attempt to make the frame lines of the first guide information overlap with the frame lines of the third guide information. Also, the subject moves their face in an attempt to make the crosshairs of the second guide information overlap with the crosshairs of the fourth guide information. Specifically, in the example shown in Fig. 6, the subject attempts to make the second guide information and the fourth guide information overlap by facing their face directly toward the camera (i.e., by moving so that the crosshairs of the second guide information are in front of them). The resulting state in which the second guide information and the fourth guide information overlap is the face angle suitable for capturing an image of the subject.

[0062] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the fifth embodiment will be described.

[0063] As described in Fig. 8, in the information processing system 10 according to the fifth embodiment, the second guide information and the fourth guide information are displayed in the shape of crosshairs of different thicknesses. In this way, by moving the face so that the second guide information overlaps the fourth guide information, the face angle can easily approach the target angle. Furthermore, by displaying the fourth guide information with a width wider than the width of the second guide information, it becomes easier to overlap the second guide information with the fourth guide information.

[0064] Sixth Embodiment An information processing system 10 according to a sixth embodiment will be described with reference to Figures 9 and 10. The sixth embodiment is an embodiment that describes a method for setting the width of the crosshairs in the fifth embodiment described above, and other aspects may be the same as those of the fifth embodiment. Therefore, the following will describe in detail the aspects that differ from the embodiments already described, and will omit a description of other overlapping aspects as appropriate.

[0065] (Setting the width of guide information) First, a method for setting the width of guide information in the information processing system 10 according to the sixth embodiment will be specifically described with reference to Fig. 9. Fig. 9 is a plan view (part 1) showing an example of guide information in the information processing system according to the sixth embodiment.

[0066] As shown in FIG. 9 , in the information processing system 10 according to the sixth embodiment, the width of the frame lines of the second guide information and the fourth guide information changes according to the allowable range for the target face angle. Specifically, when the allowable range for the face angle is wide (i.e., when an appropriate image can be captured even with a slight angle deviation), the width of the crosshairs in the fourth guide information is displayed thick. On the other hand, when the allowable range for the face angle is narrow (i.e., when even a slight angle deviation makes it impossible to capture an appropriate image), the width of the crosshairs in the fourth guide information is displayed thin. Note that, although two examples with different crosshair thicknesses have been given here, the thickness of the crosshairs may change more finely according to the allowable range. That is, the thickness of the crosshairs in the fourth guide information may change in three or more stages according to the allowable range, or may change linearly.

[0067] (Modification) Next, a modification of the guide information in the information processing system 10 according to the sixth embodiment will be described with reference to Fig. 10. Fig. 10 is a plan view (part 2) showing an example of the guide information in the information processing system according to the sixth embodiment.

[0068] 10, the fourth guide information in the information processing system 10 according to the sixth embodiment may have an asymmetric shape in accordance with partial differences in the allowable range. By partially changing the width of the crosshairs in this way, it becomes possible to properly guide the face angle even when the allowable range differs depending on the direction.

[0069] The fourth guide information may be displayed such that the width of the crosshairs is thicker in areas where the allowable range is wider. Specifically, if the allowable range for the horizontal (i.e., left-right) angle of the face is wide, the vertical lines of the crosshairs may be displayed thicker. For example, as shown in FIG. 10 (left), if the allowable range for the face angle is wide only on the left side of the screen, the width of the vertical crosshairs may be thickened so that the left side rises. Furthermore, if the allowable range for the vertical (i.e., up-down) angle of the face is wide, the horizontal lines may be displayed thicker. For example, as shown in FIG. 10 (right), if the allowable range for the face angle is wide only on the upper side of the screen, the width of the horizontal crosshairs may be thickened so that the upper side rises. Note that, although an example in which the allowable range is widened in only one direction has been given here, if the allowable range is widened in multiple directions, the line thickness may be changed in those multiple directions. In this case, the thickness of both the vertical and horizontal lines of the crosshairs may be changed.

[0070] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the sixth embodiment will be described.

[0071] 9 and 10 , in the information processing system 10 according to the sixth embodiment, the widths of the second guide information and the fourth guide information are determined according to the allowable range for the target angle of the face. In this way, it becomes easy to superimpose the second guide information on the fourth guide information within the allowable range. In other words, it is possible to prevent the range of the target angle indicated by the fourth guide from being too narrow, making it difficult to superimpose the second guide information on the fourth guide information.

[0072] Seventh Embodiment An information processing system 10 according to a seventh embodiment will be described with reference to Fig. 11. Note that the seventh embodiment differs from the first embodiment described above only in some of its operations, and other parts may be the same as the first to sixth embodiments. Therefore, the following will describe in detail the parts that differ from the first embodiment already described, and will omit a description of other overlapping parts as appropriate.

[0073] (Operation Flow) First, the operation flow of the information processing system 10 according to the seventh embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the operation flow of the information processing system according to the seventh embodiment. Note that in Fig. 11, the same processes as those shown in Fig. 3 are denoted by the same reference numerals.

[0074] As shown in Fig. 11 , when the operation of the information processing system 10 according to the seventh embodiment starts, first, the same processes as steps S101 to S104 described in the first embodiment are executed. That is, the image acquisition unit 110 acquires a target image including the face of a target person (step S101). The guide information generation unit 120 detects the face of the target person from the target image (step S102). The guide information generation unit 120 estimates the facial direction from the detected face of the target person (step S103). The guide information generation unit 120 generates first guide information and second guide information based on the detected position of the target person's face and the estimated facial direction of the target person (step S104).

[0075] Particularly in the seventh embodiment, the guide information generation unit 120 generates third guide information and fourth guide information (step S701). The guide information generation unit 120 generates the third guide information and fourth guide information based on the target image. More specifically, the guide information generation unit 120 generates third guide information and fourth guide information suitable for the shooting environment (for example, brightness, etc.) estimated from the target image.

[0076] Once the third guide information and fourth guide information have been generated, the display unit 130 displays the first guide information, second guide information, third guide information, and fourth guide information generated by the guide information generating unit 120 (step S106).

[0077] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the seventh embodiment will be described.

[0078] 11 , in the information processing system 10 according to the seventh embodiment, the third guide information and the fourth guide information are generated based on the target image. In this way, appropriate third guide information and fourth guide information can be generated according to the current shooting environment of the target image. Therefore, compared to the case where third guide information and fourth guide information prepared in advance are used, it is possible to provide more appropriate guidance to the target position and target angle.

[0079] Eighth Embodiment An information processing system 10 according to an eighth embodiment will be described with reference to Fig. 12. The eighth embodiment is an embodiment that describes display examples of the above-described guide information, and the system configuration and operation may be the same as those of the first to seventh embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0080] (Display example of guide information) First, a specific example of guide information in the information processing system 10 according to the eighth embodiment will be described with reference to Fig. 12. Fig. 12 is a plan view showing a display example of guide information in the information processing system according to the eighth embodiment.

[0081] As shown in FIG. 12 , the guide information output by the information processing system 10 according to the eighth embodiment is displayed as indicating the position and angle of the subject's face when viewed from above. For example, each piece of guide information is displayed in an oval shape with a partially protruding nose. Note that the first guide information and the second guide information according to the eighth embodiment are displayed together as a single shape. Similarly, the third guide information and the fourth guide information according to the eighth embodiment are displayed together as a single shape. The current face position of the subject indicated by the first guide information and the target face position indicated by the third guide information are represented by the position and size of the ellipse. The current face angle of the subject indicated by the second guide information and the target face angle indicated by the fourth guide information are represented by the tilt of the ellipse and the position of the nose.

[0082] In the example shown in Fig. 12, the subject moves their face in an attempt to overlap the frame lines of the ellipses corresponding to the first and second guide information with the frame lines of the ellipses corresponding to the third and fourth guide information. Specifically, in the example shown in Fig. 12, the subject faces the camera and moves closer to the camera in an attempt to overlap the frame lines of the ellipses. At this time, the subject moves their face so that their noses are also perfectly overlapped. The resulting state in which the ellipses corresponding to the first and second guide information and the ellipses corresponding to the third and fourth guide information overlap is the face position and angle appropriate for capturing an image of the subject.

[0083] (Technical Effects) Next, technical effects obtained by the information processing system 10 according to the eighth embodiment will be described.

[0084] 12, in the information processing system 10 according to the eighth embodiment, each piece of guide information is displayed as indicating the position and angle of the target person's face when viewed from above. This allows the current position and angle of the face and the target position and angle to be confirmed from a direction different from the direction in which the target image is captured, making it possible to appropriately adjust the position and angle of the face. For example, it is possible to appropriately guide the movement of the face in the depth direction, which is difficult to see from the front.

[0085] Ninth Embodiment An information processing system 10 according to a ninth embodiment will be described with reference to Fig. 13 and Fig. 14. Note that the ninth embodiment is an embodiment that describes the display pattern of each piece of guide information, and the system configuration and operation may be the same as those of the other embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0086] (Display Pattern of Guide Information) Hereinafter, a display pattern of guide information in the information processing system 10 according to the eighth embodiment will be described with reference to Fig. 13 and Fig. 14. Fig. 13 is a plan view (part 1) showing a display example of guide information in the information processing system according to the ninth embodiment. Fig. 14 is a plan view (part 2) showing a display example of guide information in the information processing system according to the ninth embodiment.

[0087] 13 is a pattern in which each piece of guide information is displayed superimposed on the target image. By displaying it in this manner, the user can move their face while checking both the actual movement of their face and the movement of the guide information.

[0088] Pattern B is a pattern in which, in addition to the display of pattern A, parts corresponding to the position of the eyes are added to each piece of guide information. By displaying in this way, each piece of guide information can be made to look more like a face. Also, by moving the face so that the eyes overlap, adjustments can be made more easily. Note that, although an example of displaying parts corresponding to the eyes has been given here, parts other than the eyes (for example, nose, ears, mouth, etc.) may also be displayed.

[0089] Pattern C is a pattern in which the target image of pattern A is not displayed, and only the guide information is displayed. In this way, it is possible to prevent the target image and the guide information from being displayed overlapping each other, making the image difficult to see. In this embodiment, the first and second guide information indicating the current position and angle of the target person's face are displayed, so the position of the face can be appropriately adjusted even if the actual face image is not displayed.

[0090] Pattern D is a pattern in which only the guide information is displayed without displaying the target image of pattern B. Even in this case, the same effect as in pattern C described above can be obtained.

[0091] 14 is a pattern in which the first guide information and the third guide information are displayed in a rectangular shape. In this way, the face frame line does not have to be a shape that follows the contours of the face. In other words, the shapes of the first guide information and the third guide information are not particularly limited and can be various shapes.

[0092] Pattern F is a pattern in which guide information (eighth embodiment, see FIG. 12 ) indicating the position and angle of the target person's face when viewed from above is displayed at the lower right of pattern A. By displaying in this manner, the target person can move their face while checking both the state when viewed from the front and the state when viewed from above.

[0093] Tenth Embodiment An information processing system 10 according to a tenth embodiment will be described with reference to Fig. 15. The tenth embodiment is an embodiment that describes an example of changing the display of each piece of guide information, and the system configuration and operation may be the same as those of the other embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0094] (Head Swing Guidance) Head swing guidance using guide information will be described with reference to Fig. 15. Fig. 15 is a plan view showing an example of displaying guide information in the information processing system according to the tenth embodiment.

[0095] As shown in Fig. 15 , in the information processing system 10 according to the tenth embodiment, the fourth guide information indicating the target face angle is gradually changed to guide the facial movement of the subject. For example, by first setting the target angle to a state in which the face is facing left, gradually shifting the target angle from there to the front, and finally setting the target angle to a state in which the face is facing right, the subject can be encouraged to rotate their face from left to right. By repeatedly displaying this information, the subject can be encouraged to shake their head from side to side. Such a head shaking movement may be performed, for example, as part of liveness determination.

[0096] When encouraging the subject to perform the head-shaking motion described above, the guidance unit 140 may output a message such as "Move your face so that the face frame overlaps" as guidance information.

[0097] <Eleventh Embodiment> An information processing system 10 according to an eleventh embodiment will be described with reference to Fig. 16. Note that, like the tenth embodiment, the eleventh embodiment is an embodiment that describes an example of changing the display of each piece of guide information, and the system configuration and operation may be the same as those of the other embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0098] (Color change of guide information) A display example in which the color of guide information is changed will be described with reference to Fig. 16. Fig. 16 is a plan view showing a display example of guide information in an information processing system according to the eleventh embodiment.

[0099] 16 , in the information processing system 10 according to the tenth embodiment, when the guide information overlaps, the color of each guide information changes. For example, when the face position and angle do not match the target position and target angle (i.e., the first guide information and the third guide information do not overlap, and the second guide information and the fourth guide information do not overlap), and when only the face position matches the target position (i.e., the first guide information and the third guide information overlap, and the second guide information and the fourth guide information do not overlap), the color of the third guide information, which indicates the target position of the face, changes. This allows a subject who is moving their face in accordance with the guide information to intuitively understand that the face position and the target position match.

[0100] Thereafter, when the face angle also matches the target angle (i.e., the first guide information and the third guide information overlap, and the second guide information and the fourth guide information also overlap), the color of the fourth guide information, which indicates the target face angle, changes. This allows the subject, who is moving their face in accordance with the guide information, to intuitively know that the face angle and the target angle match.

[0101] In the above example, the face position and face angle are matched with the target in that order, but even if the face angle and face position are matched with the target in that order, the color of the guide information may be changed sequentially. Specifically, first, when the face angle matches the target angle, the color of the fourth guide information indicating the target angle may be changed, and then, when the face position matches the target position, the color of the second guide information indicating the target position may be changed.

[0102] In the above example, a match between the guide information pieces is indicated by a change in color of the guide information piece, but a match between the guide information pieces may be notified by a method other than a change in color. For example, a match between the guide information pieces may be notified by displaying a message or outputting a sound effect.

[0103] The scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-described embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, each embodiment includes not only a recording medium on which the above-described program is recorded, but also the program itself.

[0104] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. Furthermore, the scope of each embodiment is not limited to programs that execute processes by themselves, but also includes programs that execute processes by operating on an OS in conjunction with other software or expansion board functions. Furthermore, the program itself may be stored on a server, and part or all of the program may be downloadable from the server to a user terminal. The program may be provided to the user in, for example, a SaaS (Software as a Service) format.

[0105] <Supplementary Notes> The above-described embodiment may be further described as in the following supplementary notes, but is not limited to the following.

[0106] (Supplementary Note 1) The information processing system described in Supplementary Note 1 is an information processing system including: an acquisition means for acquiring a target image including a face of a subject; a generation means for generating, based on the target image, first guide information indicating the current position of the face of the subject and second guide information indicating the current angle of the face of the subject; and a display means for displaying, together with the first guide information and the second guide information, third guide information indicating a target position of the face of the subject and fourth guide information indicating a target angle of the face of the subject.

[0107] (Appendix 2) The information processing system described in Appendix 2 is the information processing system described in Appendix 1, further comprising a guidance means for outputting guidance information that guides the facial movement of the subject so that the first guide information and the third guide information overlap, and so that the second guide information and the fourth guide information overlap.

[0108] (Supplementary Note 3) The information processing system described in Supplementary Note 3 is the information processing system described in Supplementary Note 1 or 2, in which the first guide information and the third guide information are frame shapes corresponding to the position of the face, and the width of the third guide information is wider than the width of the first guide information.

[0109] (Supplementary Note 4) The information processing system described in Supplementary Note 4 is the information processing system described in Supplementary Note 3, in which the width of the third guide information is determined based on a first allowable range set for the target position of the subject's face.

[0110] (Appendix 5) The information processing system described in Appendix 5 is the information processing system described in Appendix 1 or 2, in which the second guide information and the fourth guide information are cross-shaped extending in two axial directions indicating the angle of the face, and the width of the fourth guide information is wider than the width of the second guide information.

[0111] (Supplementary Note 6) The information processing system described in Supplementary Note 6 is the information processing system described in Supplementary Note 5, in which the width of the fourth guide information is determined according to a second allowable range set for the target angle of the subject's face.

[0112] (Supplementary Note 7) The information processing system described in Supplementary Note 7 is the information processing system described in any one of Supplementary Notes 1 to 6, wherein the generation means generates the third guide information and fourth guide information in addition to the first guide information and the second guide information based on the target image.

[0113] (Appendix 8) The information processing system described in Appendix 8 is the information processing system described in any one of Appendices 1 to 7, wherein the display means displays the first guide information, the second guide information, the third guide information, and the fourth guide information as indicating the position and angle of the subject's face when looking down from above.

[0114] (Supplementary Note 9) The information processing method described in Supplementary Note 9 is an information processing method that, by at least one computer, acquires a target image including a face of a subject, generates first guide information indicating a current position of the face of the subject and second guide information indicating a current angle of the face of the subject based on the target image, and displays, together with the first guide information and the second guide information, third guide information indicating a target position of the face of the subject, and fourth guide information indicating a target angle of the face of the subject.

[0115] (Supplementary Note 10) The recording medium described in Supplementary Note 10 is a recording medium having recorded thereon a computer program for causing at least one computer to execute an information processing method, which comprises acquiring a target image including a face of a subject, generating first guide information indicating a current position of the face of the subject and second guide information indicating a current angle of the face of the subject based on the target image, and displaying, together with the first guide information and the second guide information, third guide information indicating a target position of the face of the subject, and fourth guide information indicating a target angle of the face of the subject.

[0116] (Supplementary Note 11) The computer program described in Supplementary Note 11 is a computer program that causes at least one computer to execute an information processing method, which acquires a target image including a face of a subject, generates first guide information indicating a current position of the face of the subject and second guide information indicating a current angle of the face of the subject based on the target image, and displays, together with the first guide information and the second guide information, third guide information indicating a target position of the face of the subject and fourth guide information indicating a target angle of the face of the subject.

[0117] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and information processing systems, information processing methods, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure.

[0118] REFERENCE SIGNS LIST 10 Information processing system 11 Processor 110 Image acquisition unit 120 Guide information generation unit 130 Display unit 140 Guidance unit

Claims

1. Acquisition means for acquiring a target image including the face of a target person; Generation means for generating first guide information indicating the current position of the face of the target person and second guide information indicating the current angle of the face of the target person based on the target image; Display means for displaying third guide information indicating the target position of the face of the target person and fourth guide information indicating the target angle of the face of the target person together with the first guide information and the second guide information; An information processing system comprising:

2. Further comprising guiding means for outputting guiding information for guiding the movement of the face of the target person so that the first guide information overlaps with the third guide information and the second guide information overlaps with the fourth guide information; The information processing system according to claim 1.

3. The first guide information and the third guide information are in a frame shape corresponding to the position of the face, The width of the third guide information is thicker than the width of the first guide information, The information processing system according to claim 1 or 2.

4. The width of the third guide information is determined based on a first allowable range set for the target position of the face of the target person, The information processing system according to claim 3.

5. The second guide information and the fourth guide information are in a cross shape extending in two axial directions indicating the angle of the face, The width of the fourth guide information is thicker than the width of the second guide information, The information processing system according to claim 1 or 2.

6. The width of the fourth guide information is determined according to a second allowable range set for the target angle of the face of the target person, The information processing system according to claim 5.

7. The generation means generates the third guide information and the fourth guide information in addition to the first guide information and the second guide information based on the target image. The information processing system according to claim 1 or 2.

8. The display means displays the first guide information, the second guide information, the third guide information, and the fourth guide information as indicating the position and angle when looking down on the face of the target person from above. The information processing system according to claim 1 or 2.

9. By at least one computer, Obtain a target image including the face of the target person, Based on the target image, generate first guide information indicating the current position of the face of the target person and second guide information indicating the current angle of the face of the target person, Display the first guide information and the second guide information together with third guide information indicating the target position of the face of the target person and fourth guide information indicating the target angle of the face of the target person. Information processing method.

10. To at least one computer, Obtain a target image including the face of the target person, Based on the target image, generate first guide information indicating the current position of the face of the target person and second guide information indicating the current angle of the face of the target person, Display the first guide information and the second guide information together with third guide information indicating the target position of the face of the target person and fourth guide information indicating the target angle of the face of the target person. A computer program for executing an information processing method.