Information processing device, information processing method, and recording medium

Abstract

An information processing device according to the present invention comprises: a first unit including a first camera that captures an image of an object from a first direction, and a first irradiation unit that irradiates the object with light from the first direction; a second unit including a second camera that captures an image of the object from a second direction, and a second irradiation unit that irradiates the object with light from the second direction; a generation means for generating three-dimensional shape data of the object using the image captured by the first camera and the image captured by the second camera; and a determination means for determining whether or not the images captured by the first camera and the second camera are suitable for generating the three-dimensional shape data, on the basis of boundary periphery information pertaining to the periphery of the boundary between a face region and a neck region of the object.

Description

Information processing device, information processing method, and recording medium

[0001] The present disclosure relates to the technical fields of an information processing device, an information processing method, and a recording medium.

[0002] Known examples of this type of device include one that generates data representing a three-dimensional shape from images captured by a camera. For example, Patent Document 1 discloses a technology in which a person's head is captured by a right and left camera unit, and the three-dimensional shape is calculated from the images captured by each camera unit and output to a display or the like.

[0003] International Publication No. 2022 / 034694

[0004] An object of this disclosure is to provide an information processing device, an information processing method, and a recording medium that aim to improve upon the techniques disclosed in prior art documents.

[0005] One aspect of the information processing device disclosed herein comprises a first unit having a first camera that photographs an object from a first direction and a first illumination unit that irradiates the object with light from the first direction; a second unit having a second camera that photographs the object from a second direction and a second illumination unit that irradiates the object with light from the second direction; a generation means that generates three-dimensional shape data of the object using an image photographed by the first camera and an image photographed by the second camera; and a determination means that determines whether the images photographed by the first camera and the second camera are suitable for generating the three-dimensional shape data based on boundary surrounding information that is information regarding the periphery of the boundary between the face region and neck region of the object.

[0006] One aspect of the information processing method disclosed herein is an information processing method for controlling an information processing device that includes a first unit having a first camera that photographs an object from a first direction and a first illumination unit that irradiates light onto the object from the first direction, and a second unit having a second camera that photographs the object from a second direction and a second illumination unit that irradiates light onto the object from the second direction, wherein three-dimensional shape data of the object is generated using an image photographed by the first camera and an image photographed by the second camera, and based on boundary surrounding information that is information regarding the area around the boundary between the face area and neck area of ​​the object, it is determined whether the images photographed by the first camera and the second camera are suitable for generating the three-dimensional shape data.

[0007] One aspect of the recording medium of this disclosure is an information processing method for controlling an information processing device that includes a first unit having a first camera that photographs an object from a first direction and a first illumination unit that irradiates the object with light from the first direction, and a second unit having a second camera that photographs the object from a second direction and a second illumination unit that irradiates the object with light from the second direction, wherein a computer program is recorded that causes the information processing method to generate three-dimensional shape data of the object using an image photographed by the first camera and an image photographed by the second camera, and to determine whether the images photographed by the first camera and the second camera are suitable for generating the three-dimensional shape data based on boundary surrounding information, which is information regarding the periphery of the boundary between the face region and neck region of the object.

[0008] 1 is a block diagram showing a hardware configuration of a first information processing device. 2 is a perspective view showing the configuration of the first information processing device. 3 is a top view showing the configuration of the first information processing device. 4 is a block diagram showing a functional configuration of the first information processing device. 5 is a schematic diagram showing a defect in three-dimensional shape data caused by a depth estimation error. 6 is a flowchart showing the operation flow of the first information processing device. 7 is a block diagram showing the functional configuration of a second information processing device. 8 is a flowchart showing the operation flow of the second information processing device. 9 is a block diagram showing a functional configuration of a modified example of the second information processing device. 10 is a flowchart showing the operation flow of the modified example of the second information processing device. 11 is a schematic diagram showing the relationship between the face angle and the luminance difference around the boundary. 12 is a graph showing the relationship between the face angle and the luminance difference around the boundary. 13 is a block diagram showing the functional configuration of a third information processing device. 14 is a schematic diagram showing the relationship between the position of the Adam's apple and the chin and the face angle. 15 is a flowchart showing the operation flow of the third information processing device. 16 is a block diagram showing the functional configuration of a fourth information processing device. 17 is a schematic diagram showing a method of estimating the face angle in the fourth information processing device. 18 is a flowchart showing the operation flow of the fourth information processing device. 19 is a schematic diagram showing the operation flow of the fourth information processing device. 20 is a block diagram showing the functional configuration of a fifth information processing device. 21 is a plan view showing an example display by the fifth information processing device. 22 is a block diagram showing the functional configuration of a sixth information processing device. 23 is a flowchart showing the operation flow of the sixth information processing device.

[0009] Hereinafter, embodiments of an information processing device, an information processing method, and a recording medium will be described with reference to the drawings.

[0010] First Embodiment A first information processing apparatus will be described with reference to FIGS. 1 to 6. FIG.

[0011] (Hardware Configuration) First, the hardware configuration of the first information processing apparatus will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of the first information processing apparatus.

[0012] 1, the first information processing device 1 includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, a storage device 14, an input device 15, an output device 16, a first unit 21, and a second unit 22. The processor 11, RAM 12, ROM 13, storage device 14, input device 15, output device 16, first unit 21, and second unit 22 are connected to each other via a data bus 17. Note that the data bus 17 may be an interface other than a data bus (e.g., a LAN, a USB, etc.).

[0013] The processor 11 loads a computer program. For example, the processor 11 is configured to load a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may load a computer program stored in a computer-readable storage medium using a storage medium reading device (not shown). The processor 11 may acquire (i.e., load) the computer program from a device (not shown) located outside the first information processing device 1 via a network interface. The processor 11 executes the loaded computer program to perform various processes. When the processor 11 executes the loaded computer program, functional blocks related to the processes performed by the first information processing device 1 are realized within the processor 11. In other words, the processor 11 may function as a controller that executes each control in the first information processing device 1.

[0014] The processor 11 may be configured as, for example, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or a quantum processor. The processor 11 may be configured as one of these, or may be configured to use multiple processors in parallel.

[0015] The RAM 12 temporarily stores computer programs executed by the processor 11. The RAM 12 temporarily stores data that the processor 11 temporarily uses while it is executing the computer programs. The RAM 12 may be, for example, a dynamic random access memory (D-RAM) or a static random access memory (SRAM). Alternatively, other types of volatile memory may be used instead of the RAM 12.

[0016] The ROM 13 stores computer programs executed by the processor 11. The ROM 13 may also store fixed data. The ROM 13 may be, for example, a programmable read-only memory (PROM) or an erasable read-only memory (EPROM). Alternatively, other types of non-volatile memory may be used instead of the ROM 13.

[0017] The storage device 14 stores data that is to be saved long-term by the first information processing device 1. The storage device 14 may operate as a temporary storage device for the processor 11. The storage device 14 may store computer programs executed by the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0018] The input device 15 is a device that receives input instructions from a user of the first information processing device 1. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel. The input device 15 may also be, for example, a device that includes a microphone and is capable of voice input.

[0019] The output device 16 is a device that outputs information related to the first information processing device 1 to the outside. For example, the output device 16 may be a display device (e.g., a display or a monitor) that can display information related to the first information processing device 1. The output device 16 may also be a speaker or the like that can output information related to the information processing device 1 as audio.

[0020] The first unit 21 is configured to include a first camera 211 and a first irradiating unit 212. The first camera 211 is arranged to capture an image of the target from a first direction. The first irradiating unit 212 is arranged to irradiate light onto the target from the first direction (i.e., the shooting direction of the first camera 211). Similarly, the second unit 22 is configured to include a second camera 221 and a second irradiating unit 222. The second camera 221 is arranged to capture an image of the target from a second direction. The second irradiating unit 222 is arranged to irradiate light onto the target from the second direction (i.e., the shooting direction of the second camera 221). The specific arrangement of the first unit 21 and the second unit 22 will be described in detail later.

[0021] The first camera 211 and the second camera 221 may include a solid-state imaging element such as a CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor) image sensor, etc. The first camera 211 and the second camera 221 may also include an optical system that forms an image of the subject on the imaging surface of the solid-state imaging element, a signal processing circuit that processes the output of the solid-state imaging element to obtain a luminance value for each pixel, etc.

[0022] The first irradiating unit 212 and the second irradiating unit 222 may be configured to project a periodic light pattern. For example, the first irradiating unit 212 and the second irradiating unit 222 may be configured to project sinusoidal wave patterns with different periods. The first irradiating unit 212 and the second irradiating unit 222 may be, for example, a DLP (Digital Light Processing) projector or a liquid crystal projector.

[0023] 1. For example, the first information processing device 1 may be configured to include only the processor 11, RAM 12, and ROM 13 among the above-mentioned components. In this case, the storage device 14, input device 15, output device 16, first unit 21, and second unit 11 may be provided as devices external to the first information processing device 1. Furthermore, some of the calculation functions of the first information processing device 1 may be realized by an external server, a cloud, or the like.

[0024] (Specific Device Configuration) Next, a specific configuration of the first information processing device 1 (particularly, an example of the arrangement of the first unit 21 and the second unit 22) will be described with reference to Figures 2 and 3. Figure 2 is a perspective view showing the configuration of the first information processing device. Figure 3 is a top view showing the configuration of the first information processing device.

[0025] 2 and 3 , the first information processing device 1 is configured as a device that photographs the head of the subject 50 (specifically, a facial region including the face of the subject 50 and a neck region including the neck of the subject 50). The subject 50 may be photographed, for example, with the subject sitting in front of the first information processing device 1. Alternatively, the subject 50 may be photographed with the subject standing in front of the first information processing device 1.

[0026] The first unit 21 in the first information processing device 1 is disposed on the left front side as viewed from the target 50. Therefore, the first camera 211 photographs the head of the target 50 from the left front side. Furthermore, the first irradiating section 212 irradiates light from the left front side toward the head of the target 50. The second unit 22 in the first information processing device 1 is disposed on the right front side as viewed from the target 50. Therefore, the second camera 221 photographs the head of the target 50 from the right front side. Furthermore, the second irradiating section 222 irradiates light from the right front side toward the head of the target 50.

[0027] The first information processing device 1 includes a third camera 300 that captures an image of the head of the target 50 from the front of the target 50. The third camera 300 may be a camera similar to the first camera 211 and the second camera 221 described above. However, the third camera 300 is not an essential component of the first information processing device 1.

[0028] (Functional Configuration) Next, the functional configuration of the first information processing device 1 will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the functional configuration of the first information processing device.

[0029] 4, the first information processing device 1 is configured to include, as components for realizing its functions, a generation unit 110 and a determination unit 120 in addition to the above-described first camera 211 and second camera 221. Note that each of the generation unit 110 and the determination unit 120 may be a processing block realized by the above-described processor 11 (see FIG. 1).

[0030] The generation unit 110 is configured to acquire an image of the target 50 captured by the first camera 211 (hereinafter referred to as the "first image" as appropriate) and an image of the target 50 captured by the second camera 221 (hereinafter referred to as the "second image" as appropriate). The generation unit 110 is then configured to generate three-dimensional shape data of the target 50 using the acquired first and second images. That is, the generation unit 110 is configured to generate data indicating the three-dimensional shape of the target's head using two images captured from different directions. The generation unit 110 first calculates phase values ​​of each part of the target's head from the first and second images captured by projecting light of a predetermined pattern (e.g., a striped pattern) onto the target's head. Specifically, the generation unit 110 calculates relative phase values ​​based on the luminance values ​​of the captured images, performs phase unwrapping, and calculates absolute phase values. The generation unit 110 then estimates the depth of the target's head based on the calculated absolute phase values. Examples of such three-dimensional measurement methods include a sinusoidal grating phase shift method, a method that combines a sinusoidal wave pattern with a different period or a brightness gradient pattern in addition to the sinusoidal grating phase shift method, etc. These methods can achieve both high measurement accuracy and high measurement speed, and are suitable for measuring the face of a person, for example, who has difficulty remaining still for a long period of time.

[0031] The determination unit 120 is configured to be able to acquire boundary periphery information, which is information relating to the periphery of the boundary between the face region and neck region of the subject 50 (i.e., the periphery of the facial outline). The determination unit 120 may acquire the boundary periphery information from, for example, the first image and the second image. Alternatively, the determination unit 120 may acquire outline information from an image captured by a camera other than the first camera 211 and the second camera 221 (e.g., the third camera 300). Specific examples of the boundary periphery information will be described in detail in other embodiments described later.

[0032] The determination unit 120 is configured to determine whether the first image and the second image are suitable for generating three-dimensional shape data based on the acquired boundary periphery information. Here, "an image suitable for generating three-dimensional shape data" refers to an image from which the generation unit 110 can generate three-dimensional shape data with high accuracy, and may be, for example, an image that is less likely to cause depth estimation errors when generating three-dimensional shape data. Depth estimation errors will be specifically described below with reference to FIG. 5. FIG. 5 is a schematic diagram showing defects in three-dimensional shape data caused by depth estimation errors.

[0033] As shown in FIG. 5 , the generation unit 110 generates three-dimensional shape data using a first image captured by the first camera 211 and a second image captured by the second camera 221. When generating three-dimensional shape data, a phase unwrapping process may be performed, for example. However, if an incorrect phase unwrapping occurs during this phase unwrapping process, three-dimensional shape data that differs from the original shape may be generated. For example, as shown in the example of FIG. 5 , an error in the depth estimation result during the generation of three-dimensional shape data may result in three-dimensional shape data in which the neck region protrudes forward relative to the face region. This incorrect phase unwrapping process is likely to occur in areas where the position in the depth direction changes discontinuously. Therefore, an incorrect three-dimensional shape is likely to be generated around the boundary between the face region and the neck region of the human head. In particular, if the subject 50 is facing downward, the depth cannot be accurately estimated due to the influence of a shadow that appears around the boundary.

[0034] The determination unit 120 may determine that the first image and the second image are inappropriate when it is determined that there is a high possibility that a defect in the three-dimensional shape data such as that described above will occur. On the other hand, the determination unit 120 may determine that the first image and the second image are appropriate when it is determined that there is a low possibility that a defect in the three-dimensional shape data will occur. As already described, defects in the three-dimensional shape data may occur depending on the facial orientation of the target 50, so the determination unit 120 may determine whether the first image and the second image are appropriate depending on the facial orientation of the target 50 in the first image and the second image.

[0035] (Flow of Operation) Next, the flow of operation of the first information processing device 1 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the flow of operation of the first information processing device.

[0036] 6 , when the operation of the first information processing device 1 is started, the first camera 211 and the second camera 221 first capture a first image and a second image of the target 50 (step S101). Note that multiple first images and multiple second images may be captured.

[0037] Next, the determination unit 120 acquires boundary periphery information from the first and second images or from images captured by another camera (step S102), and then determines whether the first and second images are appropriate for generating three-dimensional shape data (step S103).

[0038] If it is determined that the first image and the second image are appropriate for generating three-dimensional shape data (step S103: YES), the generator 110 generates three-dimensional shape data of the object 50 using the first image and the second image determined to be appropriate (step S104). On the other hand, if it is determined that the first image and the second image are not appropriate for generating three-dimensional shape data (step S103: NO), the processing of step S104 described above may be omitted. That is, the generator 110 does not need to generate three-dimensional shape data.

[0039] 6 illustrates an example in which three-dimensional shape data is generated when the first image and the second image are determined to be appropriate. However, the three-dimensional shape data may be generated even when the first image and the second image are determined to be inappropriate. That is, the generation unit 110 may generate three-dimensional shape data using the first image and the second image that are determined to be inappropriate. Alternatively, when the generation unit 110 determines that the first image and the second image are inappropriate, the generation unit 110 may re-capture the first image and the second image, and generate three-dimensional shape data after the first image and the second image are determined to be appropriate.

[0040] 6 illustrates an example in which the determination regarding the first and second images is performed after the first and second images are acquired and before the three-dimensional shape data is generated, but the three-dimensional shape data may be generated before the determination. That is, the determination regarding the first and second images performed by the determination unit 120 may be performed after the generation unit 110 generates the three-dimensional shape data. For example, the process of step S104 may be performed immediately after the process of step S101, and then the processes of steps S102 and S103 may be performed.

[0041] (Technical Effects) Next, technical effects obtained by the first information processing device 1 will be described.

[0042] As described with reference to FIGS. 1 to 6 , the first information processing device 1 determines whether the first and second images are suitable for generating three-dimensional shape data based on boundary periphery information (i.e., information about the boundary periphery of the subject's neck region and face region). This can reduce problems that may occur when generating three-dimensional shape data. For example, by generating three-dimensional shape data if the first and second images are determined to be suitable, and not generating three-dimensional shape data if the first and second images are determined to be inappropriate, the possibility of problems occurring in the three-dimensional shape data can be reduced. Furthermore, even if problems occur in the three-dimensional shape data, if it is determined that the first and second images are inappropriate, the first and second images can be re-captured to generate appropriate three-dimensional shape data.

[0043] Second Embodiment A second information processing device 1 will be described with reference to Figures 7 to 12. The second information processing device 1 differs in some configurations and operations from the first information processing device 1 described above, but other parts may be similar to the first information processing device 1. Therefore, the following will describe in detail the parts that differ from the first embodiment, and will omit explanations of other overlapping parts as appropriate.

[0044] (Functional Configuration) First, the functional configuration of the second information processing device 1 will be described with reference to Fig. 7. Fig. 7 is a block diagram showing the functional configuration of the second information processing device. Note that in Fig. 7, the same elements as those described in Fig. 4 are denoted by the same reference numerals.

[0045] 7, the second information processing device 1 is configured to include, as components for realizing its functions, a first camera 211, a second camera 221, a generation unit 110, a determination unit 120, and a guiding unit 130. That is, the second information processing device 1 further includes the guiding unit 130 in addition to the configuration of the first embodiment already described (see FIG. 4). Note that the guiding unit 130 may be a processing block realized by the above-described processor 11 (see FIG. 1).

[0046] The guidance unit 130 is configured to be capable of outputting guidance information that prompts the target 50 to gradually change the angle of the face. The guidance information may be information that outputs a message such as "Look up little by little" to the target 50. Such a message may be displayed as text on a display or the like, or as an illustration. It may also be output as audio by a speaker or the like.

[0047] The second information processing device 1 captures a plurality of images while the target 50 is changing the angle of its face in response to the guidance information. The images may be captured by the first camera 211 and the second camera 221, or may be captured by another camera (e.g., the third camera 300). The determination unit 120 then acquires a luminance difference around the boundary between the face region and neck region of the target 50 from the plurality of images captured while the target 50 is changing the angle of its face. For example, the determination unit 120 may acquire a difference in luminance between the chin of the target 50 and the Adam's apple of the target 50.

[0048] The determination unit 120 in the second information processing device 1 uses the brightness difference between the face region and the neck region as boundary periphery information. That is, the determination unit 120 determines whether the first image and the second image are suitable for generating three-dimensional shape data based on the brightness difference between the face region and the neck region of the subject 50. The determination unit 120 may determine that the first image and the second image are suitable for generating three-dimensional shape data, for example, if the brightness difference between the face region and the neck region is equal to or less than a predetermined threshold. In this case, the threshold may be set to, for example, a brightness value that does not cause depth estimation errors when generating three-dimensional shape data.

[0049] (Operation Flow) Next, the operation flow of the second information processing device 1 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the operation flow of the second information processing device. Note that in Fig. 8, the same processes as those described in Fig. 5 are denoted by the same reference numerals.

[0050] 8 , when the operation of the second information processing device 1 is started, first, the first camera 211 and the second camera 221 capture a first image and a second image of the target 50 (step S101). Note that, when the determination unit 120 acquires a luminance difference from an image other than the first image and the second image, the image may be captured by another camera.

[0051] Next, the determination unit 120 obtains the luminance difference between the face region and neck region of the subject 50 from the image captured in step S101 (step S201). Then, the determination unit 120 determines whether the obtained luminance difference is an appropriate value (step S202). That is, the determination unit 120 determines whether the obtained luminance difference is a value that allows the first image and the second image to be determined as images suitable for generating three-dimensional shape data.

[0052] If the acquired brightness difference is determined to be an appropriate value (step S202: YES), the generation unit 110 generates three-dimensional shape data of the object 50 using the first image and the second image determined to be appropriate (step S104).

[0053] On the other hand, if it is determined that the acquired luminance difference is not an appropriate value (step S202: NO), the guidance unit 130 outputs guidance information (step S203). By outputting the guidance information, the target 50 gradually changes the angle of its face.

[0054] When the angle of the face of the target 50 is changed, the processes of steps S101, S201, and S203 are executed again. As a result, the first image and the second image are repeatedly captured until the brightness difference around the boundary reaches an appropriate value (i.e., until the angle of the face of the target 50 reaches an appropriate angle). Then, when the brightness difference reaches an appropriate value (step S202: YES), the generation unit 110 generates three-dimensional shape data of the target 50 (step S104).

[0055] 8 shows an example in which the guidance information is output after it is determined that the brightness difference around the boundary is not an appropriate value, but the guidance information may be output before it is determined that the brightness difference around the boundary is not an appropriate value. For example, the guidance unit 130 may output the guidance information before first executing step S101.

[0056] (Functional Configuration of Modified Example) Next, a functional configuration of a modified example of the second information processing device 1 will be described with reference to Fig. 9. Fig. 9 is a block diagram showing the functional configuration of a modified example of the second information processing device. Note that in Fig. 9, the same elements as those described in Fig. 7 are denoted by the same reference numerals.

[0057] 9, the modified example of the second information processing device 1 is configured to include, as components for realizing its functions, a first camera 211, a second camera 221, a generation unit 110, a determination unit 120, and a camera control unit 140. That is, the modified example of the second information processing device 1 includes the camera control unit 140 instead of the guiding unit 130 in the configuration of the second embodiment already described (see FIG. 7). Note that the camera control unit 140 may be a processing block realized by the above-mentioned processor 11 (see FIG. 1).

[0058] The camera control unit 140 is configured to be able to output control information that gradually changes the angles of the first camera 211 and the second camera 221 relative to the target 50. The control information may also be control information that gradually changes the angle of the third camera 300. The control information may be information that changes the imaging angle of the camera relative to the target 50. For example, the control information may be information that physically changes the angle of the camera by controlling a motor or the like. Alternatively, the control information may be information that changes the angle of the imaging range of the camera by controlling the optical system of the camera. Furthermore, the control information may be information that changes the angle of the camera by moving the entire housing of the second information processing device 1.

[0059] In a variation of the second information processing device 1, a plurality of images are taken while the camera angle relative to the target 50 is being changed in accordance with the control information. The images may be taken by the first camera 211 and the second camera 221, or may be taken by another camera (for example, the third camera 300). The determination unit 120 then obtains the brightness difference around the boundary between the face region and neck region of the target 50 from the plurality of images taken while the target 50 is changing the angle of its face.

[0060] (Operational flow according to the modified example) Next, the operational flow according to the modified example of the second information processing device 1 will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the operational flow according to the modified example of the second information processing device. Note that in Fig. 10, the same processes as those described in Fig. 8 are denoted by the same reference numerals.

[0061] 10 , when the operation of the second information processing device 1 starts, the first camera 211 and the second camera 221 first capture a first image and a second image of the target 50 (step S101). Note that if the determination unit 120 acquires a luminance difference from an image other than the first image and the second image, the image may be captured by another camera.

[0062] Next, the determination unit 120 obtains the luminance difference between the face region and neck region of the subject 50 from the image captured in step S101 (step S201). Then, the determination unit 120 determines whether the obtained luminance difference is an appropriate value (step S202). That is, the determination unit 120 determines whether the obtained luminance difference is a value that allows the first image and the second image to be determined as images suitable for generating three-dimensional shape data.

[0063] If the acquired brightness difference is determined to be an appropriate value (step S202: YES), the generation unit 110 generates three-dimensional shape data of the object 50 using the first image and the second image determined to be appropriate (step S104).

[0064] On the other hand, if it is determined that the acquired brightness difference is not an appropriate value (step S202: NO), camera control unit 140 outputs control information (step S213). By outputting the control information, the angles of first camera 211 and second camera 221 relative to target 50 are gradually changed.

[0065] While the angles of the first camera 211 and the second camera 221 are being changed, the processes of steps S101, S201, and S203 are executed again. As a result, the first image and the second image are repeatedly captured until the brightness difference around the boundary reaches an appropriate value (i.e., until the angle of the face of the target 50 as viewed from the first camera 211 and the second camera 221 reaches an appropriate angle). Then, when the brightness difference reaches an appropriate value (step S202: YES), the generation unit 110 generates three-dimensional shape data of the target 50 (step S104).

[0066] 10 shows an example in which the control information is output after it is determined that the brightness difference around the boundary is not an appropriate value, but the control information may be output before it is determined that the brightness difference around the boundary is not an appropriate value. For example, the camera control unit 140 may output the control information before executing step S101 for the first time.

[0067] (Relationship between Brightness Difference and Facial Angle) Next, the relationship between the brightness difference around the boundary used as boundary periphery information in the second information processing device 1 and the angle of the face will be specifically described with reference to Fig. 11 and Fig. 12. Fig. 11 is a schematic diagram showing the relationship between the angle of the face and the brightness difference around the boundary. Fig. 12 is a graph showing the relationship between the angle of the face and the brightness difference around the boundary.

[0068] As shown in Figure 11, the angle of the face of the subject 50 causes a shadow to appear around the boundary (specifically, a part of the neck area just below the chin). In this case, a discontinuous portion occurs at the boundary between the face area and the neck area, which may result in a depth estimation error (see Figure 6) when generating three-dimensional shape data. Therefore, the first and second images that cause a shadow to appear around the boundary are considered to be unsuitable for generating three-dimensional shape data.

[0069] On the other hand, the shadow around the boundary described above becomes lighter as the angle of the subject's 50 face increases (i.e., as the subject looks up). Therefore, by having the subject look up, it is possible to suppress the occurrence of errors in depth estimation when generating three-dimensional shape data. In other words, the first image and the second image, in which the shadow around the boundary becomes lighter as the subject looks up, can be said to be suitable images for generating three-dimensional shape data.

[0070] 12, an image suitable for generating three-dimensional shape data (specifically, the facial angle of the subject 50 suitable for generating three-dimensional shape data) can be determined from the change in brightness difference. Specifically, the point where the gradient of the brightness difference around the boundary changes with respect to the facial angle of the subject 50 (the area surrounded by the dotted line in the figure) can be determined to be the facial angle (appropriate position) of the subject 50 suitable for generating three-dimensional shape data.

[0071] (Technical Effects) Next, technical effects obtained by the second information processing device 1 will be described.

[0072] As described with reference to FIGS. 7 to 12 , the second information processing device 1 determines whether the first and second images are suitable for generating three-dimensional shape data based on the luminance difference between the face region and neck region of the target 50. In this manner, it is possible to easily and accurately determine whether the angle of the target 50's face in the first and second images is appropriate. As a result, it is possible to generate three-dimensional shape data using first and second images in which the target 50's face is captured from an appropriate angle. Specifically, by outputting guidance information as described with reference to FIGS. 7 and 8 , the target 50's face angle is gradually changed, allowing appropriate first and second images to be captured. Furthermore, by outputting control information as described with reference to FIGS. 9 and 10 , the angles of the first camera 211 and the second camera 221 relative to the target 50's face can be gradually changed, allowing appropriate first and second images to be captured.

[0073] <Third embodiment> A third information processing device 1 will be described with reference to Figures 13 to 15. The third information processing device 1 differs in some configurations and operations from the first and second information processing devices 1 described above, but other parts may be similar to the first and second information processing devices 1. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0074] (Functional Configuration) First, the functional configuration of the third information processing device 1 will be described with reference to Fig. 13. Fig. 13 is a block diagram showing the functional configuration of the third information processing device. Note that in Fig. 13, the same elements as those described in Fig. 4 are denoted by the same reference numerals.

[0075] 13 , the third information processing device 1 is configured to include, as components for realizing its functions, a first camera 211, a second camera 221, a generation unit 110, and a determination unit 120. In particular, the determination unit 120 in the third information processing device 1 includes an Adam's apple detection unit 121 and a chin detection unit 122.

[0076] The Adam's apple detection unit 121 is configured to detect the position of the Adam's apple of the subject 50. The chin detection unit 122 is configured to detect the position of the subject's chin (e.g., the position of the tip of the chin). The Adam's apple detection unit 121 and the chin detection unit 122 may detect the position of the Adam's apple and the position of the chin from the first image and the second image. Alternatively, the Adam's apple detection unit 121 and the chin detection unit 122 may detect the position of the Adam's apple and the position of the chin from an image other than the first image and the second image (e.g., an image captured by the third camera 100). The Adam's apple detection unit 121 and the chin detection unit 122 may be a learning model constructed by machine learning. For example, the Adam's apple detection unit 121 and the chin detection unit 122 may include a neural network trained by deep learning.

[0077] The determination unit 120 in the third information processing device 1 uses the position of the Adam's apple detected by the Adam's apple detection unit 120 and the position of the chin detected by the chin detection unit 122 as boundary periphery information. That is, the determination unit 120 determines whether the first image and the second image are suitable for generating three-dimensional shape data based on the positions of the Adam's apple and the chin of the target 50. A specific determination method by the determination unit 120 will be described below with reference to Fig. 14. Fig. 14 is a schematic diagram showing the relationship between the positions of the Adam's apple and the chin and the angle of the face.

[0078] As shown in FIG. 14 , the determination unit 120 may determine that the first image and the second image are suitable for generating three-dimensional shape data based on the positional relationship between the Adam's apple and the chin (specifically, the distance between the Adam's apple and the chin). For example, if the Adam's apple and the chin are relatively close to each other, it can be determined that the subject 50 is facing downward. In this case, a shadow is likely to appear around the boundary. Therefore, the determination unit 120 may determine that the angle of the subject 50's face in the first image and the second image is inappropriate (i.e., not suitable for generating three-dimensional shape data). On the other hand, if the Adam's apple and the chin are relatively far from each other, it can be determined that the subject 50 is facing upward. In this case, a shadow is unlikely to appear around the boundary. Therefore, the determination unit 120 may determine that the angle of the subject 50's face in the first image and the second image is appropriate (i.e., suitable for generating three-dimensional shape data).

[0079] (Operation Flow) Next, the operation flow of the third information processing device 1 will be described with reference to Fig. 15. Fig. 15 is a flowchart showing the operation flow of the third information processing device. Note that in Fig. 15, the same processes as those described in Fig. 5 are denoted by the same reference numerals.

[0080] As shown in FIG. 15, when the operation of the second information processing device 1 is started, first, the first camera 211 and the second camera 221 capture a first image and a second image of the target 50 (step S101).

[0081] Next, the Adam's apple detection unit 121 detects the position of the Adam's apple of the subject 50 (step S301). Furthermore, the chin detection unit 122 detects the position of the subject's chin (step S302). Note that the processes of steps S301 and S302 may be executed in succession or simultaneously in parallel.

[0082] Next, the determination unit 120 determines whether the relationship between the Adam's apple position detected by the Adam's apple detection unit 121 and the chin position detected by the chin detection unit 122 is appropriate (step S303). That is, the determination unit 120 uses the Adam's apple position and the chin position as boundary periphery information to determine whether the first image and the second image are suitable for generating three-dimensional shape data.

[0083] If it is determined that the relationship between the Adam's apple position and the chin position is appropriate (step S303: YES), the generation unit 110 generates three-dimensional shape data of the object 50 using the first image and the second image (step S104). On the other hand, if it is determined that the relationship between the Adam's apple position and the chin position is not appropriate (step S303: NO), the processing of the above-mentioned step S104 may be omitted. In other words, the generation unit 110 does not need to generate three-dimensional shape data.

[0084] If it is determined that the relationship between the Adam's apple position and the chin position is not appropriate (step S303: NO), guidance information and control information may be output as in the second embodiment described above (see FIGS. 7 to 10 ), and the first and second images may be captured so that the relationship between the Adam's apple position and the chin position can be determined to be appropriate. In this case, the generation unit 110 may generate three-dimensional shape data when it is determined that the relationship between the Adam's apple position and the chin position is appropriate.

[0085] (Technical Effects) Next, technical effects obtained by the third information processing device 1 will be described.

[0086] 13 to 15, in the third information processing device 1, the positions of the Adam's apple and the chin of the subject are used as boundary periphery information. In this way, it is possible to easily and accurately determine whether the angle of the face of the subject 50 in the first image and the second image is appropriate. As a result, it is possible to generate three-dimensional shape data using the first image and the second image in which the face of the subject 50 is captured from an appropriate angle.

[0087] <Fourth embodiment> A fourth information processing device 1 will be described with reference to Figures 16 to 18. The fourth information processing device 1 differs in some configurations and operations from the first to third information processing devices 1 described above, but other parts may be similar to the first to third information processing devices 1. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0088] (Functional Configuration) First, the functional configuration of the fourth information processing device 1 will be described with reference to Fig. 16. Fig. 16 is a block diagram showing the functional configuration of the fourth information processing device. Note that in Fig. 16, the same elements as those described in Fig. 4 are denoted by the same reference numerals.

[0089] 16 , the fourth information processing device 1 is configured to include, as components for realizing its functions, a first camera 211, a second camera 221, a generation unit 110, a determination unit 120, and a first guide unit 150. That is, the fourth information processing device 1 includes the first guide unit 150 in addition to the configuration of the first embodiment already described (see FIG. 4 ). The first guide unit 150 may be a processing block realized by the above-mentioned processor 11 (see FIG. 1 ). Furthermore, the determination unit 120 in the fourth information processing device 1 includes a face angle estimation unit 125.

[0090] The face angle estimation unit 125 is configured to be able to estimate the face angle of the target 50 based on three-dimensional shape data of the target 50. Hereinafter, a method for estimating the face angle by the face angle estimation unit 125 will be specifically described with reference to Fig. 17. Fig. 17 is a schematic diagram showing a face angle estimation method in the fourth information processing device.

[0091] As shown in Fig. 17 , the face angle estimation unit 125 may detect facial feature points from the three-dimensional shape data and estimate the face angle of the target 50. The feature points may be detected as points corresponding to each predetermined part of the face. In the example shown in Fig. 17 , feature points are detected at the inner corners of the eyes, the outer corners of the eyes, the tip of the nose, and both ends of the mouth. These feature points are merely examples, and feature points may be detected from other parts.

[0092] The face angle estimation unit 125 may estimate the face angle of the subject 50 by comparing feature points extracted from the three-dimensional shape data with feature points extracted from a planar image (e.g., a third image captured by the third camera 300).

[0093] 16 , the first guide unit 150 is configured to be able to output first guide information based on the face angle of the target 50 estimated by the face angle estimation unit 125. The first guide information is information that provides guidance so that the face angle of the target 50 in the first image and the second image becomes an angle suitable for generating three-dimensional shape data.

[0094] The first guide information may be information that guides the target 50 to change the facial angle to an appropriate value. For example, the first guide information may be information that outputs a message to the target 50 saying, "Look up a little more,". The guide information may also include a specific numerical value calculated from the estimated facial angle. For example, the first guide information may be information that outputs a message to the target 50 saying, "Look up another 5 degrees,". Such a message may be displayed as text on a display or the like, or as an illustration. It may also be output as audio through a speaker or the like.

[0095] Alternatively, the first guide information may be information for controlling the change of the angles of the first camera 211 and the second camera 221 relative to the face of the target 50. For example, the first guide information may be information for physically changing the angle of the camera by controlling a motor or the like. Alternatively, the control information may be information for changing the angle of the camera's imaging range by controlling the optical system of the camera. Furthermore, the control information may be information for changing the angle of the camera by moving the entire housing of the fourth information processing device 1.

[0096] (Operation Flow) Next, the operation flow of the fourth information processing device 1 will be described with reference to Fig. 18. Fig. 18 is a flowchart showing the operation flow of the fourth information processing device. Note that in Fig. 18, the same processes as those described in Fig. 5 are denoted by the same reference numerals.

[0097] 18 , when the operation of the second information processing device 1 starts, the first camera 211 and the second camera 221 first capture a first image and a second image of the target 50 (step S101). Then, the generation unit 110 generates three-dimensional shape data of the target 50 using the captured first image and second image (step S104).

[0098] Next, the determination unit 120 acquires boundary periphery information (step S102), and then determines whether the first image and the second image are appropriate images for generating three-dimensional shape data (step S103).

[0099] If it is determined that the first image and the second image are appropriate for generating three-dimensional shape data (step S103: YES), the series of operations ends. On the other hand, if it is determined that the first image and the second image are not appropriate for generating three-dimensional shape data (step S103: NO), the face angle estimation unit 125 estimates the face angle of the target 50 from the three-dimensional shape data generated in step S104 (step S401).

[0100] Next, the first guide unit 150 outputs first guide information based on the face angle of the target 50 estimated by the face angle estimation unit 125. As a result, the face angle of the target 50 as seen from the first camera 211 and the second camera 221 is changed.

[0101] Next, the first camera 211 and the second camera 221 capture a first image and a second image of the target 50 (step S403. That is, the angle of the face of the target 50 is changed using the first guide information, and then a new first image and a new second image are captured. Then, the generation unit 110 regenerates three-dimensional shape data of the target 50 using the newly captured first image and second image (step S404).

[0102] When regenerating the 3D shape data in step S404, the first and second images used to initially generate the 3D shape data in step S104 may also be used. Specifically, the 3D shape data may be generated using both the first and second images captured in step S101 and the first and second images captured in step S402. In this case, the 3D shape data is generated by integrating the first and second images captured at different facial angles. This allows for accurate generation of 3D shape data even for areas that become difficult to see due to changes in facial angle. For example, when the subject's face is facing upward, depth estimation errors are less likely to occur, but the upper part of the face (e.g., the forehead) becomes difficult to see. Even in such cases, more accurate 3D shape data can be generated by using the first and second images captured from different angles.

[0103] (Technical Effects) Next, technical effects obtained by the fourth information processing device 1 will be described.

[0104] 16 to 18 , in the fourth information processing device 1, the face angle of the target 50 is estimated from the three-dimensional shape data, and the first guide information is output based on the estimated face angle. In this way, it is possible to appropriately change the face angle of the target and capture the first image and the second image from an appropriate angle.

[0105] Fifth Embodiment A fifth information processing device 1 will be described with reference to Figures 19 and 20. The fifth information processing device 1 differs in some configurations and operations from the first to fourth information processing devices 1 described above, but other parts may be similar to the first to fourth information processing devices 1. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0106] (Functional Configuration) First, the functional configuration of the fifth information processing device 1 will be described with reference to Fig. 19. Fig. 19 is a block diagram showing the functional configuration of the fifth information processing device. Note that in Fig. 19, the same elements as those described in Fig. 4 are denoted by the same reference numerals.

[0107] 16 , the fifth information processing device 1 is configured to include, as components for realizing its functions, a first camera 211, a second camera 221, a third camera 200, a generation unit 110, a determination unit 120, a display control unit 160, and a display unit 170. That is, in addition to the configuration of the first embodiment already described (see FIG. 4 ), the fifth information processing device 1 further includes a third camera 300 (see FIGS. 2 and 3 ), a display control unit 160, and a display unit 170. The display control unit 170 may be a processing block realized by the above-described processor 11 (see FIG. 1 ). Furthermore, the display unit 170 may be realized by an output device (see FIG. 1 ).

[0108] As already described, the third camera 300 is a camera that captures an image of the head of a target from the front. The display unit 170 is configured to be able to display the third image captured by the third camera 300.

[0109] The display control unit 160 is configured to be able to control the display content on the display unit 170. Specifically, the display control unit 160 is able to cause the display unit 170 to display information on the determination result by the determination unit 120 together with a third image captured by the third camera. For example, the display control unit 160 may display the information on the determination result by the determination unit 120 so as to be superimposed on the third image.

[0110] (Display Example) Next, a specific display example of the fifth information processing device 1 will be described with reference to Fig. 20. Fig. 20 is a plan view showing a display example of the fifth information processing device.

[0111] 20, the display unit 170 of the fifth information processing device 1 displays the third image (i.e., a front image of the face of the subject 50) captured by the third camera 300. Furthermore, the display unit 170 displays information related to the determination result by the determination unit 120. Here, since it has been determined that the first image and the second image are not suitable for generating three-dimensional shape data, a message stating "The angle of the face is inappropriate" is displayed.

[0112] Furthermore, the display unit 170 may display information for adjusting the angle of the target's face to an appropriate angle. In the example shown in Fig. 20, a message saying "Look a little higher" and an image of an arrow indicating the upward direction are displayed. The information displayed here may be the same as the first guide information described in the fourth embodiment. Specifically, the angle of the target's face may be estimated, and information may be output based on the estimated angle of the face.

[0113] (Technical Effects) Next, technical effects obtained by the fifth information processing device 1 will be described.

[0114] 19 and 20 , in the fifth information processing device 1, the display unit 170 displays the third image and information related to the determination result of the determination unit 120. In this way, it is possible to clearly inform the target 50 and the user of the device whether the captured first and second images are suitable for generating three-dimensional shape data. Furthermore, if information guiding the target 50 to change the angle of their face is also displayed, it becomes possible to adjust the angle of the target's face to an appropriate angle and capture appropriate first and second images.

[0115] Sixth Embodiment A sixth information processing device 1 will be described with reference to Figures 21 and 22. The sixth information processing device 1 differs in some configurations and operations from the first to fifth information processing devices 1 described above, but other parts may be similar to the first to fifth information processing devices 1. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0116] (Functional Configuration) First, the functional configuration of the sixth information processing device 1 will be described with reference to Fig. 21. Fig. 21 is a block diagram showing the functional configuration of the sixth information processing device. Note that in Fig. 21, the same elements as those shown in Fig. 4 are denoted by the same reference numerals.

[0117] 21 , the sixth information processing device 1 is configured to include, as components for realizing its functions, a generation unit 110, a determination unit 120, and a second guide unit 180. That is, the sixth information processing device 1 further includes the second guide unit 180 in addition to the configuration of the first embodiment (see FIG. 4 ). Note that the second guide unit 180 may be a processing block realized by the above-described processor 11 (see FIG. 1 ).

[0118] The second guide unit 180 is configured to be able to determine whether or not both a face region and a neck region are included in the captured first and second images. If the first and second images do not include both a face region and a neck region, the second guide unit 180 outputs second guide information for capturing the first and second images that include both a face region and a neck region. Note that if the first and second images include both a face region and a neck region, the second guide unit 180 does not need to output the second guide information.

[0119] The second guide unit 180 can determine whether both a face region and a neck region are included by, for example, performing semantic segmentation on the first image and the second image and determining whether both a face region and a neck region can be extracted. Alternatively, the second guide unit 180 can determine whether both a face region and a neck region are included by using a feature point detector that detects the positions of the chin and Adam's apple and determining whether both the chin and Adam's apple are detected. Such a determination may be made using a learning model constructed by machine learning. The learning model may include, for example, a neural network trained by deep learning.

[0120] The second guide information may be information that prompts the user to move to an appropriate position (i.e., a position where both the face region and the neck region are within the imaging range of the first camera 211 and the second camera 221). For example, the second guide unit 180 may output second guide information that outputs a message such as "Please move a little further away from the camera" or "Please move your face up a little more." Such a message may be displayed as text on a display or the like, or as an illustration. It may also be output as audio by a speaker or the like.

[0121] Furthermore, the second guide information may be information that changes the imaging range by controlling the first camera 211 and the second camera 221. For example, the second guide unit 180 may output second guide information that controls the zoom or pan of the first camera 211 and the second camera 221. Alternatively, the second guide unit 180 may output guide information that changes the angle or position of the first camera 211 and the second camera 221.

[0122] (Operation Flow) Next, the operation flow of the sixth information processing device 1 will be described with reference to Fig. 22. Fig. 22 is a flowchart showing the operation flow of the sixth information processing device. Note that in Fig. 22, the same processes as those shown in Fig. 5 are denoted by the same reference numerals.

[0123] As shown in FIG. 22, when the operation of the sixth information processing device 1 is started, first, the first camera 211 and the second camera 221 capture a first image and a second image of the target 50 (step S101).

[0124] Next, the second guide unit 180 determines whether or not both the face region and the neck region are included in the first image and the second image (step S601). If both the face region and the neck region are not included in the first image and the second image (step S601: NO), the second guide unit 180 outputs second guide information (step S602). After the second guide information is output, the processing of step S101 may be executed again. That is, the first image and the second image of the target 50 may be captured again after the second guide information has been used to place both the face region and the neck region within the imaging ranges of the first camera 211 and the second camera 221.

[0125] On the other hand, if both the face region and the neck region are included in the first and second images (step S601: YES), the determination unit 120 acquires boundary periphery information (step S102), and then determines whether the first and second images are appropriate for generating three-dimensional shape data (step S103).

[0126] If it is determined that the first image and the second image are appropriate for generating three-dimensional shape data (step S103: YES), the generator 110 generates three-dimensional shape data of the object 50 using the first image and the second image determined to be appropriate (step S104). On the other hand, if it is determined that the first image and the second image are not appropriate for generating three-dimensional shape data (step S103: NO), the processing of step S104 described above may be omitted. That is, the generator 110 does not need to generate three-dimensional shape data.

[0127] (Technical Effects) Next, technical effects obtained by the sixth information processing device 1 will be described.

[0128] 21 and 22 , the sixth information processing device 1 outputs the second guide information when the captured image does not include both the face region and the neck region of the target 50. In this way, it is possible to capture an image including the face region and the neck region, and therefore it is possible to appropriately generate three-dimensional shape data of the human head including the face region and the neck region.

[0129] The scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-described embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, each embodiment includes not only a recording medium on which the above-described program is recorded, but also the program itself.

[0130] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. Furthermore, the scope of each embodiment is not limited to programs that execute processes by themselves, but also includes programs that execute processes by operating on an OS in conjunction with other software or expansion board functions. Furthermore, the program itself may be stored on a server, and part or all of the program may be downloadable from the server to a user terminal. The program may be provided to the user in, for example, a SaaS (Software as a Service) format.

[0131] <Supplementary Notes> The above-described embodiment may be further described as in the following supplementary notes, but is not limited to the following.

[0132] (Supplementary Note 1) The information processing device described in Supplementary Note 1 is an information processing device comprising: a first unit having a first camera that photographs an object from a first direction and a first illumination unit that irradiates the object with light from the first direction; a second unit having a second camera that photographs the object from a second direction and a second illumination unit that irradiates the object with light from the second direction; a generation means that generates three-dimensional shape data of the object using an image photographed by the first camera and an image photographed by the second camera; and a determination means that determines whether the images photographed by the first camera and the second camera are suitable for generating the three-dimensional shape data based on boundary surrounding information that is information related to the periphery of the boundary between the face region and the neck region of the object.

[0133] (Supplementary Note 2) The information processing device according to Supplementary Note 2 is the information processing device according to Supplementary Note 1, wherein the determining means uses a luminance difference between the face region and the neck region around the boundary as the boundary periphery information.

[0134] (Appendix 3) The information processing device described in Appendix 3 is the information processing device described in Appendix 2, further comprising a guidance means for outputting guidance information that prompts the subject to gradually change the angle of his or her face, and the determination means obtains the brightness difference between the face area and the neck area around the boundary from a plurality of images taken while the subject is changing the angle of his or her face in response to the guidance information.

[0135] (Appendix 4) The information processing device described in Appendix 4 is the information processing device described in Appendix 2, further comprising a control means for controlling the angle of the first camera and the second camera relative to the face of the subject to be gradually changed, and the determination means obtains the brightness difference between the face area and the neck area around the boundary from multiple images taken by the control means while changing the angle of the first camera and the second camera.

[0136] (Appendix 5) The information processing device described in Appendix 5 is the information processing device described in Appendix 1, wherein the determination means detects the position of the Adam's apple and the position of the chin around the boundary, and uses the positions of the Adam's apple and the chin as the boundary surrounding information.

[0137] (Supplementary Note 6) The information processing device described in Supplementary Note 6 further includes an estimation means for estimating the angle of the face of the target based on three-dimensional shape data of the target, and a first guide means for outputting first guide information for guiding the angle of the face of the target in the images captured by the first camera and the second camera to an angle suitable for generating the three-dimensional shape data based on the estimation result of the estimation means, wherein the generation means regenerates the three-dimensional shape data of the target using images captured by the first camera and the second camera after outputting the first guide information, which is the information processing device described in any one of claims 1 to 5.

[0138] (Appendix 7) The information processing device described in Appendix 7 is the information processing device described in any one of Appendices 1 to 6, further comprising a third camera that photographs the face of the subject from the front, a display unit that can display the image photographed by the third camera, and a display control means that displays information regarding the determination result by the determination means on the display unit together with the image photographed by the third camera.

[0139] (Appendix 8) The information processing device described in Appendix 8 is the information processing device described in any one of Appendices 1 to 7, further comprising a second guide means that outputs second guide information for capturing an image including both the face area and the neck area when an image captured by at least one of the first camera and the second camera does not include both the face area and the neck area.

[0140] (Supplementary Note 9) The information processing method described in Supplementary Note 9 is an information processing method for controlling an information processing device that includes a first unit having a first camera that photographs an object from a first direction and a first illumination unit that irradiates the object with light from the first direction, and a second unit having a second camera that photographs the object from a second direction and a second illumination unit that irradiates the object with light from the second direction, wherein the information processing method generates three-dimensional shape data of the object using an image photographed by the first camera and an image photographed by the second camera, and determines whether the images photographed by the first camera and the second camera are suitable for generating the three-dimensional shape data based on boundary surrounding information that is information about the periphery of the boundary between the face region and neck region of the object.

[0141] (Supplementary Note 10) The recording medium described in Supplementary Note 10 is an information processing method for controlling an information processing device including: a first unit having a first camera that photographs an object from a first direction and a first irradiation unit that irradiates the object with light from the first direction; and a second unit having a second camera that photographs the object from a second direction and a second irradiation unit that irradiates the object with light from the second direction, wherein the information processing method generates three-dimensional shape data of the object using an image photographed by the first camera and an image photographed by the second camera, and determines whether the images photographed by the first camera and the second camera are suitable for generating the three-dimensional shape data based on boundary surrounding information, which is information regarding the periphery of the boundary between the face region and the neck region of the object. The information processing method is a recording medium having recorded thereon a computer program for causing the information processing method to be executed.

[0142] (Supplementary Note 11) The computer program described in Supplementary Note 11 is an information processing method for controlling an information processing device including: a first unit having a first camera that photographs an object from a first direction and a first illumination unit that irradiates the object with light from the first direction; and a second unit having a second camera that photographs the object from a second direction and a second illumination unit that irradiates the object with light from the second direction, wherein the computer program causes the information processing method to generate three-dimensional shape data of the object using an image photographed by the first camera and an image photographed by the second camera, and to determine whether the images photographed by the first camera and the second camera are suitable for generating the three-dimensional shape data based on boundary surrounding information that is information regarding the periphery of the boundary between the face region and the neck region of the object.

[0143] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and information processing devices, information processing methods, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure.

[0144] REFERENCE SIGNS LIST 1 Information processing device 11 Processor 12 RAM 13 ROM 14 Storage device 15 Input device 16 Output device 17 Data bus 21 First unit 211 First camera 212 First irradiation unit 22 Second unit 221 Second camera 222 Second irradiation unit 300 Third camera 50 Object 110 Generation unit 120 Determination unit 121 Adam's apple detection unit 122 Chin detection unit 125 Face angle estimation unit 130 Guidance unit 140 Camera control unit 150 First guide unit 160 Display control unit 170 Display unit 180 Second guide unit

Claims

1. An information processing device comprising: a first unit having a first camera that photographs an object from a first direction and a first illumination unit that irradiates the object with light from the first direction; a second unit having a second camera that photographs the object from a second direction and a second illumination unit that irradiates the object with light from the second direction; a generation means that generates three-dimensional shape data of the object using images photographed by the first camera and images photographed by the second camera; and a determination means that determines whether the images photographed by the first camera and the second camera are suitable for generating the three-dimensional shape data based on boundary surrounding information, which is information regarding the periphery of the boundary between the face area and neck area of ​​the object.

2. The information processing device according to claim 1, wherein the determining means uses a difference in brightness between the face area and the neck area around the boundary as the boundary periphery information.

3. An information processing device as described in claim 2, further comprising a guidance means for outputting guidance information that prompts the subject to gradually change the angle of his or her face, and wherein the determination means obtains the brightness difference between the face area and the neck area around the boundary from multiple images taken while the subject is changing the angle of his or her face in response to the guidance information.

4. An information processing device as described in claim 2, further comprising a control means for controlling the angle of the first camera and the second camera relative to the face of the subject to be gradually changed, and the determination means obtains the brightness difference between the face area and the neck area around the boundary from multiple images taken by the control means while changing the angle of the first camera and the second camera.

5. The information processing device according to claim 1, wherein said determining means detects the positions of the Adam's apple and the chin around said boundary, and uses said Adam's apple and chin positions as said boundary periphery information.

6. An information processing device as described in any one of claims 1 to 5, further comprising: an estimation means for estimating the angle of the face of the target based on the three-dimensional shape data of the target; and a first guide means for outputting first guide information that guides the angle of the face of the target in the images taken by the first camera and the second camera to an angle suitable for generating the three-dimensional shape data based on the estimation results of the estimation means, wherein the generation means regenerates the three-dimensional shape data of the target using images taken by the first camera and the second camera after outputting the first guide information.

7. An information processing device as described in any one of claims 1 to 5, further comprising: a third camera that photographs the face of the subject from the front; a display unit that can display the image photographed by the third camera; and a display control means that displays information regarding the judgment result by the judgment means on the display unit together with the image photographed by the third camera.

8. An information processing device according to any one of claims 1 to 5, further comprising a second guide means for outputting second guide information for capturing an image including both the face area and the neck area when an image captured by at least one of the first camera and the second camera does not include both the face area and the neck area.

9. An information processing method for controlling an information processing device comprising: a first unit having a first camera that photographs an object from a first direction and a first illumination unit that illuminates the object from the first direction; and a second unit having a second camera that photographs the object from a second direction and a second illumination unit that illuminates the object from the second direction, wherein the information processing method generates three-dimensional shape data of the object using images photographed by the first camera and images photographed by the second camera, and determines whether the images photographed by the first camera and the second camera are suitable for generating the three-dimensional shape data based on boundary surrounding information, which is information regarding the periphery of the boundary between the face area and neck area of ​​the object.

10. An information processing method for controlling an information processing device comprising: a first unit having a first camera that photographs an object from a first direction and a first illumination unit that illuminates the object from the first direction; and a second unit having a second camera that photographs the object from a second direction and a second illumination unit that illuminates the object from the second direction, wherein the information processing method generates three-dimensional shape data of the object using images photographed by the first camera and images photographed by the second camera, and determines whether the images photographed by the first camera and the second camera are suitable for generating the three-dimensional shape data based on boundary surrounding information, which is information regarding the periphery of the boundary between the face area and neck area of ​​the object. A recording medium having recorded thereon a computer program for executing the information processing method.

Images