Information processing device, information processing system, information processing method, and computer program

Abstract

An information processing device 1 is provided with: a guiding unit 11 which guides a target such that a prescribed portion of the target is aligned with a prescribed overlapping portion where a light pattern projected onto the target from a first direction and a light pattern projected onto the target from a second direction different from the first direction overlap; and a generating unit 12 which generates three-dimensional shape data of the target on the basis of a first image obtained by imaging, from the first direction, the target that has been moved such that the prescribed portion thereof aligns with the prescribed overlapping portion and on the basis of a second image obtained by imaging, from the second direction, said target.

Description

Information processing device, information processing system, information processing method, and recording medium

[0001] The present disclosure relates to the technical fields of an information processing device, an information processing system, an information processing method, and a recording medium.

[0002] Patent Document 1 discloses a technology for generating a three-dimensional shape model of a three-dimensional object by irradiating a predetermined surface of the three-dimensional object with light simultaneously from the front, lower left, and lower right of the predetermined surface, uniformly illuminating the predetermined surface, photographing the light-irradiated surface of the three-dimensional object from at least one direction, obtaining an image of the predetermined surface of the three-dimensional object without shadows or shading, and obtaining the brightness value at each point of the three-dimensional shape of the three-dimensional object as visual information based on three-dimensional shape data representing the three-dimensional shape of the predetermined surface of the three-dimensional object and the photographed image.

[0003] Japanese Patent Application Laid-Open No. 2003-208601

[0004] An object of the present disclosure is to provide an information processing device, an information processing system, an information processing method, and a recording medium that aim to improve upon the techniques disclosed in prior art documents.

[0005] One aspect of the information processing device includes a guidance means for guiding the object so that a predetermined portion of the object is aligned with a predetermined overlapping portion between a light pattern projected onto the object from a first direction and a light pattern projected onto the object from a second direction different from the first direction, and a generation means for generating three-dimensional shape data of the object based on a first image taken from the first direction and a second image taken from the second direction of the object with the predetermined portion moved to the predetermined overlapping portion.

[0006] One aspect of the information processing system includes an information processing device having a first projection means that projects a light pattern onto an object from a first direction, a second projection means that projects the light pattern onto the object from a second direction different from the first direction, a first imaging means that photographs the object from the first direction to generate a first image, a second imaging means that photographs the object from the second direction to generate a second image, a guidance means that guides the object to align a predetermined portion of the object with a predetermined overlapping portion between the light pattern projected onto the object by the first projection means and the light pattern projected onto the object by the second projection means, and a generation means that generates three-dimensional shape data of the object based on the first image of the object with the predetermined portion moved to the predetermined overlapping portion and the second image.

[0007] One aspect of the information processing method involves guiding the object so that a predetermined portion of the object is aligned with a predetermined overlapping portion between a light pattern projected onto the object from a first direction and a light pattern projected onto the object from a second direction different from the first direction, and generating three-dimensional shape data of the object based on a first image taken from the first direction and a second image taken from the second direction of the object that has moved the predetermined portion to the predetermined overlapping portion.

[0008] One aspect of the recording medium has recorded thereon a computer program for causing a computer to execute an information processing method in which the computer guides the object to align a predetermined portion of the object with a predetermined overlapping portion between a light pattern projected onto the object from a first direction and a light pattern projected onto the object from a second direction different from the first direction, and generates three-dimensional shape data of the object based on a first image taken from the first direction and a second image taken from the second direction, after the predetermined portion of the object has been moved to the predetermined overlapping portion.

[0009] 1 is a block diagram showing a configuration of a first information processing device according to the present disclosure. FIG. 2 is a conceptual diagram of a processing operation of a first information processing device according to the present disclosure. FIG. 3 is a schematic diagram showing a configuration of an information processing system according to the present disclosure. FIG. 4 is a conceptual diagram of a processing operation of a second information processing device according to the present disclosure. FIG. 5 is a flowchart showing a processing operation of a second information processing device according to the present disclosure. FIG. 6 is a conceptual diagram of a processing operation of a second information processing device according to the present disclosure. FIG. 7 is a conceptual diagram of a processing operation of a second information processing device according to the present disclosure. FIG. 8 is a conceptual diagram of a processing operation of a second information processing device according to the present disclosure. FIG. 9 is a block diagram showing a configuration of a third information processing device according to the present disclosure. FIG. 10 is a conceptual diagram of a processing operation of a third information processing device according to the present disclosure. FIG. 11 is a block diagram showing a configuration of a fourth information processing device according to the present disclosure. FIG. 12 is a conceptual diagram of a processing operation of a fourth information processing device according to the present disclosure.

[0010] Hereinafter, embodiments of an information processing device, an information processing system, an information processing method, and a recording medium will be described with reference to the drawings. [1: First Embodiment]

[0011] A first embodiment of an information processing device, an information processing system, an information processing method, and a recording medium will be described below. Hereinafter, the first embodiment of the information processing device, the information processing system, the information processing method, and the recording medium will be described using a first information processing device 1 according to the present disclosure. [1-1: Configuration of Information Processing Device 1]

[0012] FIG. 1 is a block diagram showing the configuration of a first information processing device 1 according to the present disclosure. As shown in FIG. 1, the information processing device 1 includes a guidance unit 11 and a generation unit 12. The guidance unit 11 guides the target so that a predetermined portion of the target is aligned with a predetermined overlapping portion between a light pattern projected onto the target from a first direction and a light pattern projected onto the target from a second direction. The first direction and the second direction are different directions. The generation unit 12 generates three-dimensional shape data of the target, the predetermined portion of which has been moved to the predetermined overlapping portion, based on a first image captured from the first direction and a second image captured from the second direction.

[0013] FIG. 2 is a conceptual diagram of the processing operation of the first information processing device 1 according to the present disclosure. Each of FIGS. 2(a) to 2(e) illustrates an example of an image of a target T captured from a predetermined direction. FIG. 2(a) illustrates a case in which a predetermined portion N of the target T overlaps a predetermined overlapping portion O between a first light pattern O1 projected onto the target T from a first direction and a second light pattern O2 projected onto the target T from a second direction. On the other hand, each of FIGS. 2(b) and 2(c) illustrates a case in which a predetermined portion N of the target T does not overlap a predetermined overlapping portion O between the first light pattern O1 projected onto the target T from the first direction and the second light pattern O2 projected onto the target T from the second direction. Each of FIGS. 2(a) to 2(c) illustrates a case in which the target T is located at a desired depth in a predetermined direction. As illustrated in each of FIGS. 2( a) to 2(c), when the target T is located at a desired depth in a predetermined direction, the first light pattern O1 projected onto the target T from a first direction and the second light pattern O2 projected onto the target T from a second direction overlap. On the other hand, each of FIGS. 2(d) and 2(e) illustrates a case where the target T is not located at a desired depth in a predetermined direction. As illustrated in each of FIGS. 2(d) and 2(e), when the target T is not located at a desired depth in a predetermined direction, the first light pattern O1 projected onto the target T from a first direction and the second light pattern O2 projected onto the target T from a second direction do not overlap. FIG. 2(d) illustrates a case where the target T is located closer to the target T than the desired depth in a predetermined direction. Furthermore, FIG. 2(e) illustrates a case where the target T is located further back than the desired depth in a predetermined direction. Note that FIG. 2 illustrates an ideal situation. The first light pattern O1 and the second light pattern O2 are deformed according to the three-dimensional shape of the target T. Therefore, even when the target T is located at a desired depth in a predetermined direction, there may be a misalignment in the overlap between the first light pattern O1 and the second light pattern O2.

[0014] The guiding unit 11 guides the object T to the state illustrated in Fig. 2(a). The generating unit 12 generates three-dimensional shape data of the object T based on an image of the object T photographed from a first direction in the state illustrated in Fig. 2(a) and an image of the object T photographed from a second direction in the state illustrated in Fig. 2(a). [1-2: Technical Effects of the Information Processing Device 1]

[0015] The first information processing device 1 according to the present disclosure generates three-dimensional shape data of the object T based on an image of the object T captured while guiding the object T to align a predetermined portion N with a predetermined overlapping portion between the first light pattern O1 and the second light pattern O2, and therefore can appropriately generate three-dimensional shape data.

[0016] A second embodiment of an information processing device, an information processing system, an information processing method, and a recording medium will be described below. Hereinafter, the second embodiment of an information processing device, an information processing system, an information processing method, and a recording medium will be described using a first information processing system S according to the present disclosure. [2-1: Configuration of the information processing system S]

[0017] Fig. 3 is a schematic diagram showing the configuration of an information processing system S. As shown in Fig. 3, the first information processing system S according to the present disclosure includes a projection photography apparatus 100 and a second information processing apparatus 2. Fig. 3 illustrates an example in which the object T whose three-dimensional shape is to be measured is the head and neck of a person, mainly including the person's face.

[0018] The projection and photography device 100 projects and photographs a person's head and neck, primarily the face, and includes a first projector P1, a second projector P2, a first camera C1, a second camera C2, and a third camera C3.

[0019] The first projector P1 irradiates light onto the target T from a first direction. Fig. 3 illustrates a case where the first direction is the left side as viewed from the target T. The first camera C1 photographs the target T irradiated with light by the first projector P1 from the first direction.

[0020] The second projector P2 irradiates light onto the target T from a second direction. Fig. 3 illustrates a case where the second direction is the right side as viewed from the target T. The second camera C2 photographs the target T irradiated with light by the second projector P2 from the second direction. The third camera C3 photographs the target T from a third direction. Photographing the target T from the third direction may be photographing the target T from the front.

[0021] The first projector P1 and the second projector P2 are not particularly limited, and may be, for example, a DLP (Digital Light Processing) projector, a liquid crystal projector, or the like. DLP projectors, liquid crystal projectors, and the like are capable of projecting any light pattern at high speed, which is preferable for shortening the time required to measure the shape of the target T. Shortening the measurement time is particularly advantageous when measuring the three-dimensional shape of a moving object (moving body), such as when performing face recognition of a person. Specific examples of the light patterns projected onto the target T by the first projector P1 and the second projector P2 will be described later.

[0022] The first camera C1, the second camera C2, and the third camera C3 are configured to be able to capture images of the target T. The first camera C1, the second camera C2, and the third camera C3 may include a solid-state imaging element such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The first camera C1, the second camera C2, and the third camera C3 may also include an optical system that forms an image of the subject on the imaging surface of the solid-state imaging element, a signal processing circuit that processes the output of the solid-state imaging element to obtain a luminance value for each pixel, and the like.

[0023] 3, the posture of the person may be supported by a support device 200. Each of a seat portion 201, a back support 202, and a headrest 203 included in the support device 200 may be configured so that the position, angle, etc. can be changed so that the person can assume a desired posture. [2-2: Configuration of the information processing device 2]

[0024] 4 is a block diagram showing the configuration of a second information processing device 2 according to the present disclosure. As shown in FIG. 4, the information processing device 2 includes a calculation device 21 and a storage device 22. The information processing device 2 may further include a communication device 23, an input device 24, and an output device 25. However, the information processing device 2 does not necessarily have to include at least one of the communication device 23, the input device 24, and the output device 25. The calculation device 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26.

[0025] The arithmetic device 21 includes, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array). The arithmetic device 21 reads a computer program. For example, the arithmetic device 21 may read a computer program stored in the storage device 22. For example, the arithmetic device 21 may read a computer program stored in a computer-readable, non-transitory recording medium using a recording medium reading device (e.g., an input device 24 described later) not shown in the drawings that is included in the information processing device 2. The arithmetic device 21 may acquire (i.e., download or read) the computer program from a device (not shown) located outside the information processing device 2 via the communication device 23 (or another communication device). The arithmetic device 21 executes the read computer program. As a result, logical functional blocks for executing the operations to be performed by the information processing device 2 are realized within the arithmetic device 21. In other words, the arithmetic device 21 can function as a controller for realizing logical functional blocks for executing the operations (in other words, processing) to be performed by the information processing device 2.

[0026] FIG. 4 shows an example of logical functional blocks implemented within the computing device 21 to execute information processing operations. As shown in FIG. 3 , the computing device 21 implements a guidance unit 211, which is a specific example of a "guiding means" described in the appendix below; a generation unit 212, which is a specific example of a "generating means" described in the appendix below; a determination unit 213, which is a specific example of a "determination means" described in the appendix below; a camera control unit 214, which is a specific example of a "photography control means" described in the appendix below; and a lighting control unit 215. However, at least one of the determination unit 213, camera control unit 214, and lighting control unit 215 does not have to be implemented within the computing device 21. Details of the operations of the guidance unit 211, generation unit 212, determination unit 213, camera control unit 214, and lighting control unit 215 will be described later with reference to FIGS. 5 to 10 .

[0027] The storage device 22 can store desired data. For example, the storage device 22 may temporarily store a computer program executed by the arithmetic device 21. The storage device 22 may temporarily store data that the arithmetic device 21 temporarily uses when the arithmetic device 21 is executing a computer program. The storage device 22 may store data that the information processing device 2 stores long-term. The storage device 22 may include at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device. In other words, the storage device 22 may include a non-temporary recording medium.

[0028] The communication device 23 is capable of communicating with devices external to the information processing device 2 via a communication network (not shown). The communication device 23 may be a communication interface based on standards such as Ethernet (registered trademark), Wi-Fi (registered trademark), Bluetooth (registered trademark), or USB (Universal Serial Bus).

[0029] The input device 24 is a device that accepts information input to the information processing device 2 from outside the information processing device 2. For example, the input device 24 may include an operation device (e.g., at least one of a keyboard, a mouse, and a touch panel) that can be operated by an operator of the information processing device 2. For example, the input device 24 may include a reading device that can read information recorded as data on a recording medium that can be externally attached to the information processing device 2.

[0030] The output device 25 is a device that outputs information to the outside of the information processing device 2. For example, the output device 25 may output information as an image. That is, the output device 25 may include a display device (a so-called display) that can display an image showing the information to be output. For example, the output device 25 may output information as sound. That is, the output device 25 may include an audio device (a so-called speaker) that can output sound. For example, the output device 25 may output information on paper. That is, the output device 25 may include a printing device (a so-called printer) that can print desired information on paper. [2-3: Information Processing Operation Performed by the Information Processing Device 2]

[0031] The flow of the information processing operation performed by the information processing device 2 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of the information processing operation performed by the information processing device 2.

[0032] 5, the illumination control unit 215 controls the first projector P1 and the second projector P2 to project light patterns (step S20). The light patterns projected by the first projector P1 and the second projector P2 may be light patterns in which bright patterns overlap at desired positions, as shown in FIG. 2A. The camera control unit 214 controls the third camera C3 to acquire a third image captured by the third camera C3 (step S21).

[0033] Based on the third image, the determination unit 213 determines whether a predetermined overlapping portion (hereinafter referred to as "overlapping portion O") between the first light pattern O1 projected onto the target T by the first projector P1 and the second light pattern O2 projected onto the target T by the second projector P2 and the predetermined portion N are aligned (step S22). The overlapping portion O may be a bright pattern portion of the light patterns that overlap at a desired position, as in the example illustrated in FIG. 2A. The predetermined portion N may be the nose of the target T. The overlapping portion O may be a predetermined range including the overlapping portion of the light patterns. That is, the determination unit 213 may determine whether the overlapping portion of the light patterns and the predetermined portion N are aligned by a predetermined amount or more. In this embodiment, the overlapping portion O and the predetermined portion N do not need to be perfectly aligned, and the determination unit 213 may allow the overlapping portion O to exist within a predetermined range around the predetermined portion N.

[0034] If the overlapping portion O and the predetermined portion N are not aligned (step S22: No), the guiding unit 211 guides the target T to align the predetermined portion N with the overlapping portion O (step S23). The guiding unit 211 may guide the target T by having the target T visually recognize a third image of the target T as illustrated in Fig. 6, for example. For example, although not shown, the projection photography apparatus 100 may be provided with a display in a location where the target T facing forward can be visually recognized, and the guiding unit 211 may display the third image on this display.

[0035] Each of Figures 6(a) to 6(e) illustrates a third image of the target T. Figure 6(a) illustrates a case where a predetermined portion N of the target T overlaps with an overlapping portion O between a first light pattern O1 projected onto the target T from a first direction and a second light pattern O2 projected onto the target T from a second direction. On the other hand, each of Figures 6(b) and 6(c) illustrates a case where a predetermined portion N of the target T does not overlap with an overlapping portion O between a first light pattern O1 projected onto the target T from a first direction and a second light pattern O2 projected onto the target T from a second direction. Each of Figures 6(a) to 6(c) illustrates a case where the target T is located at a desired depth in the optical axis direction of the third camera C3. On the other hand, each of Figures 6(d) and 6(e) illustrates a case where the target T is not located at a desired depth in the optical axis direction of the third camera C3. 6(d) and 6(e), the first light pattern O1 projected onto the target T from the first direction and the second light pattern O2 projected onto the target T from the second direction do not overlap. Fig. 6(d) illustrates a case where the target T is closer to the target T than the desired depth in the optical axis direction of the third camera C3. Fig. 6(e) illustrates a case where the target T is farther from the desired depth in the optical axis direction of the third camera C3.

[0036] FIG. 7 illustrates a third image of the target T onto which a light pattern different from the light pattern illustrated in FIG. 6 is projected. A light pattern having a small-angle vertex is projected onto the target T from each of a first direction and a second direction. When this light pattern is projected, if the vertex of the first light pattern O1 and the vertex of the second light pattern O2 are present within a predetermined portion N of the target T, as illustrated in FIG. 7A, the determination unit 213 determines that the situation is equivalent to a situation in which the predetermined overlapping portion O and the predetermined portion N are overlapped. On the other hand, if the vertex of the first light pattern O1 and the vertex of the second light pattern O2 are not present within the predetermined portion N of the target T, as illustrated in FIG. 7B, the determination unit 213 determines that the situation is equivalent to a situation in which the predetermined overlapping portion O and the predetermined portion N are not overlapped.

[0037] Furthermore, the guiding unit 211 may output guidance information indicating to the target T how to specifically move so that the overlapping portion O and the predetermined portion N are aligned, depending on the positional relationship between the overlapping portion O and the predetermined portion N. For example, as illustrated in FIG. 6( b), when the predetermined portion N is to the left of the overlapping portion O as seen from the target T, the guiding unit 211 may guide the target T to move its face a little more to the right. Furthermore, as illustrated in FIG. 6( c), when the predetermined portion N is below the overlapping portion O, the guiding unit 211 may guide the target T to lift its face a little more.

[0038] If the overlapping portion O and the specified portion N are aligned (step S22: Yes), the camera control unit 214 controls the first camera C1 and the second camera C2 to acquire the first image taken by the first camera C1 and the second image taken by the second camera C2 (step S24).

[0039] The generation unit 212 generates three-dimensional shape data of the object T based on a first image captured from a first direction and a second image captured from a second direction of the object T with the predetermined portion N moved to the overlapping portion O. Ideally, the generation unit 212 generates three-dimensional shape data of the object T based on the first image of the object T captured from the first direction when the overlapping portion O and the predetermined portion N are overlapped, and the second image of the object T captured from the second direction (step S25). Here, the overlapping portion O and the predetermined portion N may be overlapped by a predetermined percentage or more. The generation unit 212 measures the three-dimensional shape of the object T based on the first image and the second image when the overlapping portion O and the predetermined portion N are overlapped, and generates three-dimensional shape data.

[0040] The generation unit 212 may measure the three-dimensional shape of the target T using, for example, a sinusoidal grating phase shift method. The sinusoidal grating phase shift method is a method in which a sinusoidal pattern is projected onto the target T while gradually shifting the phase, and the three-dimensional shape of the target T is identified based on an image of the target T onto which the light pattern is projected. The sinusoidal grating phase shift method calculates a phase value for each part of the target T corresponding to each pixel based on the luminance value of each pixel in the image, and calculates three-dimensional coordinates based on the phase value. [2-4-1: Another First Example of a Light Pattern Projected onto the Target T]

[0041] In the first example, the illumination control unit 215 controls the first projector P1 to project a first linear light pattern and the second projector P2 to project a second linear light pattern that intersects with the first linear light pattern at a predetermined position. The guidance unit 211 guides the target T to align a predetermined portion N of the target T with an overlapping portion O that includes the predetermined position.

[0042] FIG. 8 is a conceptual diagram of a first example of another processing operation of the second information processing device 2 according to the present disclosure. Each of FIGS. 8(a) to 8(c) illustrates a third image of the target T captured by the third camera C3. FIG. 8 illustrates a case in which a predetermined portion N of the target T overlaps an overlapping portion O between a first light pattern O1 projected onto the target T from a first direction and a second light pattern O2 projected onto the target T from a second direction. FIG. 8(a) illustrates a case in which a horizontal first light pattern O1 is projected from a first direction, and a vertical second light pattern O2 is projected from a second direction. FIG. 8(b) illustrates a case in which a linear first light pattern O1 inclined relative to the vertical is projected from a first direction, and a linear second light pattern O2 intersecting the linear first light pattern O1 is projected from a second direction. Figure 8 (c) illustrates an example in which a first light pattern O1 consisting of two intersecting straight lines is projected from a first direction, and a second light pattern O2 consisting of two intersecting straight lines that do not overlap with the first light pattern O1 is projected from a second direction.

[0043] The guiding unit 211 guides the target T so that a predetermined portion N of the target T is aligned with an overlapping portion O where the first light pattern O1 and the second light pattern O2 intersect, as illustrated in Fig. 8. The generating unit 212 measures the three-dimensional shape of the target T based on an image of the target T captured from a first direction and an image of the target T captured from a second direction in the state illustrated in Fig. 8. [2-4-2: Another Second Example of Light Pattern Projected onto the Target T]

[0044] In the second example, the illumination control unit 215 controls the first projector P1 to project a first light pattern O1 having a dark pattern on the inside. The illumination control unit 215 also controls the second projector P2 to project a second light pattern O2 having a dark pattern on the inside, so that the dark pattern included in the first light pattern O1 and the dark pattern included in the second light pattern O2 overlap at a desired depth to form a dark pattern portion OD. The guidance unit 211 guides the target T to align a predetermined portion N of the target T with the dark pattern portion OD as the overlapping portion O. That is, the overlapping portion O may be a portion where light patterns included in the light patterns overlap, or a portion where dark patterns included in the light patterns overlap. The guiding unit 211 guides the target T to align a predetermined portion N of the target T with a dark pattern portion OD where a dark pattern included in the first light pattern O1 projected onto the target T by the first projector P1 and a dark pattern included in the second light pattern O2 projected onto the target T by the second projector P2 overlap. In a second example, the predetermined portion N of the target T may be a central region of the face of the target T, including the eyes.

[0045] 9A and 9B are conceptual diagrams illustrating a second example of another processing operation of the second information processing device 2 according to the present disclosure. Each of FIGS. 9A to 9E illustrates a third image captured by the third camera C3. FIG. 9A illustrates an example of a first light pattern O1 projected from a first direction, and FIG. 9B illustrates an example of a second light pattern O2 projected from a second direction. As illustrated in FIGS. 9A and 9B, the light pattern in the second example may be donut-shaped.

[0046] 9(c) illustrates a case where the dark patterns included in the first light pattern O1 and the dark patterns included in the second light pattern O2 do not entirely overlap, and the target T is not located in a location that has a predetermined positional relationship with the third camera C3. On the other hand, FIG. 9(d) illustrates a case where the dark patterns included in the first light pattern O1 and the dark patterns included in the second light pattern O2 entirely overlap, and the target T is located in a location that has a predetermined positional relationship with the third camera C3. The determination unit 213 may determine whether the dark patterns included in the first light pattern O1 and the dark patterns included in the second light pattern O2 overlap at a desired depth based on the area of ​​the dark pattern portion OD included in the third image.

[0047] In the second example, the determination unit 213 determines whether the dark pattern portion OD overlaps with the predetermined portion N. As illustrated in Fig. 9(e), the determination unit 213 may determine whether the dark pattern portion OD overlaps with the central region of the face including the eyes as the predetermined portion N.

[0048] The guidance unit 211 guides the object T so that a predetermined portion N of the object T is aligned with the dark pattern portion OD, as illustrated in FIG. 9( e). Since people often dislike glare, it is relatively easy to guide the central region of the face, including the eyes, into the dark pattern portion OD. The generation unit 212 measures the three-dimensional shape of the object T based on an image of the object T photographed from a first direction and an image of the object T photographed from a second direction in the state illustrated in FIG. 9( e).

[0049] The illumination control unit 215 may control the first projector P1 and the second projector P2 to project the light pattern illustrated in Fig. 9 in addition to or instead of the light pattern illustrated in at least one of Fig. 2 and Fig. 8. [2-4-3: Third Example of Light Pattern Projected onto Target T]

[0050] In a third example, the illumination control unit 215 controls at least one of the first projector P1 and the second projector P2 to project a light pattern including multiple light points. FIG. 10 is a conceptual diagram of a third example of another processing operation of the second information processing device 2 according to the present disclosure. Each of FIGS. 10(a) to 10(c) illustrates a third image captured by the third camera C3. FIG. 10(a) shows an example of a first light pattern O1 projected from a first direction. As illustrated in FIG. 10(a), the light pattern in the third example may include multiple light points OP in an area where a predetermined portion N is desired to be positioned.

[0051] The guidance unit 211 guides the target T so that the number of light points projected onto a predetermined portion N of the target T is equal to or greater than a predetermined number. The determination unit 213 determines whether the number of light points projected onto the predetermined portion N of the target T is equal to or greater than a predetermined number. FIG. 10( b) illustrates a case where the number of light points projected onto the predetermined portion N is less than the predetermined number and the target T is not at the desired location. On the other hand, FIG. 10( c) illustrates a case where the number of light points projected onto the predetermined portion N is equal to or greater than a predetermined number and the target T is at the desired location. The generation unit 212 measures the three-dimensional shape of the target T based on an image of the target T captured from a first direction and an image of the target T captured from a second direction in the state illustrated in FIG. 10( c).

[0052] The illumination control unit 215 may control the first projector P1 and the second projector P2 to project the light pattern illustrated in Fig. 10 in addition to or instead of the light pattern illustrated in at least one of Figs. 2, 8, and 9. [2-5: Technical Effects of the Information Processing System S]

[0053] In measuring three-dimensional shapes, the allowable imaging range for the image used for measurement is narrow. Therefore, even a slight deviation of the target T from the desired location may cause the target T to fall outside the allowable imaging range. In contrast, the information processing system S according to the present disclosure guides the target T to an appropriate location and controls the target T to be photographed when the position of the target T is appropriate, thereby capturing appropriate images and generating appropriate three-dimensional shape data. Furthermore, the information processing system S guides the target T by instructing a change in the position of a predetermined portion N, so the target T can easily move the predetermined portion N to the desired position.

[0054] The illumination control unit 215 can cause the first projector P1 and the second projector P2 to project light patterns that make it easier for the target T to align a predetermined portion N with the overlapping portion O. When a linear light pattern is used, the target T can more easily see the light pattern than when a point-like light pattern is used. Also, when a dark light pattern is used as the overlapping portion O, the target T can more easily align his face (especially his eyes) with the overlapping portion O than when a bright light pattern is used as the overlapping portion O, so it is relatively easy to guide the target T to the overlapping portion O. [3: Third Embodiment]

[0055] A third embodiment of an information processing device, an information processing system, an information processing method, and a recording medium will be described below. The third embodiment of an information processing device, an information processing system, an information processing method, and a recording medium will be described below using a second information processing system S according to the present disclosure. The second information processing system S according to the present disclosure includes a projection photography apparatus 100 and a third information processing device 3. [3-1: Configuration of the information processing device 3]

[0056] The configuration of the third information processing device 3 will be described with reference to Fig. 11. Fig. 11 is a block diagram showing the configuration of the third information processing device 3.

[0057] As shown in FIG. 11 , the third information processing device 3 includes a calculation device 21 and a storage device 22, similar to the second information processing device 2. Furthermore, the third information processing device 3 may include a communication device 23, an input device 24, and an output device 25, similar to the second information processing device 2. However, the information processing device 3 does not need to include at least one of the communication device 23, the input device 24, and the output device 25. The third information processing device 3 differs from the second information processing device 2 in that the calculation device 21 further includes an estimation unit 316, which is a specific example of the "estimation means" described in the appendix below. Other features of the information processing device 3 may be the same as other features of the information processing device 2. Therefore, hereinafter, differences from the embodiments already described will be described in detail, and descriptions of other overlapping parts will be omitted as appropriate. [3-2: Information Processing Operation Performed by Information Processing Device 3]

[0058] The information processing operation described in the third embodiment may be performed in addition to or instead of the information processing operation described in the second embodiment. The information processing operation described in the third embodiment makes it possible to more accurately align the position of the target T in the depth direction.

[0059] The illumination control unit 315 controls the first projector P1 and the second projector P2 to project vertical linear light patterns. The illumination control unit 315 controls the vertical linear light patterns to overlap at desired positions.

[0060] 12A to 12E are conceptual diagrams illustrating an example of a processing operation of the third information processing device 3 according to the present disclosure. Each of Fig. 12A to 12E illustrates a third image of the target T captured by the third camera C3. Fig. 12A illustrates a case in which a first light pattern O1 in the form of a vertical line projected onto the target T from a first direction and a second light pattern O2 in the form of a vertical line projected onto the target T from a second direction overlap at a predetermined portion N.

[0061] The first light pattern O1 and the second light pattern O2 are controlled to overlap at a desired position. Therefore, when the first light pattern O1 and the second light pattern O2 overlap at a predetermined portion N, the predetermined portion N is at a desired position.

[0062] 12(b) illustrates the position and shape of the first light pattern O1 when the specified portion N is not at the desired position. As illustrated in FIG. 12(b), when the target T is closer than the desired position, the first light pattern O1N is located to the left of the target T as seen from the target T compared to when the target T is at the desired position, and has a convex shape to the left as seen from the target T. On the other hand, as illustrated in FIG. 12(b), when the target T is farther away than the desired position, the first light pattern O1F is located to the right of the target T as seen from the target T compared to when the target T is at the desired position, and has a convex shape to the left as seen from the target T.

[0063] 12(c) illustrates the position and shape of the second light pattern O2 when the specified portion N is not at the desired position. As illustrated in FIG. 12(c), when the target T is closer than the desired position, the second light pattern O2N is located to the right of the target T as seen from the target T compared to when the target T is at the desired position, and has a convex shape to the right as seen from the target T. On the other hand, as illustrated in FIG. 12(c), when the target T is farther away than the desired position, the second light pattern O2F is located to the left of the target T as seen from the target T compared to when the target T is at the desired position, and has a convex shape to the right as seen from the target T.

[0064] 12(d) illustrates the positions and shapes of the first light pattern O1N and the second light pattern O2N when the target T is closer than the desired position. As illustrated in FIG. 12(d), the first light pattern O1N and the second light pattern O2N are positioned outside the overlapping position and have outwardly convex shapes.

[0065] 12( e) illustrates the positions and shapes of the first light pattern O1F and the second light pattern O2F when the target T is farther away than the desired position. As illustrated in FIG. 12( e), the first light pattern O1F and the second light pattern O2F are positioned outside the overlapping position and have an inwardly convex shape.

[0066] The estimation unit 316 estimates the distance between the third camera C3 and the target T based on at least one of the position and shape of the light pattern projected onto the target T included in the third image. For example, the estimation unit 316 may estimate the distance between the third camera C3 and the target T by referring to a table storing a correspondence between the position of the light pattern projected onto the target T included in the third image (e.g., deviation from a predetermined position where the light pattern overlaps) and the distance between the third camera C3 and the target T. The estimation unit 316 may also estimate the distance between the third camera C3 and the target T by referring to a table storing a correspondence between the shape of the light pattern projected onto the target T included in the third image (e.g., deviation from a linear shape) and the distance between the third camera C3 and the target T. The guidance unit 311 guides the target T according to the distance between the third camera C3 and the target T. [3-3: Modification]

[0067] In a modified example, the lighting control unit 315 may control the first projector P1 to project a light pattern of a first color, and the second projector P2 to project a light pattern of a second color different from the first color. The lighting control unit 315 may control the first projector P1 to project a vertical line-shaped light pattern of the first color onto the target T, and the second projector P2 to project a vertical line-shaped light pattern of the second color onto the target T. The vertical line-shaped light pattern of the first color and the vertical line-shaped light pattern of the second color overlap when the distance between the third camera C3 and the target T is a desired distance.

[0068] The estimation unit 316 estimates the distance between the third camera C3 and the target T based on at least one of the position and shape of the first light pattern O1 of the first color projected onto the target T included in the third image, and at least one of the position and shape of the second light pattern O2 of the second color projected onto the target T included in the third image. By having the first projector P1 and the second projector P2 project light patterns of different colors, it becomes easier to determine which projector projected the light, and it becomes easier to determine whether the distance between the third camera C3 and the target T is shorter or longer than a predetermined distance. [3-4: Technical Effects of the Information Processing Device 3]

[0069] The information processing device 3 guides the target T according to the distance between the third camera C3 and the target T estimated based on at least one of the position and shape of the vertical line pattern, so that the target T can be guided to a desired location. Furthermore, by differentiating the color of the first light pattern O1 from the color of the second light pattern O2, it is possible to easily determine whether the distance between the third camera C3 and the target T is shorter or longer than a predetermined distance. [4: Fourth Embodiment]

[0070] A fourth embodiment of an information processing device, an information processing system, an information processing method, and a recording medium will be described below. The fourth embodiment of an information processing device, an information processing system, an information processing method, and a recording medium will be described below using a third information processing system S according to the present disclosure. The third information processing system S according to the present disclosure includes a projection photography apparatus 100 and a fourth information processing device 4. [4-1: Configuration of the information processing device 4]

[0071] 13, the fourth information processing device 4 may have the same configuration as the third information processing device 3. The fourth information processing device 4 differs from the third information processing device 3 in at least one of the first light pattern O1 and the second light pattern O2 projected onto the target T, and in the estimation operation by the estimation unit 416. Other features of the fourth information processing device 4 may be the same as other features of the third information processing device 3. Therefore, hereinafter, only the parts that differ from the embodiments already described will be described in detail, and descriptions of other overlapping parts will be omitted as appropriate. [4-2: Information Processing Operation Performed by the Information Processing Device 4]

[0072] The information processing operation described in the fourth embodiment may be performed in addition to or instead of at least one of the information processing operation described in the second embodiment and the information processing operation described in the third embodiment. The information processing operation described in the fourth embodiment makes it possible to more accurately align the position of the target T in the depth direction.

[0073] The illumination control unit 415 controls at least one of the first projector P1 and the second projector P2. The illumination control unit 415 causes at least one of the first projector P1 and the second projector P2 to project a light pattern in which a plurality of vertical linear beams of light are arranged at predetermined intervals.

[0074] 14A and 14B are conceptual diagrams illustrating an example of a processing operation of the fourth information processing device 4 according to the present disclosure. Each of Fig. 14A and 14B illustrates a third image of the target T captured by the third camera C3. Each of Fig. 14A and 14B illustrates a case where a light pattern in which a plurality of vertical linear beams of light are arranged at predetermined intervals is projected onto the target T from a first direction.

[0075] 14(a), the distance between the target T and the third camera C3 is greater than that in the case illustrated in Fig. 14(b). In this case, the interval Da between the vertical lines of light projected onto the target T in the case illustrated in Fig. 14(a) is narrower than the interval Db between the vertical lines of light projected onto the target T in the case illustrated in Fig. 14(b).

[0076] The estimation unit 416 measures the distance between the vertical line patterns projected onto the target T included in the third image. The estimation unit 416 may measure the distance near a predetermined portion N as the distance between the vertical line patterns. The estimation unit 416 estimates the distance between the third camera C3 and the target T based on the distance between the vertical line patterns. For example, the estimation unit 416 may estimate the distance between the third camera C3 and the target T by referring to a table that stores the correspondence between the distance between the vertical line patterns and the distance between the third camera C3 and the target T. The guidance unit 411 guides the target T according to the distance between the third camera C3 and the target T. [4-3: Technical Effects of the Information Processing Device 4]

[0077] The information processing device 4 guides the target T according to the distance between the third camera C3 and the target T, which is estimated based on the spacing between the vertical line patterns, and therefore can guide the target T to a desired location.

[0078] In each of the above embodiments, even when guidance information is output to the target T, a situation may occur in which the predetermined portion N does not assume the desired position. In this case, the information processing device may change the posture of the target T by moving at least a part of the support 200, thereby aligning the overlapping portion O with the predetermined portion N. For example, if a predetermined number of consecutive shots are taken in which the predetermined portion N does not assume the desired position, the information processing device may control the movement of the support 200. For example, the information processing device may control the movement of the headrest 203. The information processing device may fix the target T to the headrest 203 and move the headrest 203 so that the predetermined portion N assumes the desired position. [5: Supplementary Note]

[0079] The following supplementary notes are further disclosed regarding the above-described embodiment: [Supplementary Note 1] An information processing device comprising: a guiding unit that guides an object so that a predetermined portion of the object is aligned with a predetermined overlapping portion between a light pattern projected onto the object from a first direction and a light pattern projected onto the object from a second direction different from the first direction; and a generating unit that generates three-dimensional shape data of the object, the predetermined portion of which has been moved to the predetermined overlapping portion, based on a first image captured from the first direction and a second image captured from the second direction. [Supplementary Note 2] An information processing system including: a first projection means that projects a light pattern onto an object from a first direction; a second projection means that projects the light pattern onto the object from a second direction different from the first direction; a first photographing means that photographs the object from the first direction to generate a first image; a second photographing means that photographs the object from the second direction to generate a second image; a guidance means that guides the object to align a predetermined part of the object with a predetermined overlapping part between the light pattern projected onto the object by the first projection means and the light pattern projected onto the object by the second projection means; and an information processing device including: a generation means that generates three-dimensional shape data of the object based on the first image and the second image of the object with the predetermined part moved to the predetermined overlapping part. [Supplementary Note 3] The information processing system according to Supplementary Note 2, further comprising: a third imaging means for imaging the object from a third direction different from the first direction and the second direction to generate a third image, wherein the information processing device comprises: a determination means for determining whether the predetermined portion has moved to the predetermined overlapping portion based on the third image, and an imaging control means for controlling the first imaging means and the second imaging means according to a determination result of the determination means. [Supplementary Note 4] The information processing system according to Supplementary Note 2 or 3, wherein the first projection means projects a first linear light pattern, and the second projection means projects a second linear light pattern that intersects with the first linear light pattern at a predetermined position, and the predetermined overlapping portion includes the predetermined position.[Supplementary Note 5] The information processing system described in Supplementary Note 3, wherein the guiding means guides the target so that a predetermined portion of the target is aligned with a dark pattern portion where a dark pattern included in the light pattern projected onto the target by the first projection means and a dark pattern included in the light pattern projected onto the target by the second projection means overlap, and the determining means determines whether the dark pattern portion and the predetermined portion are aligned. [Supplementary Note 6] The information processing system described in Supplementary Note 3, wherein the first projection means and the second projection means project vertical line-shaped light patterns onto the target that overlap when the distance between the third imaging means and the target is a desired distance, and the information processing device includes an estimation means for estimating the distance between the third imaging means and the target based on at least one of a position and a shape of the vertical line-shaped light pattern projected onto the target and included in the third image, and the guiding means provides guidance according to the distance. [Supplementary Note 7] The information processing system described in Supplementary Note 6, wherein the first projection means projects the vertical line-shaped light pattern in a first color, and the second projection means projects the vertical line-shaped light pattern in a second color different from the first color. [Supplementary Note 8] The information processing system described in Supplementary Note 3, wherein at least one of the first projection means and the second projection means projects a light pattern in which a plurality of vertical line-shaped lights are arranged at a predetermined interval, and the information processing device includes an estimation means for estimating a distance between the third imaging means and the target based on an interval between the vertical line-shaped patterns projected onto the target included in the third image, and the guiding means provides guidance according to the distance. [Supplementary Note 9] The information processing system described in Supplementary Note 3, wherein the guiding means outputs instruction information to the target to instruct it to change the position of the predetermined portion according to a positional relationship between the predetermined overlapping portion and the predetermined portion.[Supplementary Note 10] An information processing method comprising: guiding an object so that a predetermined portion of the object is aligned with a predetermined overlapping portion between a light pattern projected onto the object from a first direction and a light pattern projected onto the object from a second direction different from the first direction; and generating three-dimensional shape data of the object based on a first image taken from the first direction and a second image taken from the second direction of the object whose predetermined portion has been moved to the predetermined overlapping portion. [Supplementary Note 11] A recording medium having recorded thereon a computer program for causing a computer to execute an information processing method comprising: guiding the object so that a predetermined portion of the object is aligned with a predetermined overlapping portion between a light pattern projected onto the object from a first direction and a light pattern projected onto the object from a second direction different from the first direction, and generating three-dimensional shape data of the object based on the first image taken from the first direction and the second image taken from the second direction of the object whose predetermined portion has been moved to the predetermined overlapping portion.

[0080] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0081] 1, 2, 3, 4 Information processing device S Information processing system 100 Projection photography device P1 First projector C1 First camera P2 Second projector C2 Second camera C3 Third camera 11, 211, 311, 411 Guidance unit 12, 212 Generation unit 213 Determination unit 214 Camera control unit 215, 315, 415 Lighting control unit 316, 416 Estimation unit

Claims

1. Guiding means for guiding an object so that a predetermined portion of the object is aligned with a predetermined overlap portion of a light pattern projected onto the object from a first direction and a light pattern projected onto the object from a second direction different from the first direction, A generation means that generates three-dimensional shape data of the object based on a first image taken from the first direction and a second image taken from the second direction of the object, after moving the predetermined portion of the object to the predetermined overlapping portion. An information processing device equipped with the following features.

2. A first projection means that projects a light pattern onto a target from a first direction, A second projection means for projecting a light pattern onto the object from a second direction different from the first direction, A first shooting means for photographing the aforementioned object from the first direction and generating a first image, A second imaging means for photographing the aforementioned object from the second direction to generate a second image, An information processing apparatus comprising: guidance means for guiding the object so that a predetermined portion of the object aligns with a predetermined overlapping portion of a light pattern projected onto the object by the first projection means and a light pattern projected onto the object by the second projection means; and generation means for generating three-dimensional shape data of the object based on a first image and a second image of the object with the predetermined portion moved to the predetermined overlapping portion. An information processing system that includes this.

3. The system includes a third imaging means for generating a third image by photographing the object from a third direction different from the first and second directions, The aforementioned information processing device is A determination means for determining whether the predetermined part has moved to the predetermined overlapping part based on the third image, and The system includes a shooting control means that controls the first shooting means and the second shooting means according to the determination result of the determination means. The information processing system according to claim 2.

4. The first projection means projects a first linear light pattern, The second projection means projects a second linear light pattern that intersects the first linear light pattern at a predetermined position. The predetermined overlapping portion includes the predetermined position. The information processing system according to claim 2 or 3.

5. The guiding means guides the object so as to align a predetermined portion of the object with the dark pattern portion where the dark pattern included in the light pattern projected onto the object by the first projection means and the dark pattern included in the light pattern projected onto the object by the second projection means overlap. The determination means determines whether the dark pattern portion and the predetermined portion are combined. The information processing system according to claim 3.

6. The first projection means and the second projection means project a vertical light pattern onto the object that overlaps when the distance between the third imaging means and the object is a desired distance. The information processing device includes estimation means for estimating the distance between the third imaging means and the object based on at least one of the position and shape of the vertical light pattern projected onto the object included in the third image. The guidance means provides guidance according to the distance. The information processing system according to claim 3.

7. The first projection means projects the vertical light pattern of the first color, The second projection means projects the vertical light pattern of a second color different from the first color. The information processing system according to claim 6.

8. At least one of the first projection means and the second projection means projects a light pattern in which a plurality of vertically aligned beams of light are arranged at predetermined intervals. The information processing device includes estimation means for estimating the distance between the third imaging means and the object based on the spacing between the vertical linear patterns projected onto the object included in the third image. The guidance means provides guidance according to the distance. The information processing system according to claim 3.

9. The object is guided so that a predetermined portion of the object aligns with a predetermined overlapping portion of a light pattern projected onto the object from a first direction and a light pattern projected onto the object from a second direction different from the first direction. Based on a first image taken from a first direction and a second image taken from a second direction, the object, with the predetermined portion moved to the predetermined overlapping portion, is used to generate three-dimensional shape data of the object. The information processing method performed by computers.

10. On the computer, The object is guided so that a predetermined portion of the object aligns with a predetermined overlapping portion of a light pattern projected onto the object from a first direction and a light pattern projected onto the object from a second direction different from the first direction. Based on a first image taken from the first direction and a second image taken from the second direction, the object, with its predetermined portion moved to the predetermined overlapping portion, is used to generate three-dimensional shape data of the object. A computer program that executes an information processing method.