Information processing device, information processing method, and information processing program

Abstract

An information processing device according to the present disclosure comprises: an acquisition unit that acquires a first image including a subject; a detection unit that detects a feature region indicating a feature of the subject in the first image; and a generation unit that generates a three-dimensional model of the subject on the basis of the first image and a second image in which the feature region included in the subject is newly captured.

Description

Information Processing Apparatus, Information Processing Method, and Information Processing Program

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and an information processing program.

[0002] Conventionally, a technique for generating a three-dimensional model of a subject by using images of the subject taken from many different viewpoints is known. For example, as a technique for generating a three-dimensional model, there is photogrammetry or the like.

[0003] For example, as a conventional technique, based on the positions of the subject at each time, a composite three-dimensional model including three-dimensional models of the subject at each time generated based on a plurality of viewpoint images at each of at least two times from the first time to the third time is generated (see Patent Document 1 below).

[0004] Japanese Unexamined Patent Application Publication No. 2019-153863

[0005] However, in the above conventional technique, there is room for improvement in the point of photographing the subject so that the subject appears large in order to generate a high-quality three-dimensional model.

[0006] Therefore, the present disclosure proposes an information processing apparatus, an information processing method, and an information processing program capable of generating a high-quality three-dimensional model.

[0007] In order to solve the above problems, an information processing apparatus according to one aspect of the present disclosure includes an acquisition unit that acquires a first image including a subject, a detection unit that detects a feature region indicating a feature of the subject in the first image, and a generation unit that generates a three-dimensional model of the subject based on the first image and a second image in which the feature region included in the subject is newly photographed.

[0008] This figure shows an overview of the information processing system according to the embodiment. This figure shows an example of the configuration of the information processing device according to the embodiment. This is a schematic diagram showing information processing according to the embodiment. This is a schematic diagram showing an example of shooting processing according to the embodiment. This is a schematic diagram showing application example (1) according to the embodiment. This is a schematic diagram showing application example (2) according to the embodiment. This is a conceptual diagram (1) of shooting according to the embodiment. This is an example of a flowchart (1) showing the flow of device control processing according to the embodiment. This is a conceptual diagram (2) of shooting according to the embodiment. This is an example of a flowchart (2) showing the flow of device control processing according to the embodiment. This is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device.

[0009] Embodiments of this disclosure will be described in detail below with reference to the drawings. In each of the following embodiments, the same parts will be denoted by the same reference numerals to avoid redundant descriptions.

[0010] This disclosure will be described in the following order of items. 1. Embodiments 1-1. Overview of the Embodiments 1-2. Overview of the Information Processing System According to the Embodiments 1-3. Configuration of the Information Processing Device According to the Embodiments 1-3-1. About Information Processing 1-3-2. About the Image Capture Process 1-3-3. About Application Example (1) 1-3-4. About Application Example (2) 1-4. Use Case (1) According to the Embodiments 1-4-1. Image Capture in Use Case (1) 1-4-2. Flowchart showing the Procedure for Device Control Processing Related to Use Case (1) 1-5. Use Case (2) According to the Embodiments 1-5-1. Image Capture in Use Case (2) 1-5-2. Flowchart showing the Procedure for Device Control Processing Related to Use Case (2) 1-6. Modifications According to the Embodiments 1-6-1. About the Detection Process (1) 1-6-2. About the Detection Process (2) 1-6-3. About the Generation Process 1-6-4. Regarding reception processing 1-6-5. Regarding the second shooting conditions 2. Other embodiments 3. Summary of the configuration of the information processing device of this disclosure 4. Hardware configuration

[0011] (1. Embodiments) (1-1. Overview of Embodiments) In recent years, there has been a growing need for photogrammetry technology that generates a three-dimensional model of a subject from images taken of the subject from multiple viewpoints. Photogrammetry can generate a high-quality three-dimensional model of a subject by taking images evenly from all directions. Here, a high-quality three-dimensional model refers to, for example, a model with a large number of meshes in terms of shape and high resolution in terms of texture. In general, in order to generate a high-quality three-dimensional model, it is preferable to photograph the subject so that it is large in the image. However, usually, taking images so that the subject is large in the image requires a large number of images, which can increase processing time.

[0012] Furthermore, in typical photogrammetry, it is generally desirable for images containing the subject to overlap by about 60%. However, if an attempt is made to generate a 3D model of a subject based on images with less than 60% overlap, it may not be possible to generate an appropriate 3D model.

[0013] Therefore, in response to the above-mentioned problems, this disclosure proposes an information processing device that generates a three-dimensional model of a subject based on, for example, a first image including the subject and a second image in which a feature region showing the characteristics of the subject is newly captured. As a result, this disclosure can generate a high-quality three-dimensional model. For example, this disclosure does not uniformly capture the subject at high resolution, but rather captures only the feature regions included in the subject that need to be captured at high resolution. As a result, this disclosure can efficiently generate a high-quality three-dimensional model.

[0014] The first image was taken under the first shooting conditions. The second image was taken under the second shooting conditions, which are different from the first shooting conditions. The image taken as the second image is not the first image itself. For example, the second shooting conditions may involve more shots and a higher zoom ratio compared to the first shooting conditions.

[0015] Feature regions include, for example, the gaze region included in the subject (e.g., the region including the face), the region containing shapes that indicate the characteristics of the subject, the region containing hole-like shapes included in the subject, the region containing textual information included in the subject, and the region containing the surface shape of the subject.

[0016] For example, areas containing shapes that represent the characteristics of the subject include areas with fine details, thin or faint areas, and areas that are hidden by the shape. For example, areas containing the surface shape of the subject include areas with fine patterns and areas with poor contrast against the background.

[0017] The issues listed above are merely examples, and the issues that this disclosure aims to solve are not necessarily limited to those listed above; other issues may also be addressed.

[0018] Furthermore, in the following, if there is no need to distinguish between the first image and the second image, they may simply be referred to as "image." Also, in the following, if there is no need to distinguish between the first shooting conditions and the second shooting conditions, they may simply be referred to as "shooting conditions."

[0019] (1-2. Overview of the Information Processing System According to the Embodiment) First, an overview of the information processing system 1 according to the embodiment will be explained using Figure 1. Figure 1 is a diagram showing an overview of the information processing system 1 according to the embodiment.

[0020] In Figure 1, the information processing system 1 includes a shooting system 10 and an information processing device 100. The shooting system 10 and the information processing device 100 are connected by wired or wireless means via a network N such as a LAN (Local Area Network). The information processing device 100 may also be connected to the shooting system 10 via USB (Universal Serial Bus) for communication. Note that the information processing system 1 shown in Figure 1 may include multiple shooting systems 10 and multiple information processing devices 100.

[0021] The shooting system 10 includes a shooting device for photographing a subject, a lighting device for illuminating the subject with light, and a turntable on which the subject is placed. In the example in Figure 1, the shooting system 10 includes a shooting device CA1 fixed to a slider ST1 and a tilter TI1, a shooting device CA2 for photographing the upper angle, a lighting device LE1, and a turntable TT1. In the example in Figure 1, the subject OB1 is placed on the turntable TT1. The shooting device CA1 is fixed so as to be movable up and down by the slider ST1. The angle of view of the shooting device CA1 is set by the tilter TI1.

[0022] A lighting device is, for example, a spotlight that illuminates a subject from a predetermined direction. The lighting device also allows for setting illumination modes such as continuous lighting or strobe light. A photographic device is, for example, a camera. A turntable is, for example, a rotating platform with a rotation mechanism. The turntable rotates at predetermined intervals. For example, the turntable rotates at intervals of 5° or 10°.

[0023] The information processing device 100 is, for example, an information processing device such as a server device, and performs the generation process according to the embodiment. For example, the information processing device 100 acquires a first image including the subject. Subsequently, the information processing device 100 detects feature regions in the first image. Then, the information processing device 100 generates a three-dimensional model of the subject based on the first image and a second image in which feature regions included in the subject have been newly captured.

[0024] (1-3. Configuration of the Information Processing Device According to the Embodiment) Next, the configuration of the information processing device 100 according to the embodiment will be described using Figure 2. Figure 2 is a diagram showing an example of the configuration of the information processing device 100 according to the embodiment.

[0025] As shown in Figure 2, the information processing device 100 includes a communication unit 110, a storage unit 120, a control unit 130, an input unit 140, and a display unit 150. The information processing device 100 may also include an input unit (e.g., a touch panel) for receiving various operations from a user operating the information processing device 100, and a display unit (e.g., a liquid crystal display) for displaying various information.

[0026] The communication unit 110 is implemented, for example, by a NIC (Network Interface Card). The communication unit 110 is connected to a network N (LAN, Internet, NFC (Near Field Communication), Bluetooth®, etc.) by wired or wireless connection and transmits and receives information with the imaging system 10, etc. via the network N.

[0027] The storage unit 120 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or by storage devices such as hard disks and optical discs. It is desirable that the storage unit 120 includes, at least a portion, non-temporary storage devices such as flash memory, hard disks, and optical discs.

[0028] As shown in Figure 2, the storage unit 120 has an image storage unit 121 and a three-dimensional model storage unit 122. Note that each of these parts of the storage unit 120 may be composed of a single semiconductor memory element or hard disk, etc., which may perform all the functions. Alternatively, each of these parts of the storage unit 120 may be composed of a combination of multiple semiconductor memory elements or hard disks, etc.

[0029] The image storage unit 121 stores images captured by the imaging device of the imaging system 10. For example, the image storage unit 121 stores the image in association with a predetermined angle at which the subject was photographed and the position of the imaging device that photographed the subject.

[0030] The 3D model storage unit 122 stores the 3D model generated by the information processing device 100. For example, the 3D model is a 3D model based on photogrammetry.

[0031] The control unit 130 is implemented, for example, by a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing a program (e.g., an information processing program) stored inside the information processing device 100 using RAM (Random Access Memory) as a working area. The control unit 130 is also a controller and may be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array). The control unit 130 also includes a reception unit 131, a device control unit 132, an acquisition unit 133, a detection unit 134, and a generation unit 135.

[0032] Furthermore, each of these parts of the control unit 130 may be composed of a single CPU or integrated circuit. Alternatively, each of these parts of the control unit 130 may be composed of a combination of multiple CPUs or integrated circuits. In the case of multiple CPUs, each of these parts of the control unit 130 does not necessarily have to have a one-to-one correspondence with each CPU; multiple CPUs may cooperate to realize the functions of multiple parts.

[0033] The reception unit 131 receives information regarding the first shooting conditions from the user. For example, the first shooting conditions include parameters of the lighting device, the shooting device, and the turntable of the shooting system 10.

[0034] For example, the reception unit 131 receives information regarding the illumination mode of the lighting device as a parameter of the lighting device. The reception unit 131 also receives information regarding the number of shots taken, zoom, position such as the height of the shooting device, and orientation of the shooting device as a parameter of the shooting device. In addition to the above examples, the reception unit 131 may also receive information regarding the aperture, shutter speed, ISO sensitivity, focal length, etc. of the shooting device.

[0035] Furthermore, the reception unit 131 receives information regarding the rotation angle per step (an example of a predetermined angle) and the interval as parameters for the turntable. The reception unit 131 also receives information from the user regarding the specified feature region, which is a feature region specified by the user. This allows the reception unit 131 to receive a specified feature region that is suitable for the user.

[0036] The device control unit 132 controls the various devices of the imaging system 10. Specifically, the device control unit 132 controls the various devices based on the first or second imaging conditions. For example, the device control unit 132 transmits control signals for controlling the various devices, which correspond to the parameters received by the reception unit 131.

[0037] For example, the device control unit 132 transmits a control signal to the lighting device to cause the lighting device to emit light. The device control unit 132 also transmits a control signal to the imaging device to cause the imaging device to photograph a subject based on the parameters of the imaging device. Furthermore, the device control unit 132 transmits a control signal to the turntable to rotate the turntable by a certain angle (for example, 5°, 10°, etc.). In this way, the device control unit 132 can suitably control the lighting device, imaging device, and turntable based on first or second imaging conditions.

[0038] The acquisition unit 133 acquires various types of information. Specifically, the acquisition unit 133 acquires a first image, including the subject, from the shooting system. The acquisition unit 133 then stores the acquired first image in the image storage unit 121.

[0039] Furthermore, the acquisition unit 133 acquires a second image from the imaging system 10 in which the feature region has been newly captured by the imaging device. The acquisition unit 133 then stores the acquired second image in the image storage unit 121.

[0040] The detection unit 134 detects a feature region in the first image stored in the image storage unit 121. For example, the detection unit 134 extracts edge portions by analyzing the first image. Further, the detection unit 134 generates a normal map and detects the direction of a subject or the like based on the normal map. Then, the detection unit 134 detects a feature region based on the edge portions and the normal map. Such detection processing can be realized by an image processing technique for detecting a feature region of a subject or the like.

[0041] Further, the detection unit 134 may detect a feature region based on a learning model learned in advance by deep learning or the like. Such a learning model is, for example, a learning model that has learned the relationship between the first image and the feature region included in the first image. For example, when the first image is input, the learning model outputs the feature region included in the first image.

[0042] For example, when the first image includes a feature region of a subject, the detection unit 134 detects the feature region in the first image. To give a more specific example, the detection unit 134 detects, in the first image, as the feature region, a fixation region, a region including a shape indicating a feature of the subject, a region including a hole-shaped shape, a region including character information, or a region including the shape of the surface of the subject.

[0043] The generation unit 135 generates a three-dimensional model of a subject based on the first image and the second image stored in the image storage unit 121. For example, the generation unit 135 generates a three-dimensional model using photogrammetry based on the first image and the second image. Then, the generation unit 135 stores the generated three-dimensional model in the three-dimensional model storage unit 122. In this way, the generation unit 135 can generate a high-quality three-dimensional model.

[0044] For example, the generation unit 135 generates a three-dimensional model using photogrammetry based on the first image and a second image including only the feature region. Then, the generation unit 135 stores the generated three-dimensional model in the three-dimensional model storage unit 122. In this way, the generation unit 135 can generate a high-quality three-dimensional model.

[0045] The input unit 140 receives the input of various types of information. For example, the input unit 140 receives the input of various types of information via an input device such as a UI (User Interface) operable by the user or a keyboard.

[0046] The display unit 150 displays the information output by the information processing apparatus 100. For example, the display unit 150 is a liquid crystal display or the like built in the information processing apparatus 100, or a liquid crystal display or the like connected to the information processing apparatus 100.

[0047] (1-3-1. Information Processing) Next, an example of the flow of information processing according to the embodiment will be described using FIG. 3. FIG. 3 is a schematic diagram showing the information processing according to the embodiment. In the example of FIG. 3, the case where the information processing apparatus 100 executes the information processing according to the embodiment will be described. For example, it is assumed that a subject is placed on the turntable included in the imaging system 10.

[0048] The information processing shown in FIG. 3 includes information processing PD11 such as imaging of an image, information processing PD12 such as detection of a fixation region, information processing PD13 such as generation of a three-dimensional model, information processing PD14 such as detection of a defect, and PD15 such as control of the imaging apparatus.

[0049] In the information processing PD11, the apparatus control unit 132 causes the imaging apparatus included in the imaging system 10 to image the subject placed on the turntable. For example, the reception unit 131 receives information regarding the first imaging condition as a user instruction. Then, the apparatus control unit 132 causes the imaging apparatus to image the subject based on the first imaging condition. This imaged image corresponds to the first image.

[0050] Subsequently, in the information processing PD12, the detection unit 134 detects the fixation region included in the subject in the captured first image (step S1). For example, the detection unit 134 generates a saliency map in the captured first image. The saliency map here indicates, for example, the degree to which the line of sight of the user who viewed the first image focused on which region of the first image. Then, the detection unit 134 detects the fixation region of the subject based on the generated saliency map.

[0051] To give a more specific example, let's assume the subject includes a face. Also, let's assume the gaze region is the subject's face. In this case, the detection unit 134 generates a saliency map with a high degree of the subject's face region. Then, based on the generated saliency map, the detection unit 134 detects the subject's face as the gaze region.

[0052] In this case, the device control unit 132 controls the imaging device (step S2). For example, the device control unit 132 changes the first imaging conditions to the second imaging conditions.

[0053] For example, the device control unit 132 calculates an overlap rate, which indicates the degree to which the first image including the gaze region and the first image including the region outside the gaze region overlap. Subsequently, the device control unit 132 changes the second shooting conditions based on the overlap rate. To give a more specific example, the device control unit 132 changes the number of shots, zoom rate, position or orientation of the shooting device, turntable rotation amount, etc., as the second shooting conditions.

[0054] For example, in the second shooting condition, the number of shots is set to be at least a predetermined number greater than in the first shooting condition, and the zoom ratio is set to be higher. In this case, the device control unit 132 causes the shooting device to take a new shot of the subject based on the second shooting condition (step S3). This captured image corresponds to the second image.

[0055] On the other hand, if the device control unit 132 does not detect the gaze area of ​​the subject, it causes the shooting device to photograph the subject based on the first shooting conditions (step S3).

[0056] In this manner, if the first image includes the subject's gaze area, the device control unit 132 causes the shooting device to photograph the subject based on the second shooting condition. On the other hand, if the first image does not include the subject's gaze area, the device control unit 132 causes the shooting device to photograph the subject based on the first shooting condition. The device control unit 132 then repeats this information processing a predetermined number of times.

[0057] Then, in information processing PD13, the generation unit 135 generates a three-dimensional model using photogrammetry based on the first and second images (step S4). Subsequently, in information processing PD14, the detection unit 134 detects defects in the generated three-dimensional model (step S5). For example, the detection unit 134 further detects the difference between the image generated from a predetermined shooting viewpoint and the first image corresponding to the predetermined shooting viewpoint in the generated three-dimensional model.

[0058] To give a more specific example, the detection unit 134 generates an image based on a single shooting viewpoint in the 3D model. The detection unit 134 then detects the difference between the image generated from the 3D model and a first image taken from the same shooting viewpoint. In this case, the detection unit 134 detects whether or not there is a difference between the image generated from the 3D model and the first image. For example, the detection unit 134 detects the difference if there is a difference between the image generated from the 3D model and the first image. On the other hand, the detection unit 134 does not detect a difference if there is no difference between the image generated from the 3D model and the first image. In this way, when the device control unit 132 detects a difference between the image generated from the 3D model and the first image, it controls the shooting device (step S6).

[0059] For example, if the device control unit 132 detects a difference between the image generated from the 3D model and the first image, it causes the imaging device to take a new image of the subject based on the same shooting viewpoint as the first image (step S3). This captured image corresponds to the second image.

[0060] Next, the generation unit 135 updates the 3D model based on the first image, the second image, and the newly captured second image (step S7). In this way, the generation unit 135 can generate a high-quality 3D model.

[0061] For example, the generation unit 135 can efficiently generate a high-quality 3D model by capturing only the necessary areas at high resolution. The device control unit 132 also changes the shooting conditions, such as the resolution of the shooting device, according to the feature areas of the subject. This allows the device control unit 132 to efficiently capture the subject with the shooting device. For example, the device control unit 132 can capture the feature areas at high resolution and the areas outside the feature areas at a lower resolution. The device control unit 132 can also change the field of view by controlling the PTZ of the shooting device. Furthermore, the device control unit 132 can control the field of view by controlling the turntable (rotation position (angle), rotation speed) or by controlling the position or orientation of the shooting device using a tilter TI1, etc.

[0062] (1-3-2. About the shooting process) Next, an example of the flow of the shooting process according to the embodiment will be explained using Figure 4. Figure 4 is a schematic diagram showing an example of the shooting process according to the embodiment. In the example of Figure 4, the shooting process performed in the shooting system 10 will be explained. In the example of Figure 4, the shooting device CA1, slider ST1, subject OB1, and turntable TT1 in the shooting system 10 are shown. In the example of Figure 4, the first image is taken based on the first shooting conditions. In the first shooting conditions, the shooting device CA1 takes the first image at three heights: upper, middle, and lower. The first shooting conditions also include the number of times to take an image at each height.

[0063] First, the device control unit 132 sets the imaging device CA1 to the upper position based on the first imaging conditions. Next, the device control unit 132 rotates the turntable TT1 at predetermined angles and causes the imaging device CA1 to photograph the subject OB1 at each predetermined angle. In this way, the device control unit 132 causes the imaging device CA1 to perform imaging a number of times based on the first imaging conditions.

[0064] The device control unit 132 then sets the imaging device CA1 to the middle position based on the first imaging conditions (step S21). Subsequently, the device control unit 132 rotates the turntable TT1 at predetermined angles and causes the imaging device CA1 to photograph the subject OB1 at each predetermined angle. In this way, the device control unit 132 causes the imaging device CA1 to perform imaging a number of times based on the first imaging conditions.

[0065] Then, the device control unit 132 sets the imaging device CA1 to the lower position based on the first imaging conditions (step S22). Subsequently, the device control unit 132 rotates the turntable TT1 at predetermined angles and causes the imaging device CA1 to photograph the subject OB1 at each predetermined angle. In this way, the device control unit 132 causes the imaging device CA1 to perform imaging a number of times based on the first imaging conditions.

[0066] (1-3-3. Regarding Application Example (1)) Next, an example of the flow of an application example according to the embodiment will be explained using Figure 5. Figure 5 is a schematic diagram showing application example (1) according to the embodiment. The first shooting conditions in Figure 5 are the same as the first shooting conditions in Figure 4, so the explanation will be omitted. Also, steps S31 to S32 shown in Figure 5 are the same as steps S21 to S22 shown in Figure 4, so the explanation will be omitted.

[0067] Next, the detection unit 134 detects the gaze region included in the subject in the first image captured based on the first shooting conditions (step S33). In the example in Figure 5, the detection unit 134 generates a saliency map SM30 that shows the gaze region in the first image.

[0068] The saliency map SM30 shown in Figure 5 is a conceptual diagram of a saliency map. For example, in the saliency map SM30, the degree to which a region of a face in an image is noticed by a user viewing the image is displayed as a heat map. In this case, the saliency map SM30 displays the region with a higher degree of attention in a darker color. The detection unit 134 then detects the gaze region of the subject in the first image based on the saliency map SM30.

[0069] Then, the device control unit 132 controls the shooting device to capture the area of ​​focus of the subject (step S34). For example, the device control unit 132 changes the first shooting conditions to the second shooting conditions. In the example in Figure 5, the device control unit 132 changes the number of shots and the zoom ratio as the second shooting conditions.

[0070] For example, in the second shooting condition, the number of shots is set to be at least a predetermined number more than in the first shooting condition, and the zoom ratio is larger. In this case, the device control unit 132 causes the shooting device to newly photograph the subject based on the second shooting condition. In the example in Figure 5, the device control unit 132 causes the shooting device CA1, which is set to the middle position, to photograph the second image IM31 which includes the gaze area.

[0071] (1-3-4. Regarding Application Example (2)) Next, an example of the flow of an application example according to the embodiment will be explained using Figure 6. Figure 6 is a schematic diagram showing application example (2) according to the embodiment. The first shooting conditions in Figure 6 are the same as the first shooting conditions in Figure 4, so the explanation will be omitted. Also, steps S41 to S42 shown in Figure 6 are the same as steps S21 to S22 shown in Figure 4, so the explanation will be omitted.

[0072] In the example shown in Figure 6, a three-dimensional model is generated based on the first image. In this case, the detection unit 134 detects defects in the generated three-dimensional model (step S43). For example, the detection unit 134 generates an image based on a single shooting viewpoint in the three-dimensional model. The detection unit 134 then detects the difference between the image generated from the three-dimensional model and the first image taken from the same shooting viewpoint.

[0073] For example, the detection unit 134 generates a heatmap MM40 that shows the difference between the image generated from the 3D model and the first image taken from the same shooting viewpoint. The heatmap MM40 shown in Figure 6 is a conceptual diagram of a heatmap that shows the difference between the image generated from the 3D model and the first image. For example, in the heatmap MM40, the greater the degree of difference, the darker the area corresponding to that difference is displayed. In the example in Figure 6, the left arm of the 3D model of subject OB1 is missing. In this case, the left arm of subject OB1 is displayed in a darker color in the heatmap MM40.

[0074] The detection unit 134 then detects whether there is a difference between the image generated from the 3D model and the first image, based on the heatmap MM40. For example, if there is a difference between the image generated from the 3D model and the first image, the detection unit 134 detects the difference.

[0075] Then, when the device control unit 132 detects a difference between the image generated from the 3D model and the first image, it controls the imaging device to capture a new image of the subject corresponding to the difference (step S44). For example, when the device control unit 132 detects a difference between the image generated from the 3D model and the first image, it causes the imaging device to capture a new image of the subject based on the same imaging viewpoint as the first image.

[0076] For example, the device control unit 132 changes the first shooting conditions to the second shooting conditions. In the example shown in Figure 6, the device control unit 132 changes the number of shots and the zoom ratio as the second shooting conditions.

[0077] For example, in the second shooting condition, the number of shots is set to be at least a predetermined number more than in the first shooting condition, and the zoom ratio is larger. In this case, the device control unit 132 causes the shooting device to newly photograph the subject based on the second shooting condition. In the example in Figure 6, the device control unit 132 causes the shooting device CA1, which is set to the middle position, to photograph a second image IM41 that includes the area of ​​the subject corresponding to the difference.

[0078] (1-4. Use Case (1) According to the Embodiment) Next, a use case (1) according to the embodiment will be described with reference to Figures 7 and 8. For example, one use case may be one in which there is a limit on the shooting time. In such a case, the shooting system 10 may be installed, for example, in a store that sells products such as toys.

[0079] (1-4-1. Shooting in Use Case (1)) Next, a conceptual diagram of shooting in Use Case (1) will be explained using Figure 7. Figure 7 is a conceptual diagram (1) of shooting according to the embodiment. The shooting device CA1, slider ST1, subject OB1, and turntable TT1 shown in Figure 7 are the same as the shooting device CA1, slider ST1, subject OB1, and turntable TT1 shown in Figure 4, so their explanation will be omitted.

[0080] In the example shown in Figure 7, the device control unit 132 calculates an overlap rate, which indicates the degree to which the area of ​​focus and the area outside the area of ​​focus overlap, based on a preset maximum number of shots and a preset focus area. Subsequently, the device control unit 132 changes the shooting conditions based on the overlap rate, such as changing the movement position of the slider ST1, the number of times the slider ST1 is moved, the rotation angle of the turntable TT1, and the zoom rate. Then, the device control unit 132 causes the shooting device CA1 to photograph the subject OB1 based on the changed shooting conditions.

[0081] (1-4-2. Flowchart showing the procedure for device control processing related to use case (1)) Next, the procedure for device control processing executed by the information processing device 100 according to the embodiment will be explained using Figure 8. Figure 8 is an example of a flowchart (1) showing the flow of device control processing according to the embodiment.

[0082] As shown in Figure 8, the device control unit 132 rotates the turntable (step S101). For example, the device control unit 132 starts controlling the rotation of the turntable based on information input by the user. For example, the device control unit 132 rotates the turntable to rotate the subject one full turn. This allows the device control unit 132 to show the user the shape of the subject, characteristic regions, etc.

[0083] Next, the reception unit 131 receives the user's designation of a gaze area as a designated feature area (step S102). For example, the reception unit 131 receives information about the gaze area from the user as a designated feature area.

[0084] Then, the detection unit 134 determines whether or not the image includes the specified gaze region (step S103). For example, if the detection unit 134 determines that the image does not include the specified gaze region (step S103; No), it determines whether or not the image includes the gaze region (step S104).

[0085] Next, if the detection unit 134 determines that the image does not include the gaze area (step S104; No), the device control unit 132 causes the shooting device to photograph the subject (step S106). For example, the device control unit 132 causes the shooting device to photograph the subject based on the first shooting conditions.

[0086] On the other hand, the device control unit 132 controls various devices (step S105) when the detection unit 134 determines that the image includes a designated gaze area (step S103; Yes), or when it determines that the image includes a gaze area (step S104; Yes). For example, the second shooting condition may be a change in the zoom level, the position or orientation of the shooting device, the rotation angle of the turntable, etc. In this case, the device control unit 132 controls the shooting device and the turntable based on the second shooting condition.

[0087] Next, the device control unit 132 causes the camera to photograph the subject (step S106). For example, the device control unit 132 causes the camera to photograph the subject based on the second shooting conditions.

[0088] Then, the device control unit 132 determines whether or not all-directional imaging has been completed (step S107). For example, if the device control unit 132 determines that all-directional imaging has not been completed (step S107; No), it resets the zoom (step S108). Subsequently, the device control unit 132 executes step S103 again.

[0089] On the other hand, if the device control unit 132 determines that all-directional imaging has been completed (step S107; Yes), it terminates the information processing.

[0090] (1-5. Use Cases (2) According to the Embodiment) Next, use cases (2) according to the embodiment will be described using Figures 9 and 10. For example, one use case may be one in which image quality is given greater importance. More specific examples include cases in which a video creator shoots their work in their own office, or cases in which they shoot collections held in art museums or museums.

[0091] (1-5-1. Shooting in Use Case (2)) Next, a conceptual diagram of shooting in Use Case (2) will be explained using Figure 9. Figure 9 is a conceptual diagram (2) of shooting according to the embodiment. The shooting device CA1, slider ST1, subject OB1, and turntable TT1 shown in Figure 9 are the same as the shooting device CA1, slider ST1, subject OB1, and turntable TT1 shown in Figure 4, so their explanation will be omitted.

[0092] In the example shown in Figure 9, the device control unit 132 causes the imaging device CA1 to photograph the subject OB1 based on the first shooting conditions. Subsequently, the detection unit 134 detects the gaze area in the captured first image. In this case, the device control unit 132 causes the imaging device CA1 to photograph the subject OB1 based on the second shooting conditions (step S51).

[0093] As another example, the generation unit 135 generates a three-dimensional model of the subject based on the captured first image. Subsequently, the detection unit 134 detects defects in the generated three-dimensional model.

[0094] For example, the detection unit 134 generates an image based on a single shooting viewpoint in the three-dimensional model. The detection unit 134 then detects the difference between the image generated from the three-dimensional model and a first image taken from the same shooting viewpoint. When the device control unit 132 detects a difference, it causes the shooting device CA1 to photograph the subject OB1 based on the second shooting conditions (step S51).

[0095] (1-5-2. Flowchart showing the procedure for device control processing related to use case (2)) Next, the procedure for device control processing executed by the information processing device 100 according to the embodiment will be explained using Figure 10. Figure 10 is an example of a flowchart (2) showing the flow of device control processing according to the embodiment.

[0096] As shown in Figure 10, the device control unit 132 rotates the turntable (step S201). Next, the device control unit 132 causes the camera to photograph the subject (step S202). For example, the device control unit 132 causes the camera to photograph the subject, thereby capturing a first image as an image of the subject.

[0097] Then, the device control unit 132 determines whether or not omnidirectional photography has been completed (step S203). For example, if the device control unit 132 determines that omnidirectional photography has not been completed (step S203; No), it repeats step S202.

[0098] Meanwhile, the device control unit 132 determines that omnidirectional imaging has been completed (step S203; Yes) and stops the rotation of the turntable (step S204). Subsequently, the generation unit 135 generates a three-dimensional model of the subject (step S205). For example, the generation unit 135 generates a three-dimensional model of the subject based on the first image captured by the imaging device.

[0099] Then, the detection unit 134 detects the gaze area (step S206). For example, the detection unit 134 detects the gaze area in an image generated from a predetermined shooting viewpoint based on a three-dimensional model of the subject.

[0100] Next, the reception unit 131 receives the user's designation of a gaze area as a designated feature area (step S207). For example, the reception unit 131 receives information about the gaze area from the user as a designated feature area.

[0101] Then, the device control unit 132 rotates the turntable (step S208). Subsequently, the detection unit 134 determines whether or not the image includes a specified gaze area (step S209). For example, if the detection unit 134 determines that the image does not include a specified gaze area (step S209; No), the device control unit 132 causes the shooting device to photograph the subject (step S211). For example, the device control unit 132 causes the shooting device to photograph the subject based on the first shooting conditions.

[0102] On the other hand, if the detection unit 134 determines that the image includes a specified gaze area, the device control unit 132 controls various devices (step S210). For example, the second shooting condition may be a change in the zoom level, the position or orientation of the shooting device, or the rotation angle of the turntable. In this case, the device control unit 132 controls the shooting device and the turntable based on the second shooting condition.

[0103] Next, the device control unit 132 causes the camera to photograph the subject (step S211). For example, the device control unit 132 causes the camera to photograph the subject based on the second shooting conditions.

[0104] Then, the device control unit 132 determines whether or not all-directional images have been captured (step S212). For example, if the device control unit 132 determines that all-directional images have not been captured (step S212; No), it resets the zoom (step S213). Subsequently, the device control unit 132 executes step S209 again.

[0105] On the other hand, if the device control unit 132 determines that images have been taken in all directions (step S212; Yes), it terminates the information processing.

[0106] (1-6. Modifications of the Embodiment) The information processing according to the embodiment described above may be modified in various ways. Modifications of the embodiment are described below.

[0107] (1-6-1. Regarding the detection process (1)) In the above embodiment, an example was given in which the detection unit 134 detects a gaze area (for example, a face) included in the subject in the captured first image, but it is not limited to this. For example, if the subject does not include a face, the detection unit 134 may detect in the first image an area that includes a shape that indicates the characteristics of the subject, an area that includes a hole-like shape, an area that includes textual information, or an area that includes the shape of the surface of the subject. This allows the detection unit 134 to efficiently detect characteristic shapes of the subject.

[0108] Furthermore, even if the subject does not include a face, the detection unit 134 may detect areas of interest to the user viewing the first image based on the saliency map. This allows the detection unit 134 to efficiently detect areas of interest to the user within the subject based on the saliency map.

[0109] (1-6-2. Regarding detection process (2)) The detection unit 134 may further detect the difference between the depth map generated from a predetermined shooting viewpoint in the three-dimensional model generated by the generation unit 135 and the depth map of the first image corresponding to the predetermined shooting viewpoint.

[0110] For example, suppose a three-dimensional model is generated based on a first image. In this case, the detection unit 134 generates an image based on one shooting viewpoint in the three-dimensional model and generates a depth map based on the generated image. The detection unit 134 may then detect the difference between the depth map generated from the three-dimensional model and the depth map generated based on the first image taken from the same shooting viewpoint.

[0111] As a result, the detection unit 134 can accurately detect the difference between the three-dimensional model and the actual subject included in the first image, specifically the difference in areas that include characteristic hole-like structures.

[0112] Furthermore, the reception unit 131 may receive information from the user regarding defects in the 3D model as a designated feature region. In this case, the detection unit 134 may detect defects specified by the user in the 3D model generated by the generation unit 135. This allows the detection unit 134 to detect defects that are suitable for the user.

[0113] Furthermore, the detection unit 134 may detect areas of the subject that are difficult to see from the field of view in the three-dimensional model generated by the generation unit 135. This allows the detection unit 134 to suitably detect areas that are difficult to reflect in the three-dimensional model.

[0114] (1-6-3. Regarding the generation process) The generation unit 135 may also generate a three-dimensional model of the subject based on the first image and update the three-dimensional model based on the three-dimensional model and the second image. This allows the generation unit 135 to update the three-dimensional model while appropriately acquiring the necessary second image, thereby improving the quality of the three-dimensional model. The area to be captured as the second image may be specified by the user.

[0115] (1-6-4. Regarding reception processing) The reception unit 131 may also receive information regarding the second shooting conditions from the user. This allows the device control unit 132 to have the shooting device photograph the subject based on shooting conditions that are suitable for the user.

[0116] (1-6-5. Regarding the second shooting conditions) Furthermore, the second shooting conditions are not limited to shooting conditions in which the number of shots is more than a predetermined number greater than the first shooting conditions and the zoom ratio is large, but any shooting conditions are acceptable as long as it is possible to capture an image with a resolution of a predetermined threshold or higher. For example, the second shooting conditions may be shooting conditions such as higher resolution, a field of view changed by PTZ control, or higher contrast compared to the first shooting conditions. As a result, the device control unit 132 can increase the planar density in the second image by having the shooting device capture the subject based on the second shooting conditions.

[0117] Furthermore, the second shooting conditions include shooting conditions that improve the accuracy of vertical angle resolution by controlling a slider or tilter, and shooting conditions that improve the accuracy of horizontal angle resolution by controlling a turntable. As a result, the device control unit 132 can increase the spatial density in the second image by having the shooting device photograph the subject based on the second shooting conditions. In addition, the second shooting conditions include shooting conditions that increase contrast and shooting conditions that increase gain compared to the first shooting conditions.

[0118] (2. Other Embodiments) The processes according to each of the embodiments described above may be carried out in various other forms besides those described above.

[0119] Furthermore, among the processes described in each of the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above documents and drawings can be changed at will unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown.

[0120] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0121] Furthermore, the embodiments and modifications described above can be combined as appropriate, provided that the processing content is not inconsistent.

[0122] Furthermore, the effects described herein are merely illustrative and not limiting; other effects may also occur.

[0123] (3. Summary of the configuration of the information processing device of the present disclosure) As described above, the information processing device of the present disclosure comprises an acquisition unit that acquires a first image including a subject, a detection unit that detects a feature region indicating the features of the subject in the first image, and a generation unit that generates a three-dimensional model of the subject based on the first image and a second image in which the feature region included in the subject has been newly captured.

[0124] This allows the information processing device to generate high-quality 3D models. For example, the information processing device can efficiently generate high-quality 3D models by capturing only the necessary areas at high resolution.

[0125] Furthermore, the detection unit detects the gaze region included in the subject as a feature region in the first image.

[0126] This allows the information processing device to detect the gaze area within the subject and identify the area that needs to be captured as a second image.

[0127] Furthermore, the detection unit detects a region in the first image that includes a shape representing the characteristics of the subject, as a feature region.

[0128] This allows the information processing device to detect regions containing shapes that represent the characteristics of the subject, thereby identifying regions containing characteristic structures within the subject that need to be captured as a second image.

[0129] Furthermore, the detection unit detects, as a feature region, a region containing a hole-like shape within the subject in the first image.

[0130] As a result, the information processing device can detect areas containing hole-like shapes within the subject, thereby identifying areas that need to be captured as a second image and that contain characteristic structures within the subject.

[0131] Furthermore, the detection unit detects, as a feature region, the region containing textual information within the subject in the first image.

[0132] This allows the information processing device to detect areas containing character information within the subject, thereby identifying areas that need to be captured as a second image.

[0133] Furthermore, the detection unit detects a region in the first image that includes the shape of the surface of the subject as a feature region.

[0134] This allows the information processing device to identify the area that needs to be captured as a second image by detecting the area that includes the shape of the surface of the subject.

[0135] Furthermore, the detection unit further detects the difference between the image generated from a predetermined shooting viewpoint and the first image corresponding to the predetermined shooting viewpoint in the three-dimensional model generated by the generation unit.

[0136] This allows the information processing device to detect with high accuracy the difference between the 3D model and the actual subject contained in the first image, specifically the difference in areas containing complex, fine shapes within the subject.

[0137] Furthermore, the detection unit further detects the difference between the depth map generated from a predetermined shooting viewpoint in the 3D model generated by the generation unit and the depth map of the first image corresponding to the predetermined shooting viewpoint.

[0138] As a result, the information processing device can accurately detect the difference between the three-dimensional model and the actual subject contained in the first image, specifically the difference in regions containing characteristic hole-like structures within the subject.

[0139] Furthermore, the generation unit generates a three-dimensional model based on the first image and a second image that includes only the feature region.

[0140] This allows the information processing device to accurately generate a three-dimensional model that reflects the shape of the feature region and other characteristics.

[0141] Furthermore, the generation unit generates a 3D model using photogrammetry.

[0142] This allows the information processing device to generate three-dimensional models with high accuracy.

[0143] Furthermore, the information processing device further includes a device control unit that controls a shooting system having a shooting device for photographing a subject, a lighting device for illuminating the subject with light, and a turntable on which the subject is placed.

[0144] This allows the information processing device to effectively control the lighting device, the imaging device, and the turntable.

[0145] Furthermore, the device control unit causes the camera to photograph the subject.

[0146] This allows the information processing device to control the imaging device appropriately, thereby enabling the imaging device to efficiently capture the subject.

[0147] Furthermore, the device control unit causes the imaging device to newly capture the characteristic region detected by the detection unit.

[0148] This allows the information processing device to cause the imaging device to capture the detected feature region based on the new imaging conditions.

[0149] Furthermore, the detection unit further detects the difference between the image generated from a predetermined shooting viewpoint in the 3D model generated by the generation unit and the first image corresponding to the predetermined shooting viewpoint, and the device control unit causes the shooting device to newly capture an image of the shooting area including the area of ​​the subject corresponding to the difference.

[0150] This allows the information processing device to cause the imaging device to capture an area of ​​the subject corresponding to the difference, based on imaging conditions including high resolution.

[0151] Furthermore, the device control unit is equipped with a reception unit that receives information from the user regarding a designated feature region, which is a feature region specified by the user, and the device control unit causes the imaging device to newly capture the designated feature region.

[0152] This allows the information processing device to cause the imaging device to capture a feature region desired by the user, based on new imaging conditions.

[0153] Furthermore, the device control unit causes the imaging device to newly capture a characteristic region based on a second imaging condition that is different from the first imaging condition under which the first image was captured.

[0154] This allows the information processing device to change the shooting conditions, such as the resolution of the imaging device, according to the characteristic regions of the subject. This enables the information processing device to efficiently capture the subject with the imaging device. For example, the information processing device can capture the characteristic regions at a high resolution and the areas outside the characteristic regions at a lower resolution.

[0155] Furthermore, the device control unit rotates the turntable at predetermined angles based on the second shooting conditions so that the subject rotates.

[0156] As a result, the information processing device can suitably control the turntable based on a second shooting condition in which the shooting conditions, such as the resolution of the shooting device, are changed according to the characteristic area.

[0157] Furthermore, the device control unit causes the lighting device to illuminate the subject based on the second shooting conditions.

[0158] As a result, the information processing device can suitably control the illumination device based on a second shooting condition in which the shooting conditions, such as the resolution of the shooting device, are changed according to the feature area.

[0159] (4. Hardware Configuration) The information processing device 100 and the imaging system 10, etc., according to each embodiment described above, are realized by a computer 1000 having a configuration such as that shown in Figure 11. The following explanation will use the information processing device 100 according to the embodiment as an example. Figure 11 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of the information processing device 100. The computer 1000 has a CPU 1100, RAM 1200, ROM (Read Only Memory) 1300, HDD (Hard Disk Drive) 1400, communication interface 1500, and input / output interface 1600. The various parts of the computer 1000 are connected by a bus 1050.

[0160] The CPU 1100 operates based on programs stored in the ROM 1300 or HDD 1400 and controls each part. For example, the CPU 1100 loads the programs stored in the ROM 1300 or HDD 1400 into the RAM 1200 and executes processing corresponding to various programs.

[0161] ROM 1300 stores boot programs such as the BIOS (Basic Input Output System) that are executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the computer 1000's hardware.

[0162] The HDD 1400 is a computer-readable recording medium that non-temporarily stores programs executed by the CPU 1100 and data used by such programs. Specifically, the HDD 1400 is a recording medium that stores an information processing program according to this disclosure, which is an example of program data 1450.

[0163] The communication interface 1500 is an interface for the computer 1000 to connect to an external network 1550 (e.g., the Internet). For example, the CPU 1100 can receive data from other devices or transmit data it has generated to other devices via the communication interface 1500.

[0164] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from input devices such as a keyboard or mouse via the input / output interface 1600. The CPU 1100 also transmits data to output devices such as a display, speaker, or printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs recorded on a predetermined recording medium (media). Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase Change Rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, or semiconductor memory.

[0165] For example, when the computer 1000 functions as an information processing device 100 according to the embodiment, the CPU 1100 of the computer 1000 realizes functions such as the control unit 130 by executing an information processing program loaded on the RAM 1200. The HDD 1400 stores the information processing program according to this disclosure and data in the storage unit 120. The CPU 1100 reads and executes the program data 1450 from the HDD 1400, but as another example, these programs may be obtained from other devices via an external network 1550.

[0166] Furthermore, this technology can also take the following configurations: (1) An information processing device comprising: an acquisition unit that acquires a first image including a subject; a detection unit that detects a feature region indicating the features of the subject in the first image; and a generation unit that generates a three-dimensional model of the subject based on the first image and a second image in which the feature region included in the subject is newly captured. (2) The information processing device according to (1), wherein the detection unit detects a gaze region included in the subject in the first image as the feature region. (3) The information processing device according to (1) or (2), wherein the detection unit detects a region including a shape indicating the features of the subject in the first image as the feature region. (4) The information processing device according to any one of (1) to (3), wherein the detection unit detects a region including a hole-like shape included in the subject in the first image as the feature region. (5) The information processing device according to any one of (1) to (4), wherein the detection unit detects a region including character information included in the subject in the first image as the feature region. (6) The information processing apparatus according to any one of (1) to (5), wherein the detection unit detects in the first image a region including the shape of the surface of the subject as the feature region. (7) The information processing apparatus according to any one of (1) to (6), wherein the detection unit further detects the difference between an image generated from a predetermined shooting viewpoint in a three-dimensional model generated by the generation unit and the first image corresponding to the predetermined shooting viewpoint. (8) The information processing apparatus according to any one of (1) to (6), wherein the detection unit further detects the difference between a depth map generated from a predetermined shooting viewpoint in a three-dimensional model generated by the generation unit and the depth map of the first image corresponding to the predetermined shooting viewpoint. (9) The information processing apparatus according to any one of (1) to (8), wherein the generation unit generates the three-dimensional model based on the first image and the second image including only the feature region. (10) The information processing apparatus according to any one of (1) to (9), wherein the generation unit generates the three-dimensional model using photogrammetry.(11) An information processing device according to any one of (1) to (10), further comprising a device control unit for controlling a shooting system having a shooting device for shooting the subject, a lighting device for illuminating the subject with light, and a turntable on which the subject is placed. (12) An information processing device according to (11), wherein the device control unit causes the shooting device to shoot the subject. (13) An information processing device according to (11) or (12), wherein the device control unit causes the shooting device to newly shoot a feature region detected by the detection unit. (14) An information processing device according to any one of (11) to (13), wherein the detection unit further detects the difference between an image generated from a predetermined shooting viewpoint in a three-dimensional model generated by the generation unit and the first image corresponding to the predetermined shooting viewpoint, and the device control unit causes the shooting device to newly shoot a shooting region including the region of the subject corresponding to the difference. (15) The information processing device according to any one of (11) to (14), further comprising a receiving unit that receives information from a user regarding a designated feature area which is a feature area specified by the user, wherein the device control unit causes the imaging device to newly photograph the designated feature area. (16) The information processing device according to any one of (13) to (15), wherein the device control unit causes the imaging device to newly photograph the feature area, imaging area, or designated feature area based on a second imaging condition which is different from the first imaging condition which is the condition under which the first image was taken. (17) The information processing device according to (16), wherein the device control unit rotates the turntable at predetermined angles based on the second imaging condition so that the subject rotates. (18) The information processing device according to (16) or (17), wherein the device control unit causes the illumination device to irradiate the subject with light based on the second imaging condition. (19) An information processing method comprising: a computer acquiring a first image including a subject; detecting a feature region in the first image that indicates the features of the subject; and generating a three-dimensional model of the subject based on the first image and a second image in which the feature region included in the subject is newly captured.(20) An information processing program that causes a computer to function as an information processing device comprising: an acquisition unit that acquires a first image including a subject; a detection unit that detects a feature region in the first image that shows the features of the subject; and a generation unit that generates a three-dimensional model of the subject based on the first image and a second image in which the feature region included in the subject is newly captured.

[0167] 1 Information Processing System 10 Imaging System 100 Information Processing Device 110 Communication Unit 120 Storage Unit 121 Image Storage Unit 122 3D Model Storage Unit 130 Control Unit 131 Reception Unit 132 Device Control Unit 133 Acquisition Unit 134 Detection Unit 135 Generation Unit 140 Input Unit 150 Display Unit

Claims

1. An information processing device comprising: an acquisition unit that acquires a first image including a subject; a detection unit that detects a feature region in the first image that indicates the features of the subject; and a generation unit that generates a three-dimensional model of the subject based on the first image and a second image in which the feature region included in the subject is newly captured.

2. The information processing apparatus according to claim 1, wherein the detection unit detects a gaze region included in the subject as the feature region in the first image.

3. The information processing apparatus according to claim 1, wherein the detection unit detects in the first image a region that includes a shape representing the characteristics of the subject as the feature region.

4. The information processing apparatus according to claim 1, wherein the detection unit detects in the first image a region that includes a hole-like shape within the subject as the feature region.

5. The information processing apparatus according to claim 1, wherein the detection unit detects in the first image a region containing character information included in the subject as the feature region.

6. The information processing apparatus according to claim 1, wherein the detection unit detects in the first image a region including the shape of the surface of the subject as the feature region.

7. The information processing apparatus according to claim 1, wherein the detection unit further detects the difference between an image generated from a predetermined shooting viewpoint in a three-dimensional model generated by the generation unit and the first image corresponding to the predetermined shooting viewpoint.

8. The information processing apparatus according to claim 1, wherein the detection unit further detects the difference between a depth map generated from a predetermined shooting viewpoint in a three-dimensional model generated by the generation unit and the depth map of the first image corresponding to the predetermined shooting viewpoint.

9. The information processing apparatus according to claim 1, wherein the generation unit generates the three-dimensional model based on the first image and the second image which includes only the feature region.

10. The information processing apparatus according to claim 1, wherein the generation unit generates the three-dimensional model using photogrammetry.

11. The information processing apparatus according to claim 1, further comprising a device control unit for controlling a shooting system having a shooting device for shooting the subject, a lighting device for illuminating the subject with light, and a turntable on which the subject is placed.

12. The information processing apparatus according to claim 11, wherein the apparatus control unit causes the imaging device to photograph the subject.

13. The information processing apparatus according to claim 11, wherein the apparatus control unit causes the imaging device to newly photograph the feature region detected by the detection unit.

14. The information processing apparatus according to claim 11, wherein the detection unit further detects the difference between an image generated from a predetermined shooting viewpoint in a three-dimensional model generated by the generation unit and the first image corresponding to the predetermined shooting viewpoint, and the device control unit causes the shooting device to newly capture an shooting area including the area of ​​the subject corresponding to the difference.

15. The information processing apparatus according to claim 11, further comprising a receiving unit that receives information from a user regarding a designated feature region which is a feature region specified by the user, wherein the device control unit causes the imaging device to newly image the designated feature region.

16. The information processing apparatus according to claim 13, wherein the apparatus control unit causes the imaging apparatus to newly image the feature region based on a second imaging condition different from the first imaging condition, which is the condition under which the first image was captured.

17. The information processing apparatus according to claim 16, wherein the apparatus control unit rotates the turntable at predetermined angles based on the second shooting conditions so that the subject rotates.

18. The information processing apparatus according to claim 16, wherein the apparatus control unit causes the illumination device to irradiate the subject with light based on the second shooting conditions.

19. An information processing method comprising: a computer acquiring a first image including a subject; detecting a feature region in the first image that indicates the features of the subject; and generating a three-dimensional model of the subject based on the first image and a second image in which the feature region included in the subject is newly captured.

20. An information processing program that causes a computer to function as an information processing device comprising: an acquisition unit that acquires a first image including a subject; a detection unit that detects a feature region in the first image that indicates the features of the subject; and a generation unit that generates a three-dimensional model of the subject based on the first image and a second image in which the feature region included in the subject is newly captured.

Images