Three-dimensional model generation system, photographing condition setting device, and photographing system

Abstract

One aspect of the present invention is to provide a three-dimensional model generation system, a photographing condition setting device, and a photographing system. A three-dimensional model generation system according to one aspect of the present invention is provided with: an illumination device having a mechanism for setting a first polarization direction, which is the polarization direction of illumination light; a photographing device having a mechanism for setting a second polarization direction, which is the polarization direction during photography; a determination unit that determines, on the basis of at least one of characteristics of an object and environmental conditions at the time of photographing the object, a first photographing condition that is a photographing condition for photographing the object; an image acquisition unit that performs first photography for photographing the object using the photographing device and the illumination device on the basis of the first photographing condition to acquire a plurality of first images; and a generation unit that generates a three-dimensional model of the object using the plurality of first images. The first photographing condition includes one of the condition of illumination by the illumination device, the first polarization direction, and the second polarization direction.

Description

Three-dimensional model generation system, photography condition setting device, and photography system

[0001] The present invention relates to a three-dimensional model generation system, a photography condition setting device, and a photography system, and more particularly to photography for generating a three-dimensional model.

[0002] Regarding the generation of three-dimensional models, for example, Patent Document 1 describes a technique for converting a real-world subject into subject information (three-dimensional model) that can be handled using CG technology or the like.

[0003] Japanese Patent Application Laid-Open No. 2022-60641

[0004] One embodiment of the technique of the present disclosure provides a three-dimensional model generation system, a photography condition setting device, and a photography system.

[0005] A three-dimensional model generation system according to a first aspect of the present invention includes an illumination device having a mechanism for setting a first polarization direction, which is the polarization direction of illumination light; an imaging device having a mechanism for setting a second polarization direction, which is the polarization direction during imaging; a determination unit that determines first imaging conditions, which are imaging conditions when imaging an object, based on at least one of the characteristics of the object and the environmental conditions when imaging the object; an image acquisition unit that performs a first imaging to image the object using the imaging device and the illumination device based on the first imaging conditions, and acquires a plurality of first images; and a generation unit that generates a three-dimensional model of the object using the plurality of first images, wherein the first imaging conditions include any of the conditions of illumination by the illumination device, the first polarization direction, and the second polarization direction.

[0006] In the three-dimensional model generation system according to the second aspect of the present invention, in the first aspect, the determination unit determines the first shooting conditions based on the features of a second image including the object acquired by a second shooting prior to the first shooting.

[0007] In the three-dimensional model generation system of the third aspect, in the first or second aspect, the determination unit extracts a first region and a second region from a second image including the object obtained by a second photographing operation prior to the first photographing operation, and determines a first photographing condition based on the characteristics of the first region and the characteristics of the second region.

[0008] A three-dimensional model generation system according to a fourth aspect is the third aspect, in which the characteristics of the first region include the size of the first region in the two images and / or the proportion of the first region that occupies the second image, and the characteristics of the second region include the size of the second region in the second image and / or the proportion of the second region that occupies the second image.

[0009] In the three-dimensional model generation system of the fifth aspect, in the third or fourth aspect, the determination unit extracts features of the low-contrast area in the second image as features of the first area, extracts features of the reflective area in the second image as features of the second area, and determines the first shooting conditions based on the features of the low-contrast area and the features of the reflective area.

[0010] The three-dimensional model generation system according to the sixth aspect is the fifth aspect, in which the determination unit determines, as the first shooting condition, at least one of a third shooting condition which is a shooting condition for a low-contrast area and a fourth shooting condition which is a shooting condition for a reflective area.

[0011] A three-dimensional model generation system according to a seventh aspect is the sixth aspect, wherein the third and fourth shooting conditions include the number of times images are acquired in the first shooting.

[0012] In the three-dimensional model generation system according to the eighth aspect, in the sixth or seventh aspect, the determiner determines as third shooting conditions: fixed lighting direction shooting in which multiple illumination lights with different first polarization directions are irradiated onto the object from one lighting direction and the object is photographed under multiple lighting conditions with different first polarization directions; and multiple illumination light shooting in which illumination lights are irradiated onto the object from multiple directions simultaneously and the object is photographed with the first polarization direction fixed.

[0013] The 3D model generation system according to the ninth aspect is the eighth aspect, in which the determination unit determines as the third shooting condition that, in shooting with a fixed lighting direction, shooting in one polarization direction is repeatedly performed while changing the polarization direction.

[0014] In the three-dimensional model generation system of the tenth aspect, in the ninth aspect, the determination unit determines as a third shooting condition that the object is photographed with a fixed lighting direction by irradiating the object with multiple illumination lights having different first polarization directions from another illumination direction different from the one lighting direction.

[0015] The 3D model generation system according to the eleventh aspect is any one of the sixth to tenth aspects, in which the determination unit determines as the fourth shooting condition that illumination light is simultaneously irradiated from a plurality of lighting directions and that shooting is performed in a plurality of second polarization directions.

[0016] In the 3D model generation system of the 12th aspect, in any one of the first to 11th aspects, the determination unit determines as the first shooting condition that in the first shooting, the object is photographed from multiple shooting directions to obtain multiple first images.

[0017] The 3D model generation system according to the thirteenth aspect is the twelfth aspect, wherein the image acquisition unit photographs the object from multiple shooting directions by moving and / or rotating the photographing device and the object relatively.

[0018] A three-dimensional model generation system according to a fourteenth aspect is any one of the first to thirteenth aspects, in which the determination unit determines, as lighting conditions in the first shooting, that the object is to be photographed under a plurality of lighting conditions with different lighting directions.

[0019] A three-dimensional model generation system according to a fifteenth aspect is the fourteenth aspect, wherein the image acquisition unit changes the illumination direction by relatively moving and / or rotating the illumination device and the object.

[0020] A three-dimensional model generation system according to a sixteenth aspect is any one of the first to fifteenth aspects, in which the lighting device includes a plurality of lighting devices, and the determination unit determines, as a lighting condition, which of the plurality of lighting devices to emit illumination light from in the first photograph.

[0021] The three-dimensional model generation system of the seventeenth aspect is any one of the first to sixteenth aspects, in which the image acquisition unit associates a plurality of first images with first shooting conditions and records them in a recording device, and the generation unit generates a three-dimensional model by referring to the association.

[0022] The 3D model generation system of the 18th aspect is any one of the 6th to 11th aspects, wherein the generation unit generates a first 3D model of the object using a plurality of first images acquired under a third shooting condition, generates a second 3D model of the object using a plurality of first images acquired under a fourth shooting condition, and generates the 3D model of the object based on the first 3D model and the second 3D model.

[0023] The 3D model generation system of the 19th aspect is the 18th aspect, in which the generation unit generates a 3D model of the object using a first 3D model for a first portion of the object corresponding to a low-contrast area, and a second 3D model for a second portion of the object corresponding to a reflective area.

[0024] A photographing condition setting device according to a twentieth aspect is a photographing condition setting device including a processor, in which the processor sets a first polarization direction, which is the polarization direction of illumination light from a lighting device, sets a second polarization direction, which is the polarization direction when photographing by a photographing device, determines first photographing conditions, which are photographing conditions when photographing an object, based on at least one of the characteristics of the object and the environmental conditions when photographing the object, and performs a first photographing of the object using the photographing device and the lighting device based on the first photographing conditions to obtain a plurality of first images, and the first photographing conditions include any of the lighting conditions from the lighting device, the first polarization direction, and the second polarization direction.

[0025] In the 20th aspect of the shooting condition setting device according to the 21st aspect, the processor acquires characteristics of low-contrast areas in the object and characteristics of reflective areas in the object, determines third shooting conditions corresponding to the characteristics of the low-contrast areas and fourth shooting conditions corresponding to the characteristics of the reflective areas, and causes the shooting device to photograph the object based on the third shooting conditions and / or the fourth shooting conditions to acquire multiple first images.

[0026] The photography system of the 22nd aspect is a photography system comprising an illumination device having a mechanism for setting a first polarization direction, which is the polarization direction of illumination light; an imaging device having a mechanism for setting a second polarization direction, which is the polarization direction during photography; and an imaging control device, wherein the imaging control device determines first photography conditions, which are photography conditions when photographing the object, based on at least one of the characteristics of the object and the environmental conditions when photographing the object, and performs a first photography to photograph the object using the imaging device and the illumination device based on the first photography conditions, thereby obtaining a plurality of first images, and the first photography conditions include any of the lighting conditions of the illumination device, the first polarization direction, and the second polarization direction.

[0027] FIG. 1 is a diagram illustrating the configuration of a 3D model generation system according to a first embodiment. FIG. 2 is a diagram illustrating the configuration of a controller. FIG. 3 is a diagram illustrating the functional configuration of a processor. FIG. 4 is a diagram illustrating an example of information recorded in a recording device. FIG. 5 is a diagram illustrating the configuration of a shooting camera. FIG. 6 is a diagram illustrating a state in which the polarization direction is changed by rotating a polarizing filter. FIG. 7 is a diagram illustrating the configuration of a lighting device. FIG. 8 is a flowchart illustrating an overview of processing in the 3D model generation system. FIG. 9 is a flowchart illustrating feature extraction and setting of shooting conditions. FIG. 10 is a flowchart illustrating processing for actual shooting that takes into account shooting control for low-contrast areas and reflective areas. FIG. 11 is a diagram illustrating a state in which shooting is performed while changing the lighting direction using shooting control for low-contrast areas. FIG. 12 is a diagram illustrating a state in which shooting is performed with all lighting devices turned on using shooting control for low-contrast areas. FIG. 13 is a diagram illustrating another example of a configuration for shooting from multiple shooting directions.

[0028] [Generation of 3D Models Using Polarization Control] Typical methods for generating 3D models (3D: three-dimensional, hereinafter the same) include the ToF (Time of Flight) method, structured light method, and photogrammetry method. Of these three methods, photogrammetry generates 3D models based on photographed images, and is known to be able to generate high-quality 3D models if the resolution and image quality of the photograph are high. However, if the subject contains low-contrast areas, reflective areas, transparent or translucent parts, etc., the shape and texture quality of those parts will be significantly reduced when the 3D model is created.

[0029] In light of these circumstances, the inventors of the present application have conducted extensive research and have discovered that "by controlling the polarization state on the camera side and / or lighting side according to the characteristics of the subject, it is possible to generate a high-quality 3D model even for a subject that is difficult to photograph using photogrammetry." The present invention was created based on this discovery, and specific aspects of the 3D model generation system, photography condition setting device, and photography system according to the present invention will be described below.

[0030] First Embodiment Overall Configuration of a 3D Model Generation System FIG. 1 is a diagram illustrating the configuration of a 3D model generation system 10 (photography condition setting device, 3D model generation system) according to a first embodiment. As illustrated in FIG. 1, the 3D model generation system 10 includes a controller 100 (processor, photography condition setting device, photography control device, generation unit), photographic cameras 200A-200F (photography devices), and lighting devices 300A-300F (lighting device, multiple lighting devices). In the following description, the photographic cameras 200A-200F may be collectively referred to as the "photography camera 200," and the lighting devices 300A-300F may be collectively referred to as the "lighting device 300." The photographic camera 200 may include some or all of the components of the controller 100. An object 99 (a bowling ball in this example) is an example of an object for which a 3D model is generated, and a reflection area 99A, described below, appears in the image.

[0031] 1 illustrates an embodiment in which a plurality of photographic cameras 200 and a plurality of lighting devices 300 are provided, the number of photographic cameras 200 and lighting devices 300 may be different from that illustrated in FIG. 1. Also, one or a few photographic cameras and lighting devices may be provided, and photographs may be taken while moving these. Also, while the photographic cameras 200 and lighting devices 300 are arranged in a specific plane in FIG. 1, the photographic cameras 200 and lighting devices 300 may be arranged in multiple planes (for example, in a horizontal plane and a vertical plane).

[0032] In addition to the above configuration, a display device for displaying a background may be provided, and photographing (preliminary photographing and main photographing) may be performed with the background image displayed on the display device under the control of controller 100. In this case, it is preferable that processor 110 (controller 100) generates or selects a background image according to the characteristics of the object.

[0033] [Configuration of Controller] Fig. 2 is a diagram showing the configuration of the controller 100 (processor, photography condition setting device, photography control device, generation unit) in the first embodiment. As shown in Fig. 2, the controller 100 includes a processor 110 (processor, photography condition setting device, image acquisition unit, determination unit, photography control device, generation unit), a ROM 130 (ROM: Read Only Memory, a non-transitory and tangible recording medium), a RAM 140 (RAM: Random Access Memory), an operation unit 150, a speaker 160 (output device), a display 170 (output device), an input / output interface 180, and a recording device 190, and these components are connected by a bus 195. The controller 100 can communicate with the photographic camera 200, the lighting device 300, and various external devices via the input / output interface 180.

[0034] [Processor Configuration] Fig. 3 is a diagram showing the functional configuration of the processor 110. As shown in Fig. 3, the processor 110 includes a lighting device control unit 112, a photographing camera control unit 114, a photographing condition determination unit 116, a feature extraction unit 118, a three-dimensional model generation unit 120, and an input / output control unit 122.

[0035] The processor 110 is configured with various processors and electrical circuits, such as a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), a programmable logic device (PLD), etc. When these processors and electrical circuits execute software (programs), the software to be executed is stored in a non-transitory, tangible recording medium such as a code ROM 130 that can be read by a computer (for example, various processors and electrical circuits that constitute the processor, and / or a combination thereof), and the computer references the software.

[0036] The software stored in the non-transitory, tangible recording medium includes the program of the present invention (a program that causes a computer to execute the method for operating the imaging condition setting device, the method for operating the imaging control device, and the 3D model generation method of the present invention) and data used in executing the program. Instead of the ROM 130, the code may be recorded in a non-transitory, tangible recording medium such as a flash ROM or an EEPROM (Electronically Erasable and Programmable Read Only Memory). Note that this "non-transitory, tangible recording medium" does not include non-tangible recording media such as carrier signals or propagation signals themselves. During processing using the software, the RAM 140 is used as a temporary storage area or working area.

[0037] The functions of the processor 110 described above may be realized by various types of AI (Artificial Intelligence). Such AI may be, for example, AI that performs image recognition, segmentation, feature extraction, etc., or AI that generates images from given information.

[0038] [Configuration of Operation Unit and Display] The operation unit 150 is composed of devices such as a keyboard, mouse, buttons, and switches (not shown). The user can issue instructions to the controller 100 via these devices, and the processor 110 accepts the instructions and performs processing according to the accepted instructions. The display 170 may be composed of a touch panel device so that the user can issue instructions via the touch panel. The display 170 is composed of such a touch panel device or a device such as a liquid crystal display device, and can display feature information of the object, preliminary captured images, actual captured images, generated 3D models, etc. The display 170 can also display information recorded in the recording device 190.

[0039] [Configuration of Input / Output Interface] The input / output interface 180 is composed of terminals and slots for connecting external devices such as a display, printer, and recording medium, and communication interfaces such as Wi-Fi (registered trademark) and Bluetooth (registered trademark). The controller 100 can acquire image data, video data, and the like from external devices (server devices, recording devices, databases, imaging devices, and the like) via the input / output interface 180. The external devices may be connected to the controller 100 by wire or wirelessly. Furthermore, the external devices may be connected via a network such as the Internet.

[0040] [Configuration of the recording device] The recording device 190 (recording device) is composed of recording media (non-transitory and tangible recording media) such as semiconductor memories such as hard disks and SSDs (Solid-State Drives), and various types of magneto-optical recording media, and their control units, and records or saves various types of information.

[0041] FIG. 4 is a diagram showing an example of information recorded in the recording device 190. The preliminary image 190A is an image obtained by preliminary shooting (second shooting) performed prior to shooting (first shooting) of an image (actually shot image 190B) used to generate a 3D model of the object. The actual shot image 190B is an image used to generate a 3D model of the object. The feature information 190C is information indicating the features of the object for which a 3D model is to be generated. The shooting conditions 190D are shooting conditions for the preliminary shooting and the actual shooting, and may include conditions such as the viewpoint position, shooting direction, and exposure. The shooting conditions 190D may also include lighting conditions by the lighting device 300 (e.g., whether the lighting light is on or off, i.e., which lighting device emits the lighting light, the color and brightness of the lighting light, etc.), conditions of the first polarization direction (polarization angle, etc.), which is the polarization direction of the lighting light, and conditions of the second polarization direction (polarization angle, etc.), which is the polarization direction during shooting. The three-dimensional model 190E is a three-dimensional model of the object generated using a plurality of images acquired by the actual photography.

[0042] It is preferable that the above-mentioned information be recorded in association with each other. The processor 110 can display the information recorded in the recording device 190 on the display 170 in response to a user instruction via the operation unit 150, or automatically without a user instruction.

[0043] [Configuration of Photographing Camera] FIG. 5 is a diagram showing the configuration of the photographing camera 200. The photographing cameras 200A to 200F can have a similar configuration, and the configuration of the photographing cameras 200A to 200F will be described using the reference symbols in FIG. 5 as representatives. As shown in FIG. 5, the photographing camera 200 includes a lens 202, a polarization direction setting mechanism 204, an image sensor 206, an AFE (Analog Front End) 208, an A / D converter (Analog-to-Digital Converter) 210, and a lens driver 212. The photographing camera 200 is controlled by the controller 100 (processor 110: photographing camera control unit 114, photographing condition determination unit 116, etc.) to photograph an object (main photographing (first photographing), preliminary photographing (second photographing)) under determined photographing conditions (zoom, focus, shutter speed, exposure, etc.) to obtain one or more images (main photographing image (first image), preliminary photographing image (second image)). As will be described later, the photographing camera 200 can be configured so that the viewpoint position and / or photographing direction can be changed.

[0044] [Lens] The lens 202 may be a single lens or a lens group consisting of multiple lenses. This lens group may include a zoom lens and / or a focus lens, and may also include an aperture. The lens driving unit 212 can control zooming and focusing by driving the zoom lens and / or the focus lens forward and backward under the control of the controller 100. In addition to being controlled by the controller 100, zooming and focusing may also be controlled in response to a user's zooming or focusing operation (such as rotation of a zoom ring or focus ring, not shown). The lens driving unit 212 may also control the aperture. Note that the lens 202 or a lens barrel including the lens 202 may be detachable from the body of the photographic camera 200.

[0045] [Polarization Direction Setting Mechanism] The polarization direction setting mechanism 204 is a mechanism that sets the second polarization direction, which is the polarization direction during image capture using the imaging camera 200, under the control of the controller 100. The polarization direction setting mechanism 204 can set an arbitrary second polarization direction, for example, by rotating a polarizing filter, polarizing film, polarizing plate, etc. (not shown) (hereinafter referred to as a "polarizing filter, etc.") inserted in the optical path. The polarization direction setting mechanism 204 may not rotate the polarizing filter, etc. itself, but may instead arrange multiple polarizing filters, etc. with different polarization directions (polarization angles), on a sliding or turret-type holding member and translate and / or rotate the holding member to insert a polarizing filter, etc. with a desired polarization direction, into the optical path. The polarizing filter, etc. may be located in front of the lens 202 (on the subject side, object side), in the middle of the lens group that constitutes the lens 202, or behind the lens 202 (on the image sensor 206 side).

[0046] Furthermore, the polarization direction setting mechanism 204 may set the polarization direction using a polarizing element based on a wire grid type, a photonic crystal, or other principles, instead of the above-mentioned polarizing film, etc. The polarization direction setting mechanism 204 is preferably capable of setting multiple polarization directions, more preferably capable of setting four or more polarization directions, and even more preferably capable of continuously changing the polarization direction.

[0047] 6 shows how the polarization direction is changed by rotating a polarizing filter. The polarization direction of the photographing camera 200 (the second polarization direction, which is the polarization direction during photography) and the first polarization direction, which is the polarization direction of the illumination light, can be changed in this manner. In the example shown in the figure, polarizing filters 400A, 400B, 400C, and 400D correspond to polarization angles of 0°, 45°, 90°, and 135°, respectively.

[0048] [Image Sensor] Returning to FIG. 5 , the image sensor 206 has a light-receiving surface on which a large number of light-receiving elements are arranged in a two-dimensional matrix. Color pixels are provided on the light-receiving surface of the image sensor 206, allowing a color image of the subject to be captured. Then, subject light transmitted through the lens 202 is focused on the light-receiving surface of the image sensor 206 and converted into an electrical signal by each light-receiving element. A shutter may be provided in front of the image sensor 206 (on the subject side). Note that various photoelectric conversion elements such as a CMOS (Complementary Metal-Oxide Semiconductor) or a CCD (Charge-Coupled Device) can be used as the image sensor 206.

[0049] [AFE, A / D Converter, and Signal Processing Unit] The AFE 208 performs noise reduction, amplification, etc. on the analog image signal output from the image sensor 206, and the A / D converter 210 converts the captured analog image signal into a digital image signal with a wide gradation range. The signal processing unit 214 performs desired signal processing and image processing on this digital image signal, and outputs the result to the controller 100 as an image of the subject.

[0050] [Configuration of Illumination Device] FIG. 7 is a diagram showing the configuration of illumination device 300. Similar configurations can be employed for illumination devices 300A to 300F, and the configurations of illumination devices 300A to 300F will be described using the reference symbols in FIG. 7 as representatives. As shown in FIG. 7, illumination device 300 includes lens 302, polarization direction setting mechanism 304, and light source device 306. Lens 302 may be a single lens or a lens group consisting of multiple lenses. Controller 100 may drive some or all of lenses 302 forward and backward to change the spread of light emitted from illumination device 300. Polarization direction setting mechanism 304 is a mechanism that sets the first polarization direction, which is the polarization direction of illumination light, and may employ a configuration similar to that of polarization direction setting mechanism 204 described above (e.g., rotation or movement of a polarizing filter). Also, as described above with respect to the polarization direction setting mechanism 204, the polarizing filter etc. may be placed in front of the lens 302 (on the subject side, object side), may be placed in the middle of the lens group that makes up the lens 302, or may be placed behind the lens 302 (on the light source device 306 side).

[0051] The configuration of the light source device 306 is not particularly limited, but for example, an illumination device configured with LEDs (Light-Emitting Diodes) of multiple colors (red, blue, green, etc.) can be used. The light source device 306 is preferably a device that can change the brightness and color (color temperature, etc.) of the illumination light in addition to turning it on and off.

[0052] Similarly to the above-described case of the photographing camera 200, the lighting device 300 can be configured so that its position and / or lighting direction can be changed.

[0053] [Processing in the 3D Model Generation System] Fig. 8 is a flowchart showing an overview of the processing (acquisition of feature information by preliminary photographing, main photographing, generation of a 3D model, etc.) in the 3D model generation system 10 configured as described above. Fig. 9 is a flowchart related to feature extraction and setting of photographing conditions (details of the processing in steps S120 and S130 in Fig. 8). Note that the following describes a case in which preliminary photographing and main photographing are performed for one viewpoint position and photographing direction, and this is repeated while changing the viewpoint position and photographing direction to obtain multiple images (images for creating a 3D model) (i.e., a case in which an object is photographed from multiple photographing directions to obtain multiple main photographed images (first images)). However, it is also possible to perform preliminary photographing for all viewpoint positions and photographing directions, and then perform main photographing for all viewpoint positions and photographing directions after the preliminary photographing is completed.

[0054] The processor 110 (including the photographing condition determination unit 116, the lighting device control unit 112, the photographing camera control unit 114, and the feature extraction unit 118) sets a viewpoint position and a photographing direction (step S105), and performs the following processing (steps S105 to S140) for the set viewpoint position and photographing direction. The processor 110 may change the viewpoint position and the photographing direction by switching the photographing camera to be used among the photographing cameras 200A to 200F, or may change the viewpoint position and the photographing direction by moving and / or rotating the photographing camera 200 and the object relatively.

[0055] [Feature Extraction of Object by Preliminary Photographing] [Preliminary Photographing] The processor 110 (the photographing condition determination unit 116, etc.) determines the photographing conditions (second photographing conditions) for the preliminary photographing (step S105). The preliminary photographing is a photographing (second photographing) performed prior to the main photographing (first photographing) and is a photographing to obtain an image (second image, preliminary photographed image) to be used for extracting the features of the object.

[0056] The shooting conditions for the preliminary shooting may include any one of the lighting conditions of the lighting device 300, the first polarization direction, and the second polarization direction. Note that the "lighting conditions" here may include on / off, i.e., which lighting device 300 emits the illumination light (one or more of the lighting devices 300 may be turned on), the irradiation position and / or direction, and the color and brightness of the illumination light, but the first polarization direction is not included in the "lighting conditions." The shooting conditions for the preliminary shooting may include at least the first polarization direction, and may further include other conditions (e.g., the viewpoint position and / or shooting direction of the shooting camera 200, zoom, focus, exposure, etc.).

[0057] There may be multiple (multiple sets of) shooting conditions for the preliminary shooting. This is because the features extracted may differ depending on the shooting conditions, such as the viewpoint position, shooting direction, etc. A device for displaying a background image may be provided in the 3D model generation system 10, and displaying the background image on that device may be included in the shooting conditions.

[0058] The processor 110 (illumination device control unit 112, shooting camera control unit 114, shooting condition determination unit 116, etc.) photographs the object 99 under the shooting conditions determined in step S100, and acquires images including the object (image for feature extraction, preliminary captured image, second image) (step S110). Note that the images acquired for one viewpoint position and shooting direction may be one or more still images, or may be moving images.

[0059] [Acquisition of Object Features] The processor 110 (feature extraction unit 118, etc.) extracts features of the object 99 (object) from the image acquired in the preliminary image capture (preliminary image, second image including the object) (step S120). The processor 110 can extract a first region and a second region from the preliminary image and extract features of those regions. The first region may be, for example, a low-contrast region, and the second region may be, for example, a reflective region, but is not limited thereto. For example, the first region and the second region may be transparent and translucent. Depending on the 3D model generation method, the model generation accuracy may decrease for regions with specific features (hereinafter referred to as "specific regions"). Therefore, it is preferable that the processor 110 extracts such specific regions as the first region and the second region according to the 3D model generation method (such as photogrammetry and the visual volume intersection method described below). Alternatively, only one type of specific region may be extracted.

[0060] Low-contrast regions are examples of the specific regions described above, and can be defined as, for example, (1) regions where there is little change in the color or shape of the object and the color gradation or shape changes smoothly, (2) regions where it is difficult to extract feature points from an image when generating a two-dimensional model, or (3) regions where there is little change between two images taken at different shooting angles (shooting directions) and therefore it is difficult to extract corresponding feature points between those images. Reflective regions are also examples of the specific regions described above, and can be defined as, for example, "regions with a large specular reflection component within the object area in an image." Regions in an image where so-called "blown-out highlights" (a phenomenon in which the image signal in bright or highly reflective parts of an object becomes saturated and gradation expression becomes impossible) occur are an example of a reflective region.

[0061] The characteristics of the low-contrast region may be, for example, the size of the low-contrast region (first region) in the second image and / or the proportion of the low-contrast region in the second image, and may include information indicating the contrast value of the "low-contrast region." The characteristics of the reflective region may be, for example, the size of the reflective region (second region) in the second image and / or the proportion of the reflective region in the second image, and may include information indicating the degree of reflectivity of the "reflective region." Below, we will explain the case where the first region is a low-contrast region and the second region is a reflective region. In this case, the processor 110 (feature extraction unit 118, etc.) extracts the characteristics of the low-contrast region (first region) from the feature extraction image (preliminary image, second image) (step S122 in FIG. 9 ) and extracts the characteristics of the reflective region (second region) (step S124 in FIG. 9 ).

[0062] The region extraction and feature extraction methods are not particularly limited. The region extraction and feature extraction may be performed by recognizing the edges and corners of the object using normal image processing, and then extracting features from the object's color, brightness, or changes or distributions thereof. Alternatively, the region extraction and feature extraction may be performed by performing segmentation using image processing AI, such as AI using deep learning techniques. Examples of such AI include, but are not limited to, neural networks such as FCN (Fully Convolutional Network), RNN (Recurrent Neural Network), and CNN (Convolutional Neural Network). Furthermore, the processor 110 (such as the feature extraction unit 118) may be equipped with AI that performs image processing functions and segmentation.

[0063] Instead of performing image processing or AI processing on the preliminary captured image as described above, region extraction and feature extraction may be performed based on user input. Region extraction and feature extraction using image processing or AI may be difficult depending on the type and characteristics of the object, the shooting conditions, etc. In such cases, the processor 110 may display the preliminary captured image on the display 170, and the user may specify the region and / or type of region of the object in the displayed image via the operation unit 150, or input feature quantities for the specified region. Specifically, the user may specify the above-mentioned low-contrast region or reflective region in the displayed image, or input feature quantities such as the degree of contrast or reflectivity. The processor 110 may perform a combination of image processing or AI processing and processing in response to user operation.

[0064] [Setting Shooting Conditions for Actual Photography] The processor 110 (e.g., the shooting condition determination unit 116) determines the shooting conditions for actual photography (first shooting conditions, which are shooting conditions when photographing the object) (step S130 in FIG. 8). It is preferable that the processor 110 determines the shooting conditions for actual photography based on at least one of the characteristics of the object and the environmental conditions when photographing the object. Here, the "characteristics of the object" are, for example, the characteristics of the low-contrast area and the characteristics of the reflective area described above, and the "environmental conditions when photographing the object" are, for example, the color and intensity of bright light and the color and intensity of ambient light. As described above for the preliminary photography, the shooting conditions for actual photography (first shooting conditions) include the lighting conditions of the lighting device and either the first polarization direction or the second polarization direction. Furthermore, the "lighting conditions" herein may include whether the illumination by any of the lighting devices 300 is turned on or off (one or more of the lighting devices 300 may be turned on), the irradiation position and / or direction, and the color and brightness of the illumination light, but the first polarization direction is not included in the "lighting conditions." The shooting conditions for the actual shooting may include at least the first polarization direction, and may further include other conditions (for example, the viewpoint position and / or shooting direction of the shooting camera 200, zoom, focus, exposure, etc.).

[0065] [Determination Regarding Shooting Control for Specific Region] Furthermore, in step S130, the processor 110 can determine whether to perform shooting control for a specific region based on the results of the region extraction and feature extraction described above. Specifically, the processor 110 can determine whether to perform shooting control for a low-contrast region (step S132 in FIG. 9 ) and whether to perform shooting control for a reflective region (step S134 in FIG. 9 ).

[0066] The processor 110 can determine to perform imaging control for low-contrast regions, for example, when the size of the low-contrast region in the second image and / or the proportion of the low-contrast region in the second image are equal to or greater than a predetermined threshold (first threshold). The processor 110 can also determine to perform imaging control for reflective regions, for example, when the size of the reflective region in the second image and / or the proportion of the reflective region in the second image are equal to or greater than a predetermined threshold (second threshold). That is, if no low-contrast or reflective regions are present in the preliminary captured image, or if they are present but their size or proportion is less than a threshold, the processor 110 does not need to perform imaging control for these regions because their impact on the three-dimensional model is minimal. The first and second thresholds can be set taking into account the accuracy required for the three-dimensional model.

[0067] The processor 110 can determine the shooting conditions for the low-contrast area as the third shooting condition, and can determine the shooting conditions for the reflective area as the fourth shooting condition. The processor 110 may determine at least one of the third shooting condition and the fourth shooting condition. The third shooting condition may include the lighting device 300 used and the polarization direction (first polarization direction) of the lighting device 300, and the fourth shooting condition may include the polarization direction (second polarization direction) of the shooting camera 200 used.

[0068] Specifically, the processor 110 can determine, as the third photographing condition, “performing fixed illumination direction photographing in which a plurality of illumination lights having different first polarization directions is irradiated onto the object from one illumination direction and the object is photographed under a plurality of illumination conditions with different first polarization directions” and “performing multiple illumination light photographing in which illumination light is irradiated onto the object from a plurality of directions simultaneously and the first polarization direction is fixed.”More specifically, the processor 110 can determine, as the third photographing condition, “performing fixed illumination direction photographing in which a plurality of illumination lights having different first polarization directions is irradiated onto the object from another illumination direction different from the one illumination direction.”Furthermore, the processor 110 can determine, as the fourth photographing condition, “simultaneously irradiating illumination light from a plurality of illumination directions and photographing under a plurality of second polarization directions.”

[0069] The third and fourth photographing conditions may include the number of times images are acquired in the first photographing of the low-contrast area and the reflective area, respectively. Furthermore, the processor 110 may determine whether to change the photographing conditions after changing the viewpoint position and the photographing direction (after returning to step S100) in response to a user operation via the operation unit 150. In other words, by fixing the photographing conditions, the time required to change the photographing conditions and the total photographing time may be shortened.

[0070] [Actual Shooting] The processor 110 (lighting device control unit 112, shooting camera control unit 114, shooting condition determination unit 116, etc.; image acquisition unit) performs actual shooting (first shooting) to photograph an object using the shooting camera 200 (shooting device) and the lighting device 300 (lighting device) based on the first shooting conditions, and acquires multiple actual shooting images (multiple first images) (step S140).

[0071] Hereinafter, a description will be given with reference to FIG. 10 (a flowchart showing the main photography process taking into consideration photography control for low contrast areas and reflective areas; details of the main photography in step S140).

[0072] [Determination of Shooting Control for Low-Contrast Area and Shooting Control for Reflective Area] The processor 110 (illumination device control unit 112, shooting camera control unit 114, shooting condition determination unit 116, etc.): determines whether to perform shooting control for the low-contrast area (step S141), and determines whether to perform shooting control for the reflective area (step S142 or step S143). Note that the order of the determinations in steps S141, S142, and S143 does not matter, as long as it is determined whether to perform shooting control for the two areas. The processor 110 can determine whether to perform shooting control for these areas using the first threshold and second threshold described above.

[0073] Based on the judgment in steps S141 to S143, (1) if shooting control for the low contrast area and shooting control for the reflective area are performed, proceed to step S144; (2) if shooting control for only the low contrast area is performed and shooting control for the reflective area is not performed, proceed to step S145; (3) if shooting control for only the reflective area is performed and shooting control for the low contrast area is not performed, proceed to step S146; or (4) if shooting control for the low contrast area and shooting control for the reflective area are not performed, proceed to step S147.

[0074] [Photography Control for Low-Contrast Areas] Fig. 11 is a diagram showing the photographing control for low-contrast areas. Fig. 11 shows the photographing using the photographing camera 200A. Note that in Fig. 11 and Fig. 12 described later, the photographing cameras 200 and the lighting devices 300 that are not used for photographing are omitted from the illustration.

[0075] In the state shown in part (a) of FIG. 11 , illumination by illumination device 300A is on, while illumination by the remaining illumination devices 300B-300F is off. That is, illumination light is irradiated onto object 99 from a single illumination direction. In this state, processor 110 repeatedly captures images while changing the polarization direction of illumination device 300A. Specifically, processor 110 (illumination device control unit 112, imaging camera control unit 114, etc.) captures images while changing the polarization direction (first polarization direction) of the illumination light between 0°, 45°, 90°, and 135° (see FIG. 7 ) using polarization direction setting mechanism 304 (see FIG. 7 ) (fixed illumination direction capture). That is, in fixed illumination direction capture, image capture is repeatedly performed for one polarization direction while changing the polarization direction, and multiple images (here, four images; multiple first images) with different polarization directions of illumination light are acquired for one illumination direction (illumination from illumination device 300A). The processor 110 (the photographing condition determination unit 116, the lighting device control unit 112, the photographing camera control unit 114, etc.) can determine such a photographing condition as the above-mentioned third photographing condition. During this time, the polarization direction (second polarization direction) on the photographing camera 200A side is fixed at an arbitrary direction.

[0076] Although four polarization directions are set for one shooting direction here, capturing images in more polarization directions per shooting direction (e.g., capturing images in polarization directions at 1-degree increments) improves the accuracy of the 3D model. However, capturing images in more polarization directions increases the number of images acquired, which in turn increases the shooting time and the time required to generate the 3D model, and requires more memory and other system resources. From this perspective, the polarization direction (first polarization direction) for one shooting direction can be determined by taking into account the relationship between the processing time and system processing load and the accuracy required for the 3D model. The processor 110 may determine the number of polarization directions and the spacing between the polarization directions (e.g., 45-degree spacing for four polarization directions, 22.5-degree spacing for eight polarization directions, etc.) in response to user operation, or may automatically determine the number of polarization directions and the spacing between the polarization directions in consideration of the above relationship, without user operation.

[0077] 11(a) is completed, the "fixed illumination direction photographing in which a plurality of illumination lights with different first polarization directions is irradiated from one illumination direction and an object is photographed under a plurality of illumination conditions with different first polarization directions" is repeated for other illumination directions. For example, as shown in FIG. 11(b), a similar photographing is performed using illumination device 300B, and when that is completed, similar photographing is performed sequentially for illumination devices 300C to 300F.

[0078] 12 is another diagram showing the state of shooting control for low-contrast areas. In shooting control for low-contrast areas, in addition to the fixed illumination direction shooting described above, the processor 110 simultaneously irradiates illumination light from all illumination devices 300 (illumination devices 300A to 300F; multiple illumination directions) to shoot the object 99 (multiple illumination light shooting). In this case, the polarization direction (first polarization direction) on the illumination device 300 side is fixed, and the polarization direction (second polarization direction) on the shooting camera 200A side is arbitrary. This type of shooting only needs to be performed once for each shooting direction.

[0079] [Photography Control for Reflective Area] When a reflective area is present, the processor 110 simultaneously turns on all lighting devices 300 as in Fig. 12 and photographs the object while changing the polarization direction (polarization direction during photographing; second polarization direction) on the photographing camera 200A (photographing camera 200) side between 0°, 45°, 90°, and 135° (photographing for a plurality of second polarization directions). The processor 110 (the lighting device control unit 112, the photographing camera control unit 114, the photographing condition determination unit 116, etc.) can determine such photographing conditions as fourth photographing conditions.

[0080] As described above with respect to the polarization direction of the illumination light (first polarization direction), it is preferable to capture images in a large number of second polarization directions from the viewpoint of the accuracy of the three-dimensional model, but the processor 110 can determine the number of polarization directions (second polarization directions) during image capture in consideration of the relationship between the processing time, the processing load of the system, and the accuracy required for the three-dimensional model. The processor 110 may determine the number of second polarization directions and / or the spacing therebetween in response to a user operation, or may automatically determine the number of second polarization directions and / or the spacing therebetween in consideration of the above relationship, without relying on a user operation.

[0081] [When the photographing control for the low-contrast area and the photographing control for the reflective area are not performed] When the photographing control for the low-contrast area and the photographing control for the reflective area described above are not performed, the processor 110 performs photographing by fixing the first polarization direction and the second polarization direction at any direction. For example, for one photographing direction, photographing can be performed in which illumination light is emitted from one of the illumination devices 300A to 300F (six times in total) and in which illumination light is emitted from the illumination devices 300A to 300F simultaneously (one time).

[0082] Once the necessary images have been captured for one viewpoint position and shooting direction, the process returns to step S100, and preliminary and actual shooting are repeated for other viewpoint positions and shooting directions. In this embodiment, the process switches from shooting camera 200A to shooting camera 200B, and shooting is repeated. This process is repeated until shooting is completed for all viewpoint positions and shooting directions (until step S150 returns YES). Once acquisition of multiple images (multiple actual captured images, multiple first images) to be used in generating a 3D model is completed (step S150 returns YES), the process proceeds to generating a 3D model (step S160 in FIG. 8).

[0083] The processor 110 (photographing condition determination unit 116, input / output control unit 122, etc.) associates the multiple actual photographed images (multiple first images) acquired in the actual photographing with the photographing conditions (first photographing conditions) of the actual photographing, and records them in the recording device 190. For example, the photographing conditions may be recorded in the header of the image, or the association between the image file and the photographing condition file may be recorded separately in a table format.

[0084] [Modifications for Changing the Viewpoint Position and Shooting Direction] Figure 13 is a diagram showing modifications for changing the viewpoint position and shooting direction. While Figure 1 and other figures illustrate an embodiment in which multiple shooting cameras are used for shooting, a single shooting camera may be used as long as the viewpoint position and shooting direction can be changed. For example, as shown in part (a) of Figure 13, the shooting camera 200 may be mounted on a dolly 220 that can pan, tilt, retract its arm, and move, and the viewpoint position and shooting direction may be changed using the dolly 220. A similar configuration may also be adopted for the lighting device 300.

[0085] Part (b) of Figure 13 is a diagram showing a rotatable stage 500 (as viewed from vertically above). By placing an object on the stage 500 and rotating it, it is possible to capture images from multiple viewpoints and shooting directions using a single camera 200. Furthermore, instead of placing the object on a stage, the camera 200 and / or the lighting device 300 may be installed on such a stage to change the viewpoint, shooting direction, and lighting direction.

[0086] [Generation and Synthesis of 3D Model] After capturing all of the required images (plurality of first images), the processor 110 (3D model generation unit 120; generation unit) uses the images to generate a 3D model of the object using a photogrammetry technique (step S160 in FIG. 8). In photogrammetry, the position and orientation of the camera are estimated from the images using SfM (Structure from Motion), and the 3D model is generated using MVS (Multi-View Stereo). The processor 110 can generate the 3D model by referring to the association between the shooting conditions (first shooting conditions) of the actual shooting and the multiple actually shot images (plurality of first images), thereby improving the accuracy of the 3D model.

[0087] The processor 110 (generation unit) generates a 3D model (first 3D model) of the object from a plurality of actual captured images (a plurality of first images captured under the third capturing condition) obtained by performing the above-described capturing control for the low-contrast region, generates a 3D model (second 3D model) of the object from a plurality of actual captured images (a plurality of first images captured under the fourth capturing condition) obtained by performing the capturing control for the reflective region, and generates a single 3D model having shape and texture information based on the first and second 3D models (synthesis of a final 3D model). In this case, the processor 110 can generate a 3D model of the object by using the first 3D model for a first portion of the object corresponding to the low-contrast region and the second 3D model for a second portion of the object corresponding to the reflective region.

[0088] Processor 110 can also generate three-dimensional models from "a group of images obtained by taking images with one of lighting devices 300A to 300F on and the remaining five off, and repeating this process for lighting devices 300A to 300F" and "a group of images taken with all lighting devices 300A to 300F on," and can use these three-dimensional models to generate three-dimensional models with shape and texture information.

[0089] In normal photography (photography without controlling the polarization direction) or photography in which the polarization direction of either the lighting device side or the shooting camera side is controlled, the influence of reflective areas can be eliminated, but the shape of the object cannot be accurately reproduced in low-contrast areas, resulting in a deterioration in the quality of the 3D model. According to the 3D model generation system 10 of this embodiment, by suppressing reflections and acquiring accurate texture information, and controlling the polarization direction on the lighting device side and the polarization direction on the shooting camera side, it is possible to acquire accurate 3D model shapes and textures even for objects with low-contrast areas.

[0090] [Generation of 3D Models Using Visual Volume Intersection Method] Photogrammetry matches feature points between images and generates a 3D model using the processing results. For this reason, objects for which a sufficient number of feature points cannot be obtained from the images, such as transparent or translucent objects, are considered unsuitable for photogrammetry. Therefore, the processor 110 may generate a 3D model using visual volume intersection. Visual volume intersection, also known as silhouette intersection, is a technique for generating a 3D model based on the silhouette of an object. Visual volume intersection uses the silhouettes of an object photographed from multiple viewpoints to create a cone (visual volume) with each viewpoint as a vertex and the silhouette as a cross section. The silhouettes are then back-projected into three-dimensional space, and their intersections (common areas) are found to generate a 3D model.

[0091] [Generating a 3D model by combining two methods] In contrast to the above-mentioned photogrammetry, the volume intersection method is a method for generating a 3D model using a silhouette, so even if the object has transparent or semi-transparent parts, a 3D model can be generated as long as the silhouette can be obtained. However, the accuracy of the reconstructed 3D shape may be inferior to that of photogrammetry.

[0092] Therefore, in the 3D model generation system 10, the processor 110 may combine these techniques to generate a 3D model. Specifically, using boundary information between the object and the background, the 3D shape of a certain portion of the object may be estimated using volume intersection, and the 3D shape of the remaining portion may be estimated using photogrammetry, and a final 3D model may be generated based on the 3D shapes obtained using these two techniques. In this case, for example, volume intersection may be used for transparent or semitransparent portions of the object, and photogrammetry may be used for the remaining portions. The processor 110 may acquire, as feature information from the above-mentioned preliminary captured image, which portions are transparent or semitransparent, and determine the 3D model generation method based on this feature information.

[0093] [Output of 3D Model] The processor 110 (3D model generation unit 120, input / output control unit 122, etc.) can output the generated 3D model to an output device (step S170 in FIG. 8). The processor 110 may display the 3D model on the display 170, or may record it in the recording device 190 as 3D model 190E (see FIG. 4). The 3D model may also be printed by a 3D printer (not shown) via the input / output interface 180.

[0094] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described aspects and various modifications are possible.

[0095] 10 3D model generation system 99 Object 99A Reflection area 100 Controller 110 Processor 112 Lighting device control unit 114 Shooting camera control unit 116 Shooting condition determination unit 118 Feature extraction unit 120 3D model generation unit 122 Input / output control unit 150 Operation unit 160 Speaker 170 Display 180 Input / output interface 190 Recording device 190A Preliminary captured image 190B Actual captured image 190C Feature information 190E 3D model 195 Bus 200 Shooting camera 200A Shooting camera 200B Shooting camera 200C Shooting camera 200D Shooting camera 200E Shooting camera 200F Shooting camera 202 Lens 204 Polarization direction setting mechanism 206 Image pickup element 210 A / D converter 212 Lens driving unit 214 Signal processing unit 220 Cart 300 Illumination device 300A Illumination device 300B Illumination device 300C Illumination device 300D Illumination device 300E Illumination device 300F Illumination device 302 Lens 304 Polarization direction setting mechanism 306 Light source device 400A Polarizing filter 400B Polarizing filter 400C Polarizing filter 400D Polarizing filter 500 Stage Steps S100 to S170 Each step of processing in the 3D model generation system

Claims

1. A three-dimensional model generation system comprising: an illumination device having a mechanism for setting a first polarization direction, which is the polarization direction of illumination light; an imaging device having a mechanism for setting a second polarization direction, which is the polarization direction during imaging; a determination unit that determines first imaging conditions, which are imaging conditions when imaging the object, based on at least one of the characteristics of the object and the environmental conditions when imaging the object; an image acquisition unit that performs a first imaging to image the object using the imaging device and the illumination device based on the first imaging conditions, and acquires a plurality of first images; and a generation unit that generates a three-dimensional model of the object using the plurality of first images, wherein the first imaging conditions include any of the conditions of illumination by the illumination device, the first polarization direction, and the second polarization direction.

2. A three-dimensional model generation system as described in claim 1, wherein the determination unit determines the first shooting conditions based on the characteristics of a second image including the object obtained by a second shooting prior to the first shooting.

3. A three-dimensional model generation system as described in claim 1 or 2, wherein the determination unit extracts a first region and a second region from a second image including the object, which is acquired by a second photographing operation prior to the first photographing operation, and determines the first photographing conditions based on the characteristics of the first region and the characteristics of the second region.

4. A three-dimensional model generation system as described in claim 3, wherein the characteristics of the first region include the size of the first region in the second image and / or the proportion of the first region that occupies the second image, and the characteristics of the second region include the size of the second region in the second image and / or the proportion of the second region that occupies the second image.

5. The three-dimensional model generation system of claim 3, wherein the determination unit extracts features of low-contrast areas in the second image as features of the first area, extracts features of reflective areas in the second image as features of the second area, and determines the first shooting conditions based on the features of the low-contrast areas and the features of the reflective areas.

6. A three-dimensional model generation system as described in claim 5, wherein the determination unit determines, as the first shooting condition, at least one of a third shooting condition which is a shooting condition for the low contrast area and a fourth shooting condition which is a shooting condition for the reflective area.

7. A three-dimensional model generation system according to claim 6, wherein the third and fourth photographing conditions include the number of times images are acquired in the first photographing.

8. The three-dimensional model generation system according to claim 6, wherein the determination unit determines as the third shooting conditions: fixed lighting direction shooting in which the object is irradiated with a plurality of illumination lights with different first polarization directions from one lighting direction and the object is photographed under a plurality of illumination conditions with different first polarization directions; and multiple illumination light shooting in which the object is irradiated with illumination lights from a plurality of directions simultaneously and the first polarization direction is fixed.

9. A three-dimensional model generation system as described in claim 8, wherein the determination unit determines as the third shooting condition that, in the fixed lighting direction shooting, shooting in one polarization direction is repeatedly performed while changing the polarization direction.

10. A three-dimensional model generation system as described in claim 9, wherein the determination unit determines as the third shooting condition that the object is shot with a plurality of illumination lights having different first polarization directions from another illumination direction different from the one illumination direction, and the shooting is performed with the illumination direction fixed.

11. A three-dimensional model generation system as described in claim 6, wherein the determination unit determines that the fourth shooting condition is to simultaneously irradiate illumination light from multiple lighting directions and shoot in multiple second polarization directions.

12. A three-dimensional model generation system as described in claim 1 or 2, wherein the determination unit determines that the first shooting condition is to photograph the object from multiple shooting directions in the first shooting to obtain the multiple first images.

13. A three-dimensional model generation system as described in claim 12, wherein the image acquisition unit photographs the object from the plurality of photographing directions by moving and / or rotating the photographing device and the object relative to each other.

14. A three-dimensional model generation system according to claim 1 or 2, wherein the determination unit determines that the lighting conditions in the first shooting are to photograph the object under a plurality of lighting conditions with different lighting directions.

15. A three-dimensional model generation system according to claim 14, wherein the image acquisition unit varies the illumination direction by relatively moving and / or rotating the illumination device and the object.

16. A three-dimensional model generation system according to claim 1 or 2, wherein the lighting device comprises a plurality of lighting devices, and the determination unit determines, as the lighting condition, from which of the plurality of lighting devices to irradiate illumination light in the first photographing.

17. A three-dimensional model generation system as described in claim 1 or 2, wherein the image acquisition unit associates the plurality of first images with the first shooting conditions and records them in a recording device, and the generation unit generates the three-dimensional model by referring to the association.

18. The three-dimensional model generation system described in claim 6, wherein the generation unit generates a first three-dimensional model of the object using the plurality of first images acquired under the third shooting conditions, generates a second three-dimensional model of the object using the plurality of first images acquired under the fourth shooting conditions, and generates a three-dimensional model of the object based on the first three-dimensional model and the second three-dimensional model.

19. A three-dimensional model generation system as described in claim 18, wherein the generation unit generates a three-dimensional model of the object using the first three-dimensional model for a first portion of the object corresponding to the low-contrast region, and using the second three-dimensional model for a second portion of the object corresponding to the reflective region.

20. A photographing condition setting device having a processor, wherein the processor: sets a first polarization direction, which is the polarization direction of illumination light from a lighting device; sets a second polarization direction, which is the polarization direction when photographing by a photographing device; determines first photographing conditions, which are photographing conditions when photographing the object, based on at least one of the characteristics of the object and the environmental conditions when photographing the object; performs first photographing to photograph the object using the photographing device and the lighting device based on the first photographing conditions, and obtains a plurality of first images; and the first photographing conditions include any of the lighting conditions by the lighting device, the first polarization direction, and the second polarization direction.

21. The photographing condition setting device of claim 20, wherein the processor: acquires characteristics of low-contrast areas in the object and characteristics of reflective areas in the object; determines a third photographing condition corresponding to the characteristics of the low-contrast areas and a fourth photographing condition corresponding to the characteristics of the reflective areas; and causes the photographing device to photograph the object based on the third photographing condition and / or the fourth photographing condition to obtain the plurality of first images.

22. A photography system comprising: an illumination device having a mechanism for setting a first polarization direction, which is the polarization direction of illumination light; an imaging device having a mechanism for setting a second polarization direction, which is the polarization direction during photography; and an imaging control device, wherein the imaging control device determines first imaging conditions, which are imaging conditions for photographing the object, based on at least one of the characteristics of the object and the environmental conditions when photographing the object; and performs first imaging to photograph the object using the imaging device and the illumination device based on the first imaging conditions, thereby obtaining multiple first images, and the first imaging conditions include any of the lighting conditions used by the illumination device, the first polarization direction, and the second polarization direction.

Images