Information processing device, imaging device, information processing method, and program

Abstract

Provided are an information processing device, an imaging device, an information processing method, and a program with which it is possible to acquire an image for generating a high-quality three-dimensional model. An information processing device (18) is provided with a processor (40) and determines a first imaging condition (60) of an image for generating the three-dimensional model of an object. The processor (40) acquires optical design information (62) of the imaging device used for imaging, information (64) related to an imaging system, and information (66) related to the feature of the object, and determines, on the basis of the acquired information, a first imaging condition (60) including a condition related to the imaging distance for capturing an image of the object.

Description

Information processing device, photographing device, information processing method, and program

[0001] The present invention relates to an information processing device, an imaging device, an information processing method, and a program, and more particularly to an information processing device, an imaging device, an information processing method, and a program for determining imaging conditions when acquiring images for generating a three-dimensional model.

[0002] Patent Literature 1 describes a technology that uses a drone to measure the tilt of a steel tower or the sag of a power line during measurement and maintenance work on power transmission and distribution facilities. The drone's photography is controlled by an inspection application module. The inspection application module executes a program that optimizes photography parameters (distance, angle, exposure, etc.).

[0003] Japanese Patent Application Laid-Open No. 2022-115759

[0004] One embodiment of the technique of the present disclosure provides an information processing device, an imaging device, an information processing method, and a program that can acquire images for generating a high-quality three-dimensional model.

[0005] A first aspect of the present invention is an information processing device that includes a processor and determines first shooting conditions for an image for generating a three-dimensional model of an object, in which the processor acquires optical design information of the shooting device used for shooting, information about the imaging system, and information about the characteristics of the object, and determines first shooting conditions including conditions related to the shooting distance for shooting an image of the object based on the acquired information.

[0006] A second aspect of the present invention is the first aspect, in which the processor acquires shooting range information indicating information about the shooting range of the object on the imaging element, which is set based on optical design information of the imaging device and information about the imaging system, acquires one or more pieces of reflection range information indicating the reflection range of the object on the imaging element based on the optical design information of the imaging device, information about the imaging system, and information about the characteristics of the object, and determines a first shooting condition based on the shooting range information and the one or more pieces of reflection range information.

[0007] In a third aspect of the present invention, in the second aspect, the processor determines the first shooting condition within a range that satisfies the following formula (1), where A is the shooting range information and B is one or more pieces of reflected range information: B≦A... formula (1)

[0008] In a fourth aspect of the present invention, in the third aspect, the processor determines the maximum captured range information that satisfies the formula (1) as the first shooting condition.

[0009] In a fifth aspect of the present invention, in the third aspect, the processor determines the first shooting conditions within a range B that satisfies the following formula (2), where C is the minimum value of the reflection range information that is allowable when generating a three-dimensional model: C≦B≦A... formula (2)

[0010] A sixth aspect of the present invention is the third aspect, wherein the processor acquires information regarding the depth of field, acquires a plurality of second shooting conditions that satisfy equation (1) based on the shooting range information and one or more pieces of reflection range information, and determines a first shooting condition from among the plurality of second shooting conditions based on the information regarding the depth of field.

[0011] A seventh aspect of the present invention is the sixth aspect, wherein the processor determines the first shooting condition from among a plurality of second shooting conditions based on information about the deepest depth of field.

[0012] An eighth aspect of the present invention is the sixth aspect, wherein the processor obtains an allowable depth of field by expanding information regarding the depth of field, and determines a first shooting condition from among a plurality of second shooting conditions based on the allowable depth of field.

[0013] A ninth aspect of the present invention is that, in any of the sixth to eighth aspects, when the processor is unable to determine a first shooting condition from among a plurality of second shooting conditions based on information about the depth of field, the processor changes the aperture value to obtain information about the depth of field, and determines a first shooting condition from among the plurality of second shooting conditions based on the information about the depth of field.

[0014] A tenth aspect of the present invention is the first aspect, wherein the processor acquires information about the depth of field, and determines the first shooting condition based on the information about the depth of field.

[0015] In an eleventh aspect of the present invention, in any one of the first to tenth aspects, the processor acquires information about the characteristics of the object through an input from a user.

[0016] A twelfth aspect of the present invention is the method according to any one of the first to eleventh aspects, wherein the processor acquires information about the characteristics of the object from the image.

[0017] A photographing device according to a thirteenth aspect of the present invention includes the information processing device according to any one of the first to twelfth aspects.

[0018] An information processing method according to a fourteenth aspect of the present invention is an information processing method for determining first shooting conditions for an image for generating a three-dimensional model of an object, and includes the steps of acquiring, by a processor, optical design information of the shooting device to be used for shooting, information about the shooting system, and information about the characteristics of the object, and determining, based on the acquired information, first shooting conditions including conditions related to the shooting distance for shooting an image of the object.

[0019] A fifteenth aspect of the present invention is the fourteenth aspect, further comprising a step of acquiring information regarding depth of field performed by a processor, wherein in the step of determining a first shooting condition, a plurality of second shooting conditions are determined, and a first shooting condition is selected from the plurality of second shooting conditions based on the information regarding depth of field.

[0020] A sixteenth aspect of the present invention is a program that causes a computer to execute an information processing method for determining first shooting conditions for images for generating a three-dimensional model of an object, and causes a processor to execute the steps of acquiring optical design information of the shooting device used for shooting, information about the shooting system, and information about the characteristics of the object, and determining first shooting conditions including conditions related to the shooting distance for shooting an image of the object based on the acquired information.

[0021] FIG. 1 is a diagram illustrating 3D scanning of an object using an imaging device. FIG. 2 is a block diagram illustrating an embodiment of the internal configuration of the imaging device. FIG. 3 is a diagram illustrating information acquired by an information processing device and imaging conditions output by the information processing device. FIG. 4 is a flowchart illustrating an information processing method executed by the information processing device. FIG. 5 is a flow diagram illustrating calculation of candidate combinations of focal length f and shooting distance D and determination of imaging conditions performed by the information processing device. FIG. 6 is a conceptual diagram illustrating a preferable imaging range. FIG. 7 is a diagram illustrating table data of imaging range A for each focus position and zoom position. FIG. 8 is a diagram illustrating table data indicating range B in which an object is captured on an image sensor. FIG. 9 is a diagram illustrating table data indicating the distance from the imaging device to the in-focus point. FIG. 10 is a diagram illustrating table data indicating focus positions and zoom positions that satisfy the relationship of capture range B≦imaging range A. FIG. 11 is a flow diagram illustrating calculation of candidate combinations of focal length f and shooting distance D and determination of imaging conditions performed by the information processing device. FIG. 12 is a diagram illustrating table data indicating the depth of field according to the focus position and zoom position. FIG. 13 is a diagram illustrating 3D scanning of an object using an imaging device.

[0022] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of an information processing device, an image capturing device, an information processing method, and a program according to the present invention will now be described with reference to the accompanying drawings.

[0023] [Summary of the Invention] First, an overview of the present invention will be described.

[0024] In recent years, the use of three-dimensional content has been expanding in fields such as games, metaverse-type communication spaces, architecture, and video production. Furthermore, the use of 3D scanning technology, which uses cameras or specialized equipment to stereoscopically scan real-world objects and convert them into three-dimensional data, is becoming increasingly common. One 3D scanning method is photogrammetry (also known as SfM (Structure from Motion) or MVS (Multi-View Stereo)), which generates 3D data from data captured from multiple viewpoints using one or more cameras.

[0025] When performing photogrammetry, it is necessary to appropriately set the distance between the camera and the object according to the characteristics of the camera and lens used for shooting. When determining the distance, factors to consider include the depth of field, distortion and / or image quality degradation caused by the lens (optical system) at the periphery of the image, etc.

[0026] Furthermore, when performing 3D scanning, it is desirable for the entire area of ​​the object to be in focus. For this reason, the lens aperture is generally narrowed to increase the depth of field, but depending on the focal length of the lens and the distance to the object, the depth of field may become shallow, and the entire object may not be in focus.

[0027] Furthermore, when trying to capture an object at high resolution to improve 3D scanning performance, it is desirable to move the camera closer to the object so that the object is captured large within the angle of view. However, depending on the lens used, distortion around the periphery of the image and degradation of image quality can become significant due to the lens (optical system). Because photogrammetry is susceptible to image distortion and degradation, it is necessary to set an appropriate angle of view taking into account the lens characteristics.

[0028] As such, it is necessary to determine appropriate imaging conditions for performing 3D scanning.

[0029] Therefore, in the present invention described below, appropriate shooting conditions are determined based on the specifications and / or performance of the camera and lens used and information about the object, and these are presented to the user (photographer), thereby acquiring images to be used for photogrammetry under appropriate conditions and generating high-quality 3D data. For example, the present invention can also be applied to image acquisition when generating a 3D model using AI (Artificial Intelligence) based on images captured from multiple viewpoints.

[0030] First Embodiment FIG. 1 is a diagram illustrating 3D scanning of an object P by an imaging device 10. As shown in FIG.

[0031] The imaging device 10 performs a 3D scan of an object P. The imaging device 10 is equipped with an information processing device 18 of the present invention. Therefore, the imaging conditions determined by the information processing device 18 are presented on an image monitor 30 (see FIG. 2 ) provided on the rear surface of the imaging device 10. This allows the imaging conditions to be presented to the user, and the imaging device 10 can be used to acquire images for generating a 3D model under appropriate conditions.

[0032] The object P is a shoe and is placed on the rotating table 1. A plurality of images (images for generating a three-dimensional model) are acquired while rotating the rotating table 1. By rotating the rotating table 1, the image capturing device 10 can capture images of the object P from different shooting directions.

[0033] Although the present embodiment describes an example in which the object P is rotated by the turntable 1, the application of the present invention is not limited to this. For example, a user may walk around the object P and photograph the object P with the photographing device 10 to obtain multiple images.

[0034] 2 is a block diagram showing an embodiment of the internal configuration of the photographing device 10. The photographing device 10 records photographed images on a memory card 54, and the operation of the entire device is centrally controlled by a control unit 40 (CPU: Central Processing Unit: processor).

[0035] The photographing device 10 is provided with an operation unit 38 including a shutter button, a power / mode switch, a mode dial, etc. Signals (commands) from this operation unit 38 are input to a control unit 40, which controls each circuit of the photographing device 10 based on the input signals, and performs drive control of the image sensor 16, lens drive control, aperture drive control, image capture operation control, image processing control, image data recording / playback control, and display control of the image monitor 30.

[0036] The light beam passing through the lens device 12 forms an image on the image sensor 16, which is a CMOS (Complementary Metal-Oxide Semiconductor) color image sensor. The image sensor 16 is not limited to the CMOS type, and other types of image sensors such as a CCD (Charge Coupled Device) type or an organic image sensor may also be used.

[0037] The image sensor 16 has a large number of light-receiving elements (e.g., photodiodes) arranged two-dimensionally, and the subject image formed on the light-receiving surface of each light-receiving element is converted (photoelectrically converted) into a signal voltage (or charge) of an amount corresponding to the amount of incident light, and then converted into a digital signal via an A / D (Analog / Digital) converter within the image sensor 16 and output.

[0038] Image signals (image data) read from the image sensor 16 when capturing a moving image or a still image are temporarily stored in a memory (SDRAM (Synchronous Dynamic Random Access Memory)) 48 via the image input controller 22 .

[0039] Furthermore, a flash memory 47 stores a camera control program and various parameters and tables used for image processing and the like.

[0040] The image processing unit 24 reads unprocessed image data that is acquired via the image input controller 22 when capturing moving images or still images and that is temporarily stored in the memory 48. The image processing unit 24 performs offset processing, pixel interpolation processing (interpolation processing for phase difference detection pixels, defective pixels, etc.), white balance correction, gain control processing including sensitivity correction, gamma correction processing, synchronization processing (also called "demosaic processing"), luminance and color difference signal generation processing, contour enhancement processing, color correction, etc. The image data that has been processed by the image processing unit 24 and that has been processed as a live view image is input to a VRAM (Video RAM Random Access Memory) 50.

[0041] The image data read from the VRAM 50 is encoded by the video encoder 28 and output to the image monitor 30 provided on the rear surface of the image capturing device 10. As a result, a live view image showing the subject image is displayed on the image monitor 30.

[0042] The image data (brightness data (Y) and color difference data (Cb), (Cr)) processed by the image processing unit 24 as a still image or a moving image for recording is stored again in the memory 48.

[0043] When recording a still image or a moving image, the compression / decompression processing unit 26 performs compression processing on the luminance data (Y) and color difference data (Cb, Cr) processed by the image processing unit 24 and stored in the memory 48. The compressed image data is recorded on a memory card 54 via a media controller 52.

[0044] In addition, in the playback mode, the compression / decompression processing unit 26 performs decompression processing on compressed image data obtained from the memory card 54 via the media controller 52. The media controller 52 records and reads compressed image data onto and from the memory card 54.

[0045] Here, the photographing device 10 is equipped with an information processing device 18 that determines photographing conditions (first photographing conditions) for acquiring images for generating a three-dimensional model of the object P. The information processing device 18 is configured with the internal configuration of the photographing device 10 described above. For example, the information processing device 18 is configured with a control unit (processor) 40, a memory 48, and a flash memory 47. Note that the configuration of the information processing device 18 is not limited to these, and may include the internal configuration of the photographing device 10 as needed.

[0046] In the above embodiment, the hardware structure of the processing unit (such as the control unit 40) that executes various processes is the following various processors: The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), whose circuit configuration can be changed after manufacture, and a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing specific processes.

[0047] A single processing unit may be configured with one of these various processors, or may be configured with two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). Multiple processing units may also be configured with a single processor. Examples of multiple processing units configured with a single processor include: a first configuration, as typified by client or server computers, in which a single processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units; and a second configuration, as typified by system-on-chip (SoC), in which a processor is used to realize the functions of an entire system including multiple processing units on a single IC (Integrated Circuit) chip. In this way, the various processing units are configured with one or more of the above-mentioned various processors as a hardware structure.

[0048] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit made up of a combination of circuit elements such as semiconductor elements.

[0049] Furthermore, each configuration and function can be realized as appropriate by any hardware, software, or a combination of both. For example, the present invention can be applied to a program that causes a computer to execute the above-described processing steps (processing procedures), a computer-readable recording medium (non-transitory recording medium) on which such a program is recorded, or a computer on which such a program can be installed.

[0050] 3 is a diagram for explaining information acquired and photographing conditions output by the information processing device 18. Note that Fig. 3 illustrates a control unit 40, which is a main part of the information processing device 18.

[0051] The control unit 40 determines the photographing conditions 60 for acquiring images for generating a three-dimensional model of the object P. The photographing conditions 60 include conditions related to the photographing distance D for photographing images for generating a three-dimensional model. Here, the photographing conditions 60 may be the distance (m) from the object P, or may be conditions for approaching (coming closer) or receding (moving away) by a distance (cm) from the current position.

[0052] The control unit 40 acquires optical design information 62 of the imaging device 10 used for imaging, information 64 about the imaging system, and information 66 about the characteristics of the object P. Then, based on the acquired information, the control unit 40 determines imaging conditions 60. The determined imaging conditions 60 are displayed on the image monitor 30 and notified to the user.

[0053] The photographing conditions 60 include conditions related to the photographing distance D for photographing an image used to generate a three-dimensional model of the object P. Therefore, by photographing the object P in accordance with the determined photographing conditions 60, the user can photograph an image for generating a three-dimensional model under appropriate conditions.

[0054] The optical design information 62 is design information (numeric design information) of the lens device 12 used in the imaging device 10. Here, the design information of the lens device 12 is not limited to numerical design information, but may be actual measurement values ​​(measurement values) of the optical characteristics of the lens device 12. The optical design information 62 may also be information regarding the type of the lens device 12, such as whether it is a zoom lens or a prime lens. The control unit 40 may, for example, acquire the optical design information 62 from the lens device 12, or may acquire optical design information stored in the memory 48.

[0055] The information 64 about the imaging system is information about the design of the imaging element 16. Specific examples of the information 64 about the imaging system include the actual size of the imaging element 16, the number of pixels, the pixel pitch, and the distance from the lens. The control unit 40 may acquire information about the imaging system stored in the memory 48, for example.

[0056] The information 66 about the characteristics of the object P is information about the size of the object P. Specific examples of the information 66 about the characteristics of the object P include the thickness, width, and height of the object P. The information 66 about the characteristics of the object P may be input by the user using the operation unit 38, for example. Alternatively, the size of the object P may be acquired by photographing the object P with the image capture device 10 (pre-scanning for size measurement). Specifically, multiple images are captured while rotating the turntable 1 at a predetermined zoom position and a predetermined shooting distance D, and masking is performed to extract portions of the object P from each image. This masking is performed using AI (artificial intelligence) processing, a green screen, and general area extraction techniques. After the size of the object P in the image is determined through masking, the actual size of the object P is calculated based on information about the focal length f and shooting distance D of the image capture device 10. This processing is performed on all images acquired in the pre-scanning for size measurement to estimate the size of the object P.

[0057] Note that Figure 3 describes a case where optical design information 62 of the imaging device 10 used for imaging, information 64 about the imaging system, and information 66 about the characteristics of the object P are acquired, but the present invention also includes a case where two of these three pieces of information use specified values ​​(predetermined numerical values ​​that cannot be changed).

[0058] Next, a description will be given of an information processing method using the information processing device 18. The information processing method is performed by causing the processor (controller 40) of the information processing device 18 to execute a dedicated program.

[0059] Fig. 4 is a flowchart showing an information processing method executed by the information processing device 18. Note that the flowchart shown in Fig. 4 mainly explains the acquisition of information by the control unit 40.

[0060] First, the information processing device 18 acquires optical design information 62 and information 64 relating to the imaging system (step S10).

[0061] These pieces of information are used by the information processing device 18 to calculate shooting range information. These pieces of information are also required to calculate the depth of field, which will be described in the second embodiment. Specifically, the information processing device 18 acquires optical design information 62 from the lens device 12 and acquires information related to the imaging system from the memory 48.

[0062] Here, the shooting range information is the range of angle of view within which optical degradation is tolerable when performing photogrammetry, and differs for each zoom position and focus position of the lens, and also differs for each lens device 12. Therefore, the lens device 12 may store shooting range information corresponding to the zoom position and focus position in advance in an internal memory (not shown) or in the flash memory 47 of the imaging device 10.

[0063] Next, the information processing device 18 acquires information about the characteristics of the object (step S11).

[0064] The information processing device 18 acquires information on the size of the object P (thickness T, height H, width W).

[0065] The user inputs information about the size of the object P, for example, from the operation unit 38. When the user inputs information about the size of the object P, the value input by the user is set as the size of the object P (step S12).

[0066] On the other hand, if the user does not input any information regarding the size of the object P, default values ​​(for example, T (thickness), H (height), and W (width) are each 100 mm) are set as the size of the object P (step S15).

[0067] Next, the information processing device 18 calculates candidate combinations of focus positions and zoom positions (step S13).

[0068] The information processing device 18 calculates candidate combinations of focal length f and shooting distance D based on the acquired optical design information of the image capturing device 10, information about the image capturing system, and information about the characteristics of the object.

[0069] The information processing device 18 also determines the photographing conditions from the candidate combinations of the focal length f and the photographing distance D, and presents the photographing conditions to the user (step S14).

[0070] The calculation of candidate combinations of focal length f and shooting distance D by the information processing device 18 and the determination of shooting conditions from the candidate combinations of focal length f and shooting distance D will be described in detail later.

[0071] Next, the calculation of candidate combinations of focal length f and shooting distance D (step S13 in FIG. 4) and the determination of shooting conditions (step S14 in FIG. 4) in the information processing device 18 will be described.

[0072] FIG. 5 is a flow diagram illustrating the calculation of candidate combinations of focal length f and shooting distance D and the determination of shooting conditions performed by the information processing device 18.

[0073] First, the information processing device 18 acquires a preferred imaging range A (step S20). The preferred imaging range A indicates information related to the preferred imaging range A of the object P in the image sensor 16 (imaging range information). The imaging range A is expressed, for example, as an area. The imaging range A may also be expressed, for example, as the proportion of the object P that occupies the acquired image. The imaging range A may also be the actual size of the image sensor 16 and the physical size (in millimeters) of the object P captured therein, or it may be the ratio of the range of the object P to the entire image range in the output image on a pixel size basis (number of pixels).

[0074] The information processing device 18 acquires a preferable photographing range A based on optical design information of the lens and design information related to the imaging element.

[0075] FIG. 6 is a diagram conceptually showing a preferable photographing range A. In FIG.

[0076] 6, the image sensor 16, the image pickup surface 16a, and the preferable image pickup range A of the image pickup surface 16a are indicated by dotted lines. Also, the state in which the object P is photographed in the image pickup range A is shown.

[0077] The shooting range A is calculated as a range that satisfies all of the conditions for optical degradation, such as a range in which the amount of light is 95% or more of the center of the image (the image position where the maximum amount of light is obtained) and a range in which the absolute amount of distortion is less than 5%. Note that the method for calculating the shooting range A is not limited to this, and other methods may be used as long as they can determine an optimal range for the image sensor 16. For example, the shooting range A may be set to a range that satisfies one of the conditions for optical degradation.

[0078] FIG. 7 is a diagram showing table data of the photographing range A for each focus position and zoom position.

[0079] In table data D1, the zoom position is normalized so that the wide end is 1 and the telephoto end is 10, and the focus position is normalized so that INF (infinity) is 1 and MOD (minimum object distance) is 10. Each value indicates the diameter of a circle based on the center of the image sensor 16, when the diagonal length of the image sensor 16 is 1. This is equivalent to the image height.

[0080] The photographing range A is determined according to the image sensor 16 and is a value specific to each image sensor 16. The photographing range A is also a value determined by the combination of the zoom position and the focus position.

[0081] For example, based on the table data D1, when the focus position is "3" and the zoom position is "3", a circle having a diameter of 0.8 with the center of the image sensor 16 as the reference becomes the preferred shooting range A.

[0082] The table data D1 is stored in, for example, the memory 48, and the information processing device 18 acquires the table data D1 from the memory 48.

[0083] Next, the information processing device 18 calculates the reflection range B of the object P (step S21).

[0084] The information processing device 18 acquires one or more pieces of reflection range information indicating the reflection range B of the object on the image sensor 16, based on the optical design information of the lens, the design information related to the image sensor 16, and information related to the size of the object. Here, the reflection range B is information indicating the range on the imaging surface 16a in which the object P is reflected. The reflection range B is expressed, for example, in terms of area, just like the shooting range A. Furthermore, the reflection range B may be expressed, for example, as the proportion of the object P that occupies the acquired image. It is preferable that the reflection range B and the shooting range A are expressed in the same format (same unit system).

[0085] FIG. 8 is a diagram showing table data indicating the range B in which the object P is captured on the image sensor 16. In FIG.

[0086] Each value in the table data D2 is calculated assuming that the object P has a width of 100 mm, a height of 100 mm, a depth of 100 mm, an image sensor horizontal dimension of 24 mm, and an image sensor vertical dimension of 16 mm. The zoom position is normalized so that the wide end is 1 and the telephoto end is 10, and the focus position is normalized so that INF is 1 and MOD is 10.

[0087] Each value of the reflected range B in the table data D2 is calculated as follows.

[0088] The focal length f (angle of view) is determined by determining the zoom position. Furthermore, the distance D from the image capture device 10 to the focal point (= distance to object P, shooting distance) is uniquely determined according to the zoom position and focus position, as shown in table data D3 in FIG. 9. Note that table data D3 indicates the distance to the focal point (shooting distance D) for combinations of focus position and zoom position. Furthermore, table data D3 is acquired from the internal memory of the lens device 12 as optical design information for the image capture device 10. Therefore, it is possible to calculate the size (horizontal Ih, vertical Iv) of object P captured on the image sensor 16. Here, for simplicity, calculation using a pinhole model yields the following equation:

[0089]

[0090] Ih and Iv are calculated using the above formulas, and the larger of Ih and Iv is I, the diagonal length of the image sensor 16 is DD, and the reflection range of the object is calculated as B=I / DD.

[0091] Next, the information processing device 18 calculates candidate combinations of focal length f and shooting distance D that satisfy the condition that the reflection range B≦shooting range A (B≦A...formula (1)) (step S22).

[0092] Based on the shooting range A (table data D1) and the reflection range B (table data D2), the information processing device 18 calculates candidate combinations of focal length f and shooting distance D such that the reflection range B is less than or equal to the shooting range A. Note that a combination that satisfies the relation of reflection range B less than or equal to the shooting range A means that an area with less optical degradation can be utilized. Furthermore, the shooting range A (table data D1) and the reflection range B (table data D2) are expressed in the same unit system.

[0093] FIG. 10 is a diagram showing table data indicating focus positions and zoom positions that satisfy the condition of the reflection range B≦the photographing range A. In FIG.

[0094] In table data D4, the value of the reflection range B in table data D2 ( FIG. 8 ) is entered for combinations of zoom position and focus position that satisfy reflection range B≦shooting range A, and an “x” is entered for combinations of zoom position and focus position that do not satisfy reflection range B≦shooting range A. In this way, in this example, multiple combinations of zoom position and focus position satisfy reflection range B≦shooting range A.

[0095] Next, the information processing device 18 determines the photographing conditions (step S23).

[0096] The information processing device 18 determines, as the shooting conditions, all or some of a plurality of combinations of zoom positions and focus positions that satisfy the above-mentioned reflection range B≦shooting range A. Note that the shooting conditions are configured from the combination of focus position and zoom position, shooting distance D and focal length f using table data D3.

[0097] A specific example of determining the shooting conditions from a plurality of combination candidates will be described below, but the aspects of the present invention are not limited to the following specific example.

[0098] (Example 1-1) In Example 1-1, the information processing device 18 determines, as the shooting conditions, the combination that results in the largest reflection range B among the candidate combinations that satisfy the condition that the reflection range B≦the shooting range A. When the reflection range B is largest, it is possible to capture the object P large in the image. In the case shown in FIG. 10 , the reflection range B is 0.65, which is the largest, when the zoom position is 8 and the focus position is 9. Therefore, the information processing device 18 determines the zoom position 8 and the focus position 9 as the shooting conditions. Note that the information processing device 18 may determine the shooting distance 450 at the zoom position 8 and the focus position 9 as the shooting conditions using table data D3.

[0099] By having the information processing device 18 determine the photographing conditions in this way, it is possible to acquire images for generating a high-quality three-dimensional model of the object P.

[0100] (Example 1-2) In Example 1-2, a minimum value C of the reflection range B that is permissible when generating a three-dimensional model is determined in advance. If the reflection range B of the object P is small, small irregularities of the object P may be lost when the three-dimensional model is generated, which may result in a lack of quality as a three-dimensional model. Therefore, it is necessary to capture an image within a range that captures the object P at a certain size. Therefore, the minimum value C is determined in advance from this perspective. Then, the information processing device 18 determines the reflection range B that satisfies the following formula (2) as the shooting condition. Minimum value C≦reflection range B≦shooting range A (C≦B≦A)... formula (2)

[0101] In this way, the information processing device 18 determines the photographing conditions, and it is possible to acquire images from which a realistic three-dimensional model having unevenness and the like can be generated.

[0102] (Example 1-3) In Example 1-3, the information processing device 18 determines all combinations that satisfy the condition of reflection range B≦capture range A as the shooting conditions. As described above, when the condition of reflection range B≦capture range A is satisfied, high-quality images can be acquired as images for generating a three-dimensional model. Therefore, the information processing device 18 determines multiple shooting conditions that satisfy the condition of reflection range B≦capture range A, allowing the user to select shooting conditions that meet the desired conditions.

[0103] As described above, the information processing device 18 can determine shooting conditions that cause less optical degradation, and can therefore acquire images that can be used to generate higher quality three-dimensional models.

[0104] Second Embodiment Next, a second embodiment of the present invention will be described. The information processing device 18 of this embodiment determines the shooting conditions by adding information about the depth of field to the information processing device 18 of the first embodiment. This makes it possible to shoot under conditions with a deep depth of field, and obtain images that can generate a higher quality 3D model. Note that the following description focuses on the depth of field, which is one specific example of information about the depth of field, but is not limited to this. For example, the information about the depth of field includes an F-number (aperture value).

[0105] FIG. 11 is a flow diagram illustrating the calculation of candidate combinations of focal length f and shooting distance D and the determination of shooting conditions performed by the information processing device 18 in this embodiment.

[0106] First, the information processing device 18 acquires the shooting range A (step S30). Next, the information processing device 18 calculates the reflection range B (step S31). Then, the information processing device 18 calculates candidate combinations of the focal length f and the shooting distance D such that the reflection range B is less than or equal to the shooting range A (step S32).

[0107] Thereafter, the information processing device 18 calculates the depth of field (step S33).

[0108] The information processing device 18 acquires the permissible circle of confusion diameter r as information 64 related to the imaging system. Then, the information processing device 18 fixes the F-number and calculates the depth of field for each combination of zoom position (focal length f) and focus position (shooting distance = subject distance) using the following formula. The shooting distance (subject distance) corresponding to the focus position is acquired based on table data D3.

[0109]

[0110] FIG. 12 is a diagram showing table data indicating the depth of field according to the focus position and the zoom position.

[0111] Table data D5 indicates depths of field corresponding to combinations of focus positions and zoom positions. The information processing device 18 acquires depths of field corresponding to combinations of focus positions and zoom positions as shown in table data D5. Note that the information processing device 18 may calculate the information in table data D5 after acquiring information about the imaging system (permissible circle of confusion diameter r), for example, or may acquire table data D5 that has been calculated and stored in advance.

[0112] Next, the information processing device 18 determines the photographing conditions (step S33).

[0113] The information processing device 18 determines as shooting conditions (first shooting conditions) all or some of a plurality of candidate combinations (second shooting conditions) of zoom positions and focus positions that satisfy the above-mentioned reflection range B≦shooting range A. Note that the shooting conditions are configured from the combination of focus position and zoom position, shooting distance D and focal length f using table data D3.

[0114] A specific example of determining the shooting conditions from a plurality of combination candidates will be described below, but the aspects of the present invention are not limited to the following specific example.

[0115] (Example 2-1) In Example 2-1, the information processing device 18 determines, as the shooting condition, the case where the depth of field is deepest among the candidate combinations (second shooting conditions) that satisfy the condition that the reflection range B is equal to or less than the shooting range A. In the case shown in table data D5, the combination of zoom position 10 and focus position 2 results in the deepest depth of field, so the shooting distance D for zoom position 10 and focus position 2 is determined as the shooting condition. Note that in table data D5, the case where focus position is 1 is not included as a candidate combination.

[0116] In this way, by having the information processing device 18 determine the shooting conditions, it is possible to perform shooting under conditions with the deepest depth of field, and to generate a high-quality three-dimensional model.

[0117] (Example 2-2) In Example 2-2, the information processing device 18 determines the shooting conditions (first shooting conditions) that will place the object P within the depth of field from among the candidate combinations (second shooting conditions) that satisfy the relation of the reflection range B≦the shooting range A. For example, the information processing device 18 determines the shooting conditions that will place the object P within the depth of field by setting the larger of the thickness T and width W of the object P to X.

[0118] In this way, by the information processing device 18 determining the photographing conditions, the object P can be photographed under conditions within the depth of field range, and a high-quality three-dimensional model can be generated.

[0119] (Example 2-3) In Example 2-3, the information processing device 18 calculates an allowable depth of field by expanding the depth of field (information regarding the depth of field) among candidate combinations (second shooting conditions) that satisfy the relation of the reflection range B≦the shooting range A, and determines shooting conditions (first shooting conditions) such that the object P falls within this allowable depth of field. The allowable depth of field is calculated, for example, by multiplying the depth of field by the allowable depth magnification. Here, for shoes of a standard size, the depth of field range is several centimeters, and the allowable magnification is approximately 20 times. The allowable depth magnification may be set by the user. The allowable depth of field may also be calculated by taking into account the amount of adjustment to the depth of field, i.e., the maximum magnification in front and / or behind the object P that can be considered in focus. The information processing device 18 determines shooting conditions such that the larger of the thickness T and width W of the object P is set to X, and X falls within the above-mentioned allowable depth of field range.

[0120] (Example 2-4) In Example 2-4, if there is no candidate combination of zoom position and focus position in Example 2-2 or Example 2-3, the information processing device 18 increases (narrows) the F-number, recalculates the depth of field, and acquires new table data D5. Then, if there is a candidate combination of zoom position and focus position in the new table data D5 that satisfies the conditions of Example 2-2 or Example 2-3, the information processing device 18 may present the changed F-number to the user. On the other hand, if there is no candidate combination of zoom position and focus position in the new table data D5 that satisfies the conditions of Example 2-2 or Example 2-3, the information processing device 18 further increases the F-number and similarly calculates table data D5. Note that if no candidate combination that satisfies the conditions of Example 2-2 or Example 2-3 is found even when the F-number is maximized, the information processing device 18 notifies the user of this and terminates the process.

[0121] (Others) In this embodiment, an example has been described in which a candidate combination that satisfies the depth of field condition is determined from among candidate combinations that satisfy the condition of the reflection range B≦the shooting range A. However, the present invention is not limited to this. For example, the combination of the zoom position and focus position that meets the condition of the depth of field in Examples 2-1 to 2-4 described above may be used as the shooting condition, without considering the condition of the reflection range B≦the shooting range A.

[0122] As described above, in this embodiment, it is possible to determine the conditions under which the object P falls within the depth of field. This allows a user who is not skilled in photography such as 3D scanning to capture images that can be used to generate a high-quality 3D model.

[0123] Third Embodiment Next, a third embodiment will be described.

[0124] 13 is a diagram illustrating 3D scanning of an object P by the image capturing device 10 according to the image capturing conditions determined by the information processing device 80 of this embodiment. The information processing device 80 of this embodiment is configured as a personal computer separate from the image capturing device 10. Note that, although this example describes an example in which the information processing device 80 is configured as a personal computer, the application of this embodiment is not limited to this. For example, the information processing device 80 may be configured as a smartphone, a terminal, or the like.

[0125] The information processing device 80 of this embodiment can communicate with the image capturing device 10 and transmit and receive necessary information. For example, the information processing device 80 acquires optical design information of the image capturing device 10 and information about the image capturing system from the image capturing device 10. Note that information about the characteristics of the object P is directly input to the information processing device 80 by the user.

[0126] Furthermore, the personal computer constituting the information processing device 80 may function as an information processing device by installing a dedicated application. In this case, the dedicated application may store a plurality of pieces of optical design information for the image capturing device 10 and information about the image capturing system, and the user may select the image capturing device 10 and lens device 12 they are using. The information processing device 80 then performs the above-described processing based on that information. Furthermore, if the lens device selected by the user is not suitable for acquiring images for generating a 3D model, recommended lens information may be presented to the user.

[0127] Although examples of the present invention have been described above, it goes without saying that the present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the present invention.

[0128] 1: Rotating table 10: Image capturing device 12: Lens device 16: Image sensor 18: Information processing device 22: Image input controller 24: Image processing unit 26: Compression / expansion processing unit 28: Video encoder 30: Image monitor 38: Operation unit 40: Control unit 47: Flash memory 48: Memory 52: Media controller 54: Memory card

Claims

1. An information processing device that includes a processor and determines first image capture conditions for generating a three-dimensional model of an object, wherein the processor acquires optical design information of the image capture device used for capture, information about the image capture system, and information about the characteristics of the object, and determines the first image capture conditions, including conditions related to the shooting distance for capturing the image of the object, based on the acquired information.

2. The information processing device described in claim 1, wherein the processor: acquires shooting range information indicating information about the shooting range of the object on the imaging element, which is set based on optical design information of the imaging device and information about the imaging system; acquires one or more pieces of reflection range information indicating the reflection range of the object on the imaging element based on the optical design information of the imaging device, information about the imaging system, and information about the characteristics of the object; and determines the first shooting condition based on the shooting range information and the one or more pieces of reflection range information.

3. The information processing device according to claim 2, wherein the processor determines the first shooting condition within a range that satisfies the following formula (1), where A denotes the shooting range information and B denotes one or more pieces of the reflection range information: B≦A... formula (1) 4. The information processing device according to claim 3, wherein the processor determines the maximum information on the range of reflection that satisfies the formula (1) as the first shooting condition.

5. The information processing device according to claim 3, wherein the processor determines the first shooting conditions within the range B that satisfies the following formula (2), where C is the minimum value of the reflection range information that is allowable when generating the three-dimensional model: C≦B≦A... formula (2) 6. The information processing device described in claim 3, wherein the processor acquires information regarding the depth of field, acquires a plurality of second shooting conditions that satisfy formula (1) based on the shooting range information and one or more pieces of reflection range information, and determines the first shooting condition from among the plurality of second shooting conditions based on the information regarding the depth of field.

7. The information processing device according to claim 6, wherein the processor determines the first shooting condition from among the plurality of second shooting conditions based on information about the deepest depth of field.

8. The information processing device described in claim 6, wherein the processor acquires an allowable depth of field by expanding information regarding the depth of field, and determines the first shooting condition from among the plurality of second shooting conditions based on the allowable depth of field.

9. The information processing device described in claim 6, wherein the processor, when unable to determine the first shooting condition from among the plurality of second shooting conditions based on the information regarding the depth of field, changes the aperture value to obtain information regarding the depth of field, and determines the first shooting condition from among the plurality of second shooting conditions based on the information regarding the depth of field.

10. The information processing device according to claim 1, wherein the processor acquires information about a depth of field, and determines the first shooting condition based on the information about the depth of field.

11. The information processing device according to claim 1, wherein the processor acquires information about the characteristics of the object through input from a user.

12. The information processing device according to claim 1, wherein the processor obtains information about the characteristics of the object from an image.

13. An imaging device equipped with the information processing device according to any one of claims 1 to 12.

14. An information processing method for determining first image capturing conditions for generating a three-dimensional model of an object, comprising the steps of: acquiring optical design information of the image capturing device used for capturing the image, information about the image capturing system, and information about the characteristics of the object; and determining the first image capturing conditions, including conditions related to the shooting distance for capturing the image of the object, based on the acquired information.

15. An information processing method as described in claim 14, further comprising a step performed by the processor of acquiring information relating to depth of field, wherein in the step of determining the first shooting condition, a plurality of second shooting conditions are determined, and the first shooting condition is selected from the plurality of second shooting conditions based on the information relating to depth of field.

16. A program that causes a computer to execute an information processing method for determining first photographing conditions for an image to generate a three-dimensional model of an object, the program causing a processor to execute the following steps: acquiring optical design information of the photographing device used for photographing, information about the photographing system, and information about the characteristics of the object; and determining the first photographing conditions, including conditions related to the photographing distance for photographing the image of the object, based on the acquired information.

17. A non-transitory computer-readable recording medium on which the program according to claim 16 is recorded.

Images