Image capturing system, image capturing method, and program product

Abstract

Disclosed are an image capturing system, an image capturing method, and a program product, the image capturing system being provided with: a portable imaging device having an illumination unit, a camera, and a measurement unit; the measuring unit is capable of measuring the distance between the imaging device and an object to be imaged and the shape of the object to be imaged, and the processor measures the distance between the imaging device and the object to be imaged and the shape of the object to be imaged. The imaging device assists a user in imaging a specular reflection image when the imaging device moves to a position and direction at which a partial region of the surface of the imaging object, the distance and shape of which have been measured, becomes a specular reflection condition when the imaging object is imaged by the imaging device.

Description

Technical Field

[0001] This invention relates to an image photography system, image photography method, and program product. Background Technology

[0002] Japanese Patent Application Publication No. 2005-050015 discloses a method for reading glossiness. This method involves moving a light source and a camera unit that captures images of a subject using at least one channel relative to each other while capturing multiple images of the subject using the camera unit. The method compares the channel values ​​of each pixel in the multiple images to determine the maximum value of each channel and creates a maximum value image representing the maximum channel value of each pixel. This allows the method to read the specular reflection intensity of the entire surface of the subject, even when the subject is a large curved surface.

[0003] Japanese Patent Application Publication No. 2019-082838 discloses a display system that, compared to acquiring multiple image data from multiple angles using cameras and using these image data to display the texture of an object's surface, can display the texture of a large area using simple calculations with less image data.

[0004] Japanese Patent Application Publication No. 2022-047371 discloses an inspection apparatus that acquires image information representing continuously captured photographic images taken by moving a scanning unit equipped with a camera, a laser sensor, a white light source, and a non-visible laser light source relative to an object. It extracts area images corresponding to a range predetermined for the specular reflection angle from the image information, synthesizes the extracted area images, and generates a defect detection image for detecting defects in the object. Thus, compared with the case of capturing orthographic reflection images, it is possible to obtain an image suitable for inspecting defects on the surface of the object. Summary of the Invention

[0005] A 3D model of a photographic object is generated using images taken from various angles. To reproduce the object's texture in this 3D model, it's sometimes desirable to include its glossiness. Furthermore, to generate a 3D model that includes glossiness, it's necessary to capture specular reflection images of the entire surface of the object without omission. Here, a specular reflection image refers to an image captured when light from illumination is reflected off a portion of the object, and the orthogonal reflected light component directly enters the camera. However, when the object's surface shape includes complex shapes such as curved surfaces, it's difficult to determine which position to capture the object under specular reflection conditions.

[0006] The purpose of this invention is to provide an image photography system, image photography method, and program product that can capture a specular reflection image of the entire surface of a photographic object when photographing the object using a photographic device with illumination.

[0007] According to a first aspect of the present invention, an image capturing system is provided, comprising: a portable photographic device having an illumination unit, a camera, and a measuring unit capable of measuring the distance between the photographic device and a photographic object and the shape of the photographic object; and a processor that measures the distance between the photographic device and the photographic object and the shape of the photographic object, and assists a user in capturing a specular reflection image when the photographic device moves toward a position and direction that becomes a specular reflection condition when the photographic device is used to photograph a portion of the surface of the photographic object whose distance and shape have been measured.

[0008] According to a second aspect of the invention, in the image capturing system of the first aspect, the processor informs the user of the position and orientation of the photographic device as a condition in which a portion of the surface of the photographic object becomes specularly reflective when the object is photographed, thereby assisting the user in capturing specularly reflective images.

[0009] According to a third aspect of the invention, in the image capturing system of the first aspect, the processor, when the position and orientation of the photographic device are such that a portion of the surface of the photographic object becomes specularly reflective when the object is photographed, captures the object using the photographic device without accepting user input, thereby assisting the user in capturing a specularly reflective image.

[0010] According to a fourth aspect of the invention, in the image capturing system of the first aspect, the processor divides the surface of the photographic object with a specific shape into multiple regions, and displays in different display modes the regions in which images are acquired at positions and directions that are specular reflection conditions and the regions in which images are not acquired at positions and directions that are specular reflection conditions, thereby assisting the user in capturing specular reflection images.

[0011] According to a fifth aspect of the invention, in the image capturing system of the fourth aspect, the processor, when the uncaptured area is selected, performs a guidance display, which guides the image capture of the uncaptured area to a position and direction that can be captured under specular reflection conditions, relative to the current position and direction of the image capturing device determined based on information about the distance between the image capturing device and the object being captured as measured by the measuring unit.

[0012] According to a sixth aspect of the invention, in the image imaging system according to any one of the first to fifth aspects, the more complex the surface shape of the photographic object, the higher the required precision of the captured specular reflection image, or the narrower the illumination area, the smaller the area that divides the surface of the photographic object into.

[0013] According to a seventh aspect of the present invention, in the image capturing system according to any one of the first to sixth aspects, the measuring unit is a sensor capable of measuring the distance between multiple points of the photographic device and the photographic object.

[0014] According to an eighth aspect of the invention, in the image capture system of the seventh aspect, the sensor is a LiDAR sensor capable of measuring the distance between each pixel captured by the camera and the photographic object.

[0015] According to a ninth aspect of the present invention, in the image imaging system according to any one of the first to eighth aspects, the processor calculates the position and orientation of the imaging device capable of capturing the area under specular reflection conditions using information about the positional and angular relationship between the illumination and the camera in the imaging device, and the position of the area to be captured under specular reflection conditions in the surface shape of the object to be photographed, which is specific to the object being photographed.

[0016] According to a tenth aspect of the present invention, a program product is provided, comprising a program for executing processing on a computer, wherein the processing includes the steps of: measuring the distance between the camera and the object being photographed and the shape of the object being photographed using a portable photographic device having a lighting, a camera, and a measuring unit, the measuring unit being capable of measuring the distance between the camera and the object being photographed and the shape of the object being photographed; and assisting a user in capturing a specular reflection image when the camera moves toward a position and direction that becomes a specular reflection condition when the object being photographed using the camera.

[0017] According to an eleventh aspect of the present invention, an image photography method is provided, comprising the steps of: measuring the distance between the photography device and the photographic object and the shape of the photographic object using a portable photography device having an illumination, a camera, and a measuring unit, wherein the measuring unit is capable of measuring the distance between the photography device and the photographic object and the shape of the photographic object; and assisting a user in capturing a specular reflection image when the photography device moves toward a position and direction that becomes a specular reflection condition when the photographic object is photographed using the photography device, provided that the distance and shape of the photographic object have been measured.

[0018] (Effect)

[0019] According to the first and eleventh aspects, when photographing a photographic object using a photographic device with illumination, it is possible to capture a specular reflection image of the entire surface of the photographic object.

[0020] According to the second aspect, the user can determine the position and orientation of the photographic device and the position and orientation of a part of the surface of the photographed object that becomes a specular reflection condition simply by moving the photographic device.

[0021] According to the third aspect, the user can automatically capture a specular reflection image of a portion of the surface of the photographic object simply by moving the photographic device.

[0022] According to the fourth aspect, users can identify areas where images were not acquired in locations and directions that are subject to specular reflection.

[0023] According to the fifth aspect, the user can move the camera device to the position and direction for capturing the selected unacquired area based on the specular reflection conditions simply by following the guidance display.

[0024] According to the sixth aspect, compared to the case where the size of the area segmented on the surface of the photographic object is constant, it is possible to make the accuracy of the captured specular reflection image approach the required accuracy.

[0025] According to the seventh aspect, when photographing a photographic object using a photographic device with illumination, it is possible to capture a specular reflection image of the entire surface of the photographic object without omission.

[0026] According to the eighth aspect, when photographing a photographic object using a photographic device with illumination, it is possible to capture a specular reflection image of the entire surface of the photographic object without omission.

[0027] According to the ninth aspect, by simply setting the positional and angular relationship between the lighting and the camera, it is possible to calculate the position and orientation of the photographic device capable of capturing images of a certain area under specular reflection conditions.

[0028] According to the tenth aspect, when photographing a photographic object using a photographic device with illumination, it is possible to capture a specular reflection image of the entire surface of the photographic object. Attached Figure Description

[0029] Figure 1 This figure shows a situation where a portable terminal device 10, which is an image photography system according to an embodiment of the present invention, takes an image of the photographic object 50.

[0030] Figure 2 This is a block diagram illustrating the hardware structure of a portable terminal device 10 according to an embodiment of the present invention.

[0031] Figure 3 This is a block diagram illustrating the functional structure of a portable terminal device 10 according to an embodiment of the present invention.

[0032] Figure 4 This is a flowchart illustrating the actions of capturing a specular reflection image of the entire surface of a photographic object 50 using a portable terminal device 10 according to an embodiment of the present invention.

[0033] Figure 5 This is a diagram illustrating the calculation method for the position and angle of a portable terminal device 10 that is subject to specular reflection conditions relative to a certain block;

[0034] Figure 6 This diagram illustrates the different positions and angles of the portable terminal device 10 relative to each of the blocks that become mirror reflection conditions.

[0035] Figure 7 This is an example of a specular reflection image taken when a certain block of the photographic object 50 is under specular reflection conditions;

[0036] Figure 8 This diagram illustrates an example of displaying blocks for which no specular reflection image has been captured and blocks for which a specular reflection image has been captured using different display colors.

[0037] Figure 9 This diagram illustrates the scenario where a block within a block on the touch panel is selected that has not yet acquired a specular reflection image.

[0038] Figure 10 This is a diagram of Example 1 showing a guide display that moves the portable terminal device 10 to a position where it can capture the selected block under specular reflection conditions;

[0039] Figure 11 This is a diagram of Example 2 showing a guide display that moves the portable terminal device 10 to a position where it can capture the selected block under specular reflection conditions;

[0040] Figure 12 This is a diagram of Example 3 showing a guide display that moves the portable terminal device 10 to a position where it can capture the selected block under specular reflection conditions;

[0041] Figure 13 This figure shows an example of a portable terminal device 10 with external lighting;

[0042] Figure 14 This is another example of a portable terminal device 10 with external lighting;

[0043] Figure 15 This is another example of a portable terminal device 10 with external lighting;

[0044] Figure 16 This is another example of a portable terminal device 10 with external lighting. Detailed Implementation

[0045] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0046] An example of capturing an image of a photographic object 50 using a portable terminal device 10, which is an image capturing system according to an embodiment of the present invention, is shown below. Figure 1 .

[0047] Here, images taken from various angles of the photographic subject are used to generate a 3D model of the subject. To reproduce the texture of the subject in this 3D model, it is sometimes desirable to include the subject's glossiness. Furthermore, to generate a 3D model that includes the glossiness, it is necessary to capture specular reflection images of the entire surface of the subject without omission. Here, a specular reflection image refers to an image captured when light from illumination is reflected from a portion of the subject, and the orthogonal reflected light component directly enters the camera. Moreover, a specular reflection image refers to an image where a specific area of ​​the subject is subject to specular reflection. It does not mean that the entire area of ​​the subject is subject to specular reflection. That is, a single specular reflection image includes both the area subject to specular reflection and the area not subject to specular reflection.

[0048] Furthermore, in this embodiment, the photographic object is as follows: Figure 1The case shown is a photographic object 50 with a three-dimensional shape, including curved surfaces. However, photographic objects are not limited to three-dimensional objects; they also include objects that are nearly planar. Furthermore, the present invention is not only applicable to cases where an entire object is used as the photographic object, but also to cases where only a part of an object is used. For example, the present invention is also applicable to cases where only the tabletop of a table is used as the photographic object, rather than the entire table.

[0049] In the image capture system of this embodiment, in order to generate such Figure 1 The three-dimensional model of the photographic object 50 shown is photographed from various angles. Furthermore, in order to include the glossiness of the photographic object 50 in the generated three-dimensional model, specular reflection images of the entire surface of the photographic object 50 are captured without omission. However, in order to generate a model like... Figure 1 In the case of the three-dimensional model of the photographic object 50 shown, since the surface shape of the photographic object 50 includes curved surfaces, it is difficult to determine from which position the photographic object 50 can be photographed under specular reflection conditions.

[0050] Therefore, in the image photography system of this embodiment, by performing the processing described below, when photographing a photographic object using a portable terminal device 10 with illumination, it is possible to capture a specular reflection image of the entire surface of the photographic object 50.

[0051] Here, the portable terminal device 10 is implemented, for example, by a smartphone or tablet terminal device. The portable terminal device 10 is a portable photographic device having illumination, a camera, and a measuring unit capable of measuring the distance between the device and the photographed object, as well as the shape of the photographed object. Furthermore, the measuring unit is a sensor capable of measuring the distance between multiple points of the portable terminal device 10 and the photographed object. Specifically, this sensor is a LiDAR (Light Detection and Ranging) sensor capable of measuring the distance between each pixel captured by the camera and the photographed object. While it has been described that the LiDAR sensor can measure the distance between the portable terminal device 10 and the photographed object, in reality, it measures the distance between the LiDAR sensor and the photographed object.

[0052] Furthermore, while it has been explained that LiDAR sensors can measure the shape of the object being photographed, the measurement of the object's shape based on information about its distance from the object is also included within the scope of measuring both the distance to the object and the object's shape. Alternatively, both the distance to the object and the object's shape can be measured separately.

[0053] Next, the hardware structure of the portable terminal device 10 constituting the image capturing system of this embodiment will be shown. Figure 2 .

[0054] like Figure 2 As shown, the portable terminal device 10 includes a CPU 11, a memory 12, a storage device such as flash memory 13, a communication interface (hereinafter referred to as IF) for sending and receiving data with external devices via a wireless communication line 14, a user interface (hereinafter referred to as UI) device including a touch panel or liquid crystal display 15, a camera 16, a LiDAR sensor 17, and a light source 18.

[0055] CPU 11 is a processor that controls the operation of portable terminal device 10 by executing predetermined processes based on a control program stored in memory 12 or storage device 13. Furthermore, in this embodiment, the case where CPU 11 reads and executes the control program stored in memory 12 or storage device 13 has been described, but this is not a limitation. The control program may also be provided in a computer-readable recording medium. For example, it may be provided on optical discs such as CD (CompactDisc)-ROM and DVD (Digital Versatile Disc)-ROM, or on semiconductor memory such as USB (Universal Serial Bus) memory and memory cards. Alternatively, the control program may be obtained from an external device via a communication line connected to communication interface 14. Furthermore, the control program may be provided as separate application software, or it may be incorporated into the software of the portable terminal device 10 as a function of the device.

[0056] Figure 3 This is a block diagram illustrating the functional structure of the portable terminal device 10 achieved by executing the aforementioned control program.

[0057] like Figure 3 As shown, the portable terminal device 10 of this embodiment includes an operation input unit 31, a display unit 32, a control unit 33, a data storage unit 34, a camera 16, a LiDAR sensor 17, and a light source 18. Furthermore, in Figure 3 In this embodiment, functional structures such as communication functions that are not related to the technology of this invention are omitted. Furthermore, in this embodiment, the camera 16 and the LiDAR sensor 17 are described as having different structures, but they can also be used as a single unit, incorporating a LiDAR-equipped camera.

[0058] The display unit 32 is controlled by the control unit 33 and displays various information to the user. The operation input unit 31 inputs various operation information performed by the user. In addition, the operation input unit 31 and the display unit 32 constitute a touch panel.

[0059] The control unit 33 controls the operation of the camera 16, LiDAR sensor 17, and illumination 18 to capture images of the photographed object and store them in the data storage unit 34. Additionally, the control unit 33 receives operation information from the operation input unit 31 and displays various information on the display unit 32.

[0060] Furthermore, when capturing a specular reflection image of the photographic object, the control unit 33 first measures the distance between the portable terminal device 10 and the photographic object, as well as the shape of the photographic object. Then, when the portable terminal device 10 moves to a portion of the surface of the photographic object, to which the measured distance and shape have been determined, in a position and direction that would create a specular reflection condition when the portable terminal device 10 is used to capture the photographic object, the control unit 33 performs processing to assist the user in capturing a specular reflection image.

[0061] For example, the control unit 33 notifies the user of the position and orientation of the portable terminal device 10 when a portion of the surface of the photographed object becomes a specular reflection condition during the photographing of the object, thereby assisting the user in taking a specular reflection image.

[0062] In addition, when the position and orientation of the portable terminal device 10 become a condition where a part of the surface of the photographed object becomes a specular reflection when photographing the object, the control unit 33 does not accept user operation but uses the camera 16 of the portable terminal device 10 to photograph the object, thereby assisting the user in taking a specular reflection image.

[0063] Furthermore, the control unit 33 divides the surface of the photographic object with a specific shape into multiple regions and displays the regions in different display modes, including the regions where images are acquired at positions and directions that are specular reflection conditions and the regions where images are not acquired at positions and directions that are specular reflection conditions, thereby assisting the user in capturing specular reflection images.

[0064] At this time, the control unit 33 can also provide guidance display when the user selects an area that has not been captured. This guidance display is used to guide the user to a position and direction in which the area that has not been captured can be captured under conditions of specular reflection, relative to the current position and direction of the portable terminal device 10 determined based on the information of the distance between the portable terminal device 10 and the object to be photographed as measured by the LiDAR sensor 17.

[0065] Here, regarding the control unit 33, the more complex the surface shape of the photographed object, the higher the required precision of the captured specular reflection image; or, the narrower the illumination area of ​​the illumination 18, the smaller the area that divides the surface of the photographed object into.

[0066] In addition, the control unit 33 uses the positional and angular relationship between the illumination 18 and the camera 16 in the portable terminal device 10, and the information on the position of the area to be photographed under specular reflection conditions in the surface shape of a specific photographic object, to calculate the position and orientation of the portable terminal device 10 that can photograph the area under specular reflection conditions.

[0067] Next, refer to Figure 4 Detailed description of the portable terminal device 10 pairs using this embodiment Figure 1 The action of taking a mirror image of the entire surface of the photographic object 50 shown.

[0068] First, in step S101, the control unit 33 acquires information about the relative position / angle of the camera 16 and the illumination 18, as well as the size of the illumination 18. Specifically, the control unit 33 may also acquire this information from the user via the operation input unit 31. Alternatively, the control unit 33 may use the portable terminal device 10 to photograph the mirror surface and acquire information such as the positional relationship between the camera 16 and the illumination 18 in this device. Furthermore, when using the camera 16 and illumination 18 provided by the portable terminal device 10, the control unit 33 may also use pre-set information to acquire information such as the positional relationship between the camera 16 and the illumination 18.

[0069] Next, in step S102, the control unit 33 uses the LiDAR sensor 17 to measure the shape of the entire circumference of the photographic object 50.

[0070] Then, in step S103, the control unit 33 divides the surface of the photographed object 50 into multiple regions, i.e. multiple blocks, and calculates the position and angle of the portable terminal device 10 under specular reflection conditions for each block.

[0071] Here, refer to Figure 5 This document describes the method for calculating the position and angle of a portable terminal device 10 when a certain block becomes a specular reflection.

[0072] like Figure 5 As shown, a certain block becomes a specular reflection condition when light from illumination 18 is reflected from a certain block of the photographed object 50, and the orthogonal reflected light component directly enters the camera 16 in such a positional relationship. That is, as... Figure 6 As shown, when the surface shape of the photographed object 50 is curved, the position and angle of the portable terminal device 10 that becomes a mirror reflection condition are different for each piece.

[0073] Furthermore, an example of a specular reflection image taken under conditions where a certain block of the photographic object 50 is in specular reflection is shown below. Figure 7 . Reference Figure 7 It is understood that a specular reflection image is an image in which light reflected orthogonally from a specific portion of the photographic object 50 is incident on the camera 16. That is, a specular reflection image refers to an image where a specific area of ​​the photographic object emits light under specular reflection conditions, not an image where the entire photographic object emits light. Thus, a single specular reflection image includes both the area under specular reflection conditions and the area not under specular reflection conditions. Furthermore, by capturing such a specular reflection image of a portion of the photographic object emitting light on its entire surface and generating a 3D model, a 3D model with a glossy appearance can be generated.

[0074] Next, the control unit 33 instructs the user to take pictures around the photographic object 50, and in step S104, it takes pictures of the photographic object 50.

[0075] Then, in step S105, the control unit 33 determines whether the specular reflection images of all blocks have been captured.

[0076] In step S105, if it is determined that the specular reflection images of all blocks have been captured, the control unit 33 ends the processing.

[0077] However, in step S105, if it is determined that not all blocks have been captured with specular reflection images, the control unit 33 performs a process to assist in capturing images of the blocks for which no specular reflection images were acquired under specular reflection conditions. For example, the control unit 33 displays the blocks for which no specular reflection images were acquired and the blocks for which specular reflection images were acquired using different display methods, such as changing the display color.

[0078] Thus, an example is shown where blocks for which no specular reflection image has been acquired are displayed in different display colors, and blocks for which a specular reflection image has been acquired. Figure 8 . Reference Figure 8 Blocks with captured specular reflection images are displayed with black diagonal lines, while blocks without captured specular reflection images are displayed in white. By viewing this display, users can determine the location of blocks without captured specular reflection images.

[0079] Then, in step S106, the control unit 33 provides guidance to the portable terminal device 10, such as adjusting the position and angle for capturing images of mirror reflections that have not been obtained under specular reflection conditions. For example, as Figure 9As shown, a user selects a block from the blocks where no specular reflection image has been captured on the touch panel. The control unit 33 then performs a guiding display to move the portable terminal device 10 to a position where a picture of the block selected by the user can be captured under specular reflection conditions.

[0080] A specific example of this guided display is shown in Figures 10-12 .exist Figure 10 In the middle, a guide display is provided using arrows to indicate the direction of movement and rotation of the mobile terminal device 10, and using the size of the arrows to indicate the amount of movement and rotation. Additionally, in Figure 11 In this design, the arrow in the center of the touch panel indicates the direction of movement of the portable terminal device 10, and the arrows at the ends of the touch panel indicate the direction of rotation for rotating the portable terminal device 10. Furthermore, in... Figure 11 In the display, the amount of movement and rotation are shown as numerical values.

[0081] Furthermore, in Figure 12 The example shown illustrates a guided display where a virtual mobile terminal device 10 is displayed using AR (Augmented Reality) to indicate the position and angle at which the mobile terminal device 10 on the touch panel should be moved. By moving the portable terminal device 10 in a manner that overlaps with the position of the AR-displayed portable terminal device 10, the user can move the portable terminal device 10 to a position where the selected block can be photographed under specular reflection conditions.

[0082] Alternatively, the control unit 33 can notify the user of the position and orientation of the portable terminal device 10 by having the user take another picture of the entire circumference of the photographed object 50 and issuing a notification tone to the user, thus enabling the user to capture the position and orientation of the block that did not obtain a specular reflection image under specular reflection conditions, thereby assisting the user in capturing specular reflection images.

[0083] In addition, the control unit 33 can also automatically photograph the object 50 using the camera 16 when the position and orientation of the portable terminal device 10 are such that the block that has not been photographed under specular reflection conditions is in a position and orientation that can be photographed. This assists the user in taking specular reflection images.

[0084] Then, through the process described above, the photographing of the photographic object 50 is repeated. In step S105, when it is determined that the specular reflection images of all blocks have been photographed, the control unit 33 ends the process.

[0085] In this way, by following the instructions to photograph the object 50, the user can capture a mirror reflection image of the entire surface of the object 50 without missing anything.

[0086] Furthermore, in the embodiment described above, the case of using the illumination 18 built into the portable terminal device 10 to illuminate the photographic object 50 and capture an image has been described. However, it is also possible to configure the structure such that the illumination is externally provided outside the portable terminal device 10 to capture an image of the photographic object 50.

[0087] A specific example of how to externally illuminate a portable terminal device 10 is shown below. Figures 13-16 .

[0088] Figure 13 This diagram shows a state where a spherical illumination 51 is externally mounted on the portable terminal device 10. Additionally, Figure 14 This diagram shows a configuration where a wide strip of light 52 is externally mounted on the portable terminal device 10. Furthermore, Figure 15 This diagram shows the state in which two strip lights 53 are externally mounted on the left and right sides of the portable terminal device 10. Finally, Figure 16 This diagram shows a state where an external ring light 54 is installed on the portable terminal device 10. In this way, even when an external light is installed on the portable terminal device 10, the processing described above can be performed as long as the relative position / angle between the camera 16 and the light, as well as the size of the light, are preset.

[0089] As described above, by externally illuminating the portable terminal device 10, the illumination area can be increased, and the size of the blocks when dividing the surface of the photographic object can be increased. As a result, the time required to capture a specular reflection image of the entire surface of the photographic object is shortened, enabling efficient image capture.

[0090] In the above embodiments, processor refers to processor in a broad sense, including general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and special-purpose processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, Programmable Logic Device, etc.).

[0091] Furthermore, the actions of the processors in the above embodiments are not performed by a single processor, but can also be accomplished collaboratively by multiple processors located in physically separate positions. Additionally, the order of the processor's actions is not limited to the order described in the above embodiments and can be appropriately modified.

[0092] In this embodiment, "system" includes both systems consisting of multiple devices and systems consisting of a single device.

[0093] The technology of this invention can also be applied to programs and program products.

[0094] [Postscript] (((1)))

[0096] An image capture system, comprising:

[0097] A portable photographic device, comprising an illumination unit, a camera, and a measuring unit capable of measuring the distance between the photographic device and the photographic object, and the shape of the photographic object; and

[0098] processor,

[0099] The processor measures the distance between the photographic device and the photographic object, as well as the shape of the photographic object.

[0100] When the photographic device moves to a position and direction that becomes a specular reflection condition when photographing the object, based on the measured distance and shape, the user is assisted in capturing a specular reflection image. (((2)))

[0102] According to the image capture system described in (((1))), wherein,

[0103] The processor informs the user of the position and orientation of the photographic device when a portion of the surface of the photographic object becomes a specular reflection condition during the photographing of the object, thereby assisting the user in capturing specular reflection images. (((3)))

[0105] According to the image capture system described in (((1))), wherein,

[0106] Regarding the processor, when the position and orientation of the photographic device are such that a portion of the surface of the photographic object becomes subject to specular reflection when photographing the object, the processor does not accept user operation but instead uses the photographic device to photograph the object, thereby assisting the user in capturing a specular reflection image. (((4)))

[0108] According to the image capture system described in (((1))), wherein,

[0109] The processor divides the surface of the photographic object of a specific shape into multiple regions and displays the regions in different display modes, including the regions where images are acquired at positions and directions that are specular reflection conditions and the regions where images are not acquired at positions and directions that are specular reflection conditions, thereby assisting the user in capturing specular reflection images. (((5)))

[0111] According to the image capture system described in (4), wherein,

[0112] Regarding the processor, when the unacquired area is selected, a guidance display is performed. The guidance display is used to guide the viewer to a position and direction where the unacquired area can be captured under specular reflection conditions, relative to the current position and direction of the photographic device determined based on information about the distance between the photographic device and the photographic object measured by the measuring unit. (((6)))

[0114] The image capturing system according to any one of ((1))) to ((5))) wherein,

[0115] Regarding the processor, the more complex the surface shape of the photographed object, the higher the required precision of the captured specular reflection image; or, the narrower the illumination area, the smaller the area that divides the surface of the photographed object into. (((7)))

[0117] The image capturing system according to any one of ((1))) to ((6))) wherein,

[0118] The measuring unit is a sensor capable of measuring the distance between multiple points of the photographic device and the photographic object. (((8)))

[0120] According to the image capture system described in ((7)), wherein,

[0121] The sensor is a LiDAR sensor capable of measuring the distance between each pixel captured by the camera and the photographed object. ((9)))

[0123] The image capturing system according to any one of ((1))) to ((8))) wherein,

[0124] The processor calculates the position and orientation of the camera device capable of capturing images of the area under specular reflection conditions, using the positional and angular relationships between the lighting and the camera in the photographic device, and information about the location of the area to be photographed under specular reflection conditions in the surface shape of the object to be photographed. (((10)))

[0126] A program product comprising a program that causes a computer to perform a process, wherein the process includes the following steps:

[0127] A portable photographic device equipped with lighting, a camera, and a measuring unit is used to measure the distance between the photographic device and the photographic object, as well as the shape of the photographic object. The measuring unit is capable of measuring both the distance between the photographic device and the photographic object, and the shape of the photographic object.

[0128] When the photographic device moves to a position and direction that becomes a specular reflection condition when photographing the object, based on the measured distance and shape, the user is assisted in capturing a specular reflection image.

[0129] According to the image photography system (((1)), when photographing a photographic object using a photographic device with illumination, it is possible to capture a specular reflection image of the entire surface of the photographic object.

[0130] According to the image photography system ((2)), the user can grasp the position and orientation of the photography device by simply moving the photography device, and the position and orientation of a part of the surface of the photographed object as a mirror reflection condition.

[0131] According to the image photography system of ((3)), the user can automatically capture a specular reflection image of a portion of the surface of the photographic object simply by moving the photography device.

[0132] According to the image capture system ((4)), the user can identify the areas where no image was captured in the location and direction that became a specular reflection condition.

[0133] According to the image photography system ((5)), the user can move the camera device to the position and direction of the selected unacquired area by simply following the guidance display.

[0134] According to the image photography system ((6)), compared with the case where the size of the area of ​​the surface of the photographed object is constant, it is possible to make the accuracy of the captured specular reflection image close to the required accuracy.

[0135] According to the image photography system of ((7)), when photographing a photographic object using a photographic device with illumination, it is possible to capture a specular reflection image of the entire surface of the photographic object without omission.

[0136] According to the image photography system of ((8)), when photographing a photographic object using a photographic device with illumination, it is possible to capture a specular reflection image of the entire surface of the photographic object without omission.

[0137] According to the image photography system ((9)), the position and orientation of the photographic device that can photograph a certain area under specular reflection conditions can be calculated simply by setting the positional and angular relationship between the lighting and the camera.

[0138] According to the program product (((10))), when photographing a photographic object using a photographic device with illumination, it is possible to capture a specular reflection image of the entire surface of the photographic object.

Claims

1. An image capturing system, characterized in that, have: A portable photographic device, comprising an illumination unit, a camera, and a measuring unit capable of measuring the distance between the photographic device and the photographic object, as well as the shape of the photographic object; as well as processor, The processor measures the distance between the photographic device and the photographic object, as well as the shape of the photographic object. When the photographic device moves to a position and direction that becomes a specular reflection condition when photographing the object, based on the measured distance and shape, the user is assisted in capturing a specular reflection image.

2. The image capturing system according to claim 1, wherein, Regarding the processor, The position and orientation of the photographic device are communicated to the user when a portion of the surface of the photographic object becomes a specular reflection condition during the photographing of the object, thereby assisting the user in capturing specular reflection images.

3. The image capturing system according to claim 1, wherein, Regarding the processor, When the position and orientation of the photographic device are such that a portion of the surface of the photographic object becomes a specular reflection condition when photographing the object, the photographic device is used to photograph the object without user operation, thereby assisting the user in capturing a specular reflection image.

4. The image capturing system according to claim 1, wherein, Regarding the processor, The surface of the photographic object with a specific shape is divided into multiple regions, and the regions in which images are acquired at positions and directions that are conducive to specular reflection are displayed in different ways, as well as the regions in which images are not acquired at positions and directions that are conducive to specular reflection, thereby assisting the user in capturing specular reflection images.

5. The image capturing system according to claim 4, wherein, Regarding the processor, When the unacquired area is selected, a guidance display is provided. This guidance display is used to guide the viewer to a position and direction where the unacquired area can be captured under specular reflection conditions, relative to the current position and direction of the photographic device determined based on information about the distance between the photographic device and the photographic object measured by the measuring unit.

6. The image capturing system according to any one of claims 1 to 5, wherein, Regarding the processor, The more complex the surface shape of the photographed object, the higher the required precision of the captured specular reflection image; or, the narrower the illumination area, the smaller the area that the photographed object's surface is divided into.

7. The image capturing system according to any one of claims 1 to 6, wherein, The measuring unit is a sensor capable of measuring the distance between multiple points of the photographic device and the photographic object.

8. The image capturing system according to claim 7, wherein, The sensor is a LiDAR sensor capable of measuring the distance between each pixel captured by the camera and the photographed object.

9. The image capturing system according to any one of claims 1 to 8, wherein, Regarding the processor, Using the positional and angular relationships between the lighting and the camera in the photographic device, and information about the location of the area to be photographed under specular reflection conditions in the surface shape of the specific photographic object, the position and orientation of the photographic device capable of photographing the area under specular reflection conditions are calculated.

10. A program product comprising a program that causes a computer to perform processing, characterized in that, The process comprises the following steps: A portable photographic device equipped with lighting, a camera, and a measuring unit is used to measure the distance between the photographic device and the photographic object, as well as the shape of the photographic object. The measuring unit is capable of measuring the distance between the photographic device and the photographic object, as well as the shape of the photographic object. as well as When the photographic device moves to a position and direction that becomes a specular reflection condition when photographing the object, based on the measured distance and shape, the user is assisted in capturing a specular reflection image.

11. An image capturing method, characterized in that, It has the following steps: A portable photographic device equipped with lighting, a camera, and a measuring unit is used to measure the distance between the photographic device and the photographic object, as well as the shape of the photographic object. The measuring unit is capable of measuring both the distance between the photographic device and the photographic object, and the shape of the photographic object. When the photographic device moves to a position and direction that becomes a specular reflection condition when photographing the object, based on the measured distance and shape, the user is assisted in capturing a specular reflection image.

Images