Scanning system, method, and program

Abstract

Provided are a scanning system, method, and program for detecting at least one of a transmittance and a transmission color of a region of an object that is at least partially transparent. The scanning system comprises: an illuminating device (20) for emitting illuminating light onto an object (50); a first imaging device (10) for imaging the object (50) illuminated by the illumination device (20); and a processor. The processor acquires, from the first imaging device (10), a plurality of first captured images having different imaging directions with respect to the object (50), and detects at least one of a transmittance and a transmission color of a region of the object (50) through which the illumination light passes, on the basis of the plurality of first captured images.

Description

Scanning system, method and program

[0001] The present invention relates to a scanning system, method and program, and more particularly to a technique for scanning an object that is at least partially transparent.

[0002] Conventionally, a method for measuring the height (height profile) of a substantially transparent object having a refractive index, such as a pellicle or a coating, has been proposed (Patent Document 1).

[0003] The method described in Patent Document 1 is based on high-speed Moiré interferometry, which acquires an image of an object corresponding to an intensity pattern projected onto a pellicle, and uses the acquired image to measure the height of the object from a reference surface using the phase of the intensity pattern associated with the object, the refractive index of the object, and a reference phase of the intensity pattern corresponding to the reference surface.

[0004] Furthermore, Patent Document 2 describes a three-dimensional position measurement system having a plurality of photographing means for photographing an object from different directions, and a processing means for acquiring images photographed by the plurality of photographing means and measuring the three-dimensional position of a measurement target point by processing the images.

[0005] The three-dimensional position measurement system described in Patent Document 2 measures the three-dimensional position of a measurement target point, which is the apex of ice that has adhered to the surface of an object, particularly when ice has adhered to the surface of the object.

[0006] JP 2008-506957 A JP 2018-146363 A

[0007] One embodiment of the technique of the present disclosure provides a scanning system, method, and program for detecting at least one of the transmittance and transmitted color of a region of an object that is at least partially transparent.

[0008] The invention of a first aspect is a scanning system comprising an illumination device that irradiates an object with illumination light, a first imaging device that images the object illuminated by the illumination device, and a processor, wherein the processor acquires a plurality of first captured images from the first imaging device that are captured in different imaging directions relative to the object, and detects at least one of the transmittance and transmitted color of a region of the object through which the illumination light passes based on the plurality of first captured images.

[0009] In the scanning system according to the second aspect of the present invention, in the first aspect, it is preferable that the processor extracts the area of ​​the object through which the illumination light passes based on information on changes in the illumination light that reaches the screen on the opposite side of the object from the illumination device.

[0010] In the scanning system according to the third aspect of the present invention, in the second aspect, the processor detects at least one of the transmittance and the transmitted color based on image information of the area of ​​the object in the first captured image through which the illumination light has passed and image information of the area of ​​the screen through which the illumination light is incident directly without passing through the object.

[0011] In a scanning system according to a fourth aspect of the present invention, in any of the first to third aspects, it is preferable that the illumination device includes a projection device that projects a projection pattern, and the processor extracts an area of ​​the object through which the projection pattern passes based on information about changes in the projection pattern that reaches a screen on the opposite side of the object from the projection device.

[0012] In the scanning system according to the fifth aspect of the present invention, in any one of the first to fourth aspects, it is preferable that the processor generates a 3D model of the object by a volume intersection method based on silhouettes of the object included in a plurality of first captured images captured from different directions.

[0013] In a scanning system according to a sixth aspect of the present invention, in any one of the first to fifth aspects, it is preferable that the processor generates a 3D model of the object by photogrammetry based on a plurality of first captured images captured in different directions.

[0014] A seventh aspect of the present invention is directed to a scanning system according to the fifth or sixth aspect, wherein a texture generated using a plurality of first captured images is preferably mapped onto the surface of the 3D model.

[0015] In the scanning system according to the eighth aspect of the present invention, in the fifth or sixth aspect, it is preferable that a texture corresponding to at least one of transmittance and transmitted color is mapped onto the surface of the region of the object of the 3D model through which the illumination light passes, or at least one of the transmittance and transmitted color is added as ancillary information of the 3D model.

[0016] A scanning system according to a ninth aspect of the present invention is preferably one in which, in any of the first to eighth aspects, the illumination device includes a projection device that projects a projection pattern, and the processor extracts an area of ​​the object through which the projection pattern passes based on the projection pattern contained in the first captured image captured by the first imaging device and that reaches the screen on the opposite side of the object from the projection device, or extracts an area of ​​the object through which the projection pattern passes based on the projection pattern on the screen and the projection pattern on the object.

[0017] A scanning system according to a tenth aspect of the present invention is, in any of the first to ninth aspects, provided with a second imaging device that images the screen from the same side as the object with respect to the screen on the opposite side of the lighting device across the object, or a third imaging device that images the screen from the opposite side of the object across the screen, and it is preferable that the processor detects at least one of the transmittance and transmitted color of the area of ​​the object through which the illumination light reaching the screen passes based on the image captured by the second imaging device or the third imaging device.

[0018] In an eleventh aspect of the present invention, in the scanning system of any of the first to tenth aspects, the illumination device includes a projection device that projects a projection pattern, and the processor acquires, as a reference projection pattern, a projection pattern that reaches a screen on the opposite side of the object from the projection device, which is a projection pattern that would be projected onto the screen if no object were present, and extracts an area of ​​the object through which the projection pattern passes based on at least one of the first captured image and the projection pattern that reaches the screen, and the reference projection pattern.

[0019] A scanning system according to a twelfth aspect of the present invention is preferably any of the first to eleventh aspects, further comprising a rotary table on which an object is placed, and the plurality of first captured images are images captured by the first imaging device at different rotational positions of the rotary table.

[0020] A thirteenth aspect of the present invention provides the scanning system of the twelfth aspect, wherein the surface of the rotary table functions as part of the screen on the opposite side of the object from the illumination device.

[0021] A scanning system according to a fourteenth aspect of the present invention is the scanning system of the twelfth or thirteenth aspect, wherein the surface of the turntable is preferably provided with a pattern that can be used for photogrammetry based on a plurality of first captured images captured in different directions.

[0022] A fifteenth aspect of the invention is a scanning method in a scanning system including an illumination device that irradiates an object with illumination light, a first imaging device that images the object illuminated by the illumination device, and a processor, the scanning method including the steps of the processor acquiring a plurality of first captured images from the first imaging device that have different imaging directions relative to the object, and the processor detecting at least one of the transmittance and transmitted color of a region of the object through which the illumination light passes, based on the plurality of first captured images.

[0023] A scanning method according to a sixteenth aspect of the present invention is the same as that of the fifteenth aspect, and preferably includes a step in which the processor generates a 3D model of the object by a volume intersection method based on silhouettes of the object included in a plurality of first captured images captured in different directions.

[0024] A scanning method according to a seventeenth aspect of the present invention is the same as that of the fifteenth or sixteenth aspect, and preferably includes a step in which a processor generates a 3D model of the object by photogrammetry based on a plurality of first captured images captured in different directions.

[0025] The invention of the 18th aspect is a scanning program that causes a computer to realize the following functions: an image of an object illuminated with illumination light from a lighting device is captured by a first imaging device, whereby a plurality of first captured images are captured in different imaging directions relative to the object; a function of acquiring the plurality of first captured images captured by the first imaging device; and a function of detecting at least one of the transmittance and transmitted color of a region of the object through which the illumination light passes based on the plurality of first captured images.

[0026] In the 19th aspect of the present invention, the scanning program in the 18th aspect preferably includes at least one of a function for generating a 3D model of an object by volume intersection based on the silhouette of the object contained in a plurality of first captured images captured in different directions, and a function for generating a 3D model of an object by photogrammetry based on a plurality of first captured images captured in different directions.

[0027] FIG. 1 is an external view of a main part showing a first embodiment of a scanning system according to the present invention. FIG. 2 is a diagram showing an example of a first captured image of an object or the like captured by the imaging device shown in FIG. 1. FIG. 3 is a perspective view of the rotary table shown in FIG. 1. FIG. 4 is a block diagram showing an embodiment of the hardware configuration of an information processing device constituting the scanning system according to the present invention. FIG. 5 is a flowchart showing an embodiment of a scanning method according to the present invention. FIG. 6 is an external view of a main part showing a second embodiment of a scanning system according to the present invention. FIG. 7 is a diagram showing another example of a first captured image captured by the first imaging device.

[0028] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of a scanning system, method, and program according to the present invention will now be described with reference to the accompanying drawings.

[0029] [Summary of the Invention] When generating a 3D model of an object by scanning it in three dimensions (3D), if the object is not transparent, it can often be modeled well using photogrammetry, which uses multiple images of the object taken from different angles.

[0030] On the other hand, if the object is a transparent or semi-transparent material, it is often impossible to generate a 3D model correctly, and there is also the problem that transparent areas cannot be detected even when detecting objects on a 2D captured image.

[0031] Therefore, the object to be scanned is irradiated with illumination light (preferably a projection pattern), a screen is placed in the background, and an image of the object together with the screen is captured. Based on the captured image, the area of ​​the object through which the illumination light passes (transparent area) is extracted, and at least one of the transmittance and transmitted color of the transparent area is detected. Note that in this example, the transparent area includes a semi-transparent area.

[0032] FIG. 1 is an external view of a main part showing a first embodiment of a scanning system according to the present invention.

[0033] The scanning system shown in FIG. 1 comprises a first imaging device 10, an illumination device 20, a rotary table 30, and a screen 40.

[0034] An object 50 is placed on the turntable 30. In this example, the object 50 is a transparent cup. The screen 40 is installed on the opposite side of the object 50 on the turntable 30 from the lighting device 20.

[0035] 1, the first imaging device 10 is fixed to a tripod installed on the floor 60, and the lighting device 20 can be fixed to, for example, a hosiery of the first imaging device 10. Note that the lighting device 20 is not limited to being fixed to the first imaging device 10.

[0036] 1, L1 indicates the optical axis of the first imaging device 10, and L2 indicates the optical axis of the lighting device 20. The optical axes L1 and L2 shown in Fig. 1 are parallel to each other in the vertical direction in Fig. 1, but they may be parallel to each other in the horizontal direction or in the vertical and horizontal directions, or the directions of the optical axes L1 and L2 may be different.

[0037] The first imaging device 10 can be a general-purpose imaging device, but may also be one that has a 3D model imaging mode for performing scan imaging or the like according to the present invention.

[0038] The lighting device 20 irradiates the object 50 with illumination light, and the illumination light forms a shadow (silhouette) of the object 50 on the screen 40 .

[0039] Since there is parallax between the first imaging device 10 and the lighting device 20, when the object 50 is opaque, the first imaging device 10 can capture an image of the shadow of the object 50 (the shadow of a part of the object 50) on the screen 40. When the object 50 is partially or entirely transparent, the first imaging device 10 can acquire (image) information on changes in the illumination light that has passed through the object 50 and reached the screen 40. Details of the information on changes in the illumination light will be described later.

[0040] The rotary table 30 and the screen 40 are provided on the floor surface 60 in the same manner as the tripod. The screen 40 is not limited to a dedicated projection screen, and may be a wall, a partition, a table, or the like with a uniform density.

[0041] The turntable 30 rotates at a constant rotational speed together with the object 50, and the first imaging device 10 captures images multiple times during at least one rotation of the object 50. As a result, the first imaging device 10 captures images of the object 50 at different rotational positions of the turntable 30, and as a result, obtains multiple captured images (first captured images) of the object 50 in different imaging directions.

[0042] It is also preferable to change the height and / or tilt angle of the first imaging device 10 each time the rotary table 30 makes one revolution, so that the object can be imaged at various heights and angles.

[0043] Furthermore, it is preferable that the first imaging device 10 has a small aperture (large aperture value (F-number)) and uses a wide-angle lens as the imaging lens so that the entire object 50 and the screen 40 are in focus (so that the object 50 and the screen 40 are within the depth of field). This is to enable the contours of shadows on the screen 40 to be captured clearly. Therefore, it is preferable that the screen 40 be located immediately behind the object 50.

[0044] FIG. 2 is a diagram showing an example of a first captured image of an object or the like captured by the imaging device shown in FIG.

[0045] 1 includes a projection device that projects a projection pattern. The illumination device 20 can be capable of irradiating (projecting) illumination light or projecting a projection pattern (e.g., a grid-shaped pattern light) instead of illumination light.

[0046] The first captured image I shown in Fig. 2 is an image captured by the first imaging device 10 when a grid-shaped pattern light is projected from the lighting device 20. The grid-shaped pattern shown in the first captured image I shown in Fig. 2 is an image showing the grid-shaped pattern light projected from the lighting device 20, and is represented by black lines for convenience.

[0047] In FIG. 2, the first captured image I includes an area 30A corresponding to the turntable 30, an area 40A corresponding to the screen 40, an area 50A corresponding to the object 50, and an area 60A corresponding to the floor surface 60.

[0048] In this example, the object 50 is a transparent cup, and therefore the pattern light that has passed through the object 50 reaches the screen 40, and the first image capturing device 10 captures the reflected light of the pattern light that has reached the screen 40, and the pattern light that has been reflected in an area other than the area corresponding to the object 50 (for example, the pattern light that has been reflected in an area of ​​the screen 40 where the pattern light is directly incident without passing through the object 50). Needless to say, pattern light other than the grid-shaped pattern light can also be captured.

[0049] The pattern light that has passed through the object 50 and reached the screen 40 is attenuated according to the transmittance of the object 50, and if the object 50 has a transmitted color, the color changes corresponding to the transmitted color. The pattern light that has reached the screen 40 is reflected by the screen 40 and passes through the object 50 again, whereby it is imaged by the first imaging device 10.

[0050] The pattern light that passes through the object 50, reaches the screen 40, is reflected by the screen 40, and passes through the object 50 again is dimmed in accordance with the transmittance of the object 50 compared to the pattern light that is directly reflected by the screen 40, and if the object 50 has a transmitted color, it changes color in accordance with the transmitted color.

[0051] Furthermore, when the object 50 is a transparent cup as in this example, the transmittance of the pattern light is low at the edge and rim of the cup, and the pattern light is diffused due to the lens effect caused by the change in thickness.

[0052] Furthermore, if information about the pattern light to be projected (shape, color, etc.) is known, information about the changes between the pattern light projected directly onto the screen 40 and the pattern light that passes through the object 50 and is projected onto the screen 40 (misalignment and deformation of the pattern light) can be used to extract the transparent areas of the object 50, potentially making it possible to determine the identity of the transparent areas, including information about the refractive index.

[0053] Therefore, based on the image showing the pattern light captured in the first captured image I, the area where the pattern light passes through the object 50 (transparent area) can be separated and extracted from the background.

[0054] In addition, the transmittance of the transparent region of the object 50 can be detected based on the ratio of the luminance values ​​of an image showing the pattern light that has transmitted through the object 50 and an image showing the pattern light that is directly incident on the screen 40 without transmitting through the object 50. Furthermore, the transmitted color of the transparent region of the object 50 can be detected based on the difference in color information between the image showing the pattern light that has transmitted through the object 50 and the image showing the pattern light that is directly projected onto the screen 40.

[0055] Furthermore, the reference projection pattern may be obtained by projecting a pattern of light onto the screen 40 when the object 50 is not present, without placing the object 50 on the turntable 30. In this case, the object 50 is placed on the turntable 30, and a plurality of first captured images are taken from different imaging directions relative to the object 50. Based on at least one of the plurality of first captured images and the projection pattern that has reached the screen 40, and a separately captured reference projection pattern, a transparent region of the object 50 through which the projection pattern is transmitted is extracted, and at least one of the transmittance and transmitted color of the transparent region can be detected.

[0056] FIG. 3 is a perspective view of the rotary table shown in FIG.

[0057] The surface 30B of the turntable 30 shown in FIG. 3 is provided with a pattern that can be used for photogrammetry based on a plurality of first captured images captured in different directions. In this example, a random point cloud pattern 32 with a known positional relationship is provided.

[0058] The rotation angle of the turntable 30 can be detected by matching the point cloud pattern 32 captured in multiple first captured images captured in different directions, and the actual size of the object 50 can be determined from the spacing between each point of the point cloud pattern 32 captured in the first captured images. Note that the surface 30B of the turntable 30 on which the object 50 is placed can be used as part of a screen (see FIG. 2).

[0059] [Information Processing Device] FIG. 4 is a block diagram showing an embodiment of the hardware configuration of an information processing device that constitutes the scan system according to the present invention.

[0060] The information processing device 100 shown in FIG. 4 may be, for example, built into the first imaging device 10, or may be configured as a personal computer, workstation, or the like separate from the first imaging device 10.

[0061] The information processing device 100 includes a processor 110 , a memory 120 , a display 130 , an input / output interface 140 , and an operation unit 150 .

[0062] The processor 110 is composed of a CPU (Central Processing Unit) and the like, and controls each part of the information processing device 100 in an integrated manner, executes a scan program to cause the first imaging device 10 to perform scan imaging, and acquires a plurality of first captured images of the object 50 captured by the first imaging device 10 through scan imaging, the first captured images being captured in different imaging directions. The processor 110 generates a 3D model of the object 50 based on the acquired plurality of first captured images, and executes information processing such as detecting at least one of the transmittance and transmitted color of a transparent region of the object 50 through which illumination light irradiated from the illumination device 20 (or a projected pattern such as a grid-like pattern light) passes. Details of the information processing by the processor 110 will be described later.

[0063] The memory 120 includes a flash memory, a read-only memory (ROM), a random access memory (RAM), a hard disk drive, etc. The flash memory, ROM, or hard disk drive is a non-volatile memory that stores an operating system, various programs including the scanning program according to the present invention, etc. The scanning program may include a program that generates a 3D model of the object based on a plurality of first captured images of the object captured from different directions.

[0064] In addition, non-volatile memory such as a flash memory and a hard disk drive can store multiple first captured images of an object captured by the first imaging device 10 in different imaging directions, a 3D model of the object generated from the multiple first captured images, and information on the projection pattern (if a grid-shaped pattern light is projected from the lighting device 20, information on the shape of the pattern light), etc.

[0065] The RAM functions as a work area for processing by the processor 110. It also temporarily stores various programs stored in a flash memory or the like, the first captured image, etc. Note that the processor 110 may have a part of the memory 120 (RAM) built in.

[0066] The display 130 can display a screen for operating the information processing device 100 as well as the first captured image and the 3D model read from the memory 120 .

[0067] The input / output interface 140 includes a connection unit connectable to an external device, a communication unit connectable to a network, etc. As the connection unit connectable to an external device, a Universal Serial Bus (USB), a High-Definition Multimedia Interface (HDMI) (HDMI is a registered trademark), etc. can be applied.

[0068] The information processing device 100 can be configured as a device independent of the first imaging device 10. In this case, the processor 110 can acquire a plurality of first captured images of an object captured in different imaging directions from the first imaging device 10 via the input / output interface 140, or, if a plurality of first captured images are stored in the cloud, acquire the plurality of first captured images from the cloud via the input / output interface 140. The processor 110 can also store the plurality of first captured images acquired in this manner in the memory 120.

[0069] The operation unit 150 includes a pointing device such as a mouse, a keyboard, and the like, and functions as part of a GUI (Graphical User Interface) that uses the display screen of the display device 130 and accepts instruction inputs by user operation.

[0070] When the information processing device 100 is included in the first imaging device 10, the display 130 and the operation unit 150 correspond to a monitor, operation buttons, etc. on the back surface of the first imaging device 10.

[0071] When creating a 3D model of the object 50, the processor 110 of the information processing device 100 causes the lighting device 20 to irradiate the object 50 with a grid-like pattern light, and also rotates the turntable 30 at a constant speed, thereby rotating the object 50 on the turntable 30.

[0072] Furthermore, the processor 110 causes the first imaging device 10 to capture images of the object 50 multiple times during one rotation of the turntable 30. Because the object 50 is rotating, the first imaging device 10 can capture multiple first captured images of the object 50 in different imaging directions.

[0073] The driving of the rotary table 30 and the irradiation of the pattern light from the lighting device 20 are not limited to cases where they are performed in response to instructions received from the processor 110, but may also be performed in response to instructions from the user.

[0074] Furthermore, the scanning of the object 50 by the first imaging device 10 may be performed not only in response to an instruction from the processor 110 but also in response to an instruction from the user. For example, the first imaging device 10 may be set to continuous shooting (continuous shooting), and the user may hold down the shutter button while the turntable 30 makes one rotation.

[0075] When the first imaging device 10 scans and captures a plurality of first captured images of the object 50 in different imaging directions, the processor 110 acquires the plurality of first captured images of the object 50 in different imaging directions from the first imaging device 10 via the input / output interface 140. If the plurality of first captured images are stored in the cloud, the processor 110 can acquire the plurality of first captured images from the cloud via the input / output interface 140, or can acquire the plurality of first captured images via the recording medium of the first imaging device 10.

[0076] [Scanning Method] FIG. 5 is a flowchart showing an embodiment of a scanning method according to the present invention, and in particular a flow from acquisition of a plurality of first captured images by the processor 110 to generation of a 3D model of the object 50.

[0077] In FIG. 5, the processor 110 sets a parameter i indicating a specific first captured image among the plurality of first captured images to 1 (step S10).

[0078] Next, the processor 110 acquires a plurality of first captured images I from the first imaging device 10, each captured in a different direction relative to the object 50, and stores the acquired plurality of first captured images I in the memory 120 (step S20). At this time, the processor 110 assigns numbers 1 to n corresponding to the order in which the plurality of first captured images I were captured, and designates the plurality of first captured images I as first captured images (I1, I2, ..., In). Note that n is a value corresponding to the number of the plurality of first captured images I.

[0079] The processor 110 acquires the first captured image Ii, which is the i-th captured image indicated by the parameter i, from the first captured images (I1, I2, ..., In) (step S30). Note that when acquiring the first captured image Ii for the first time, the first captured image I1 is acquired because i=1 (see step S10).

[0080] Next, the processor 110 extracts transparent regions of the object 50 based on the first captured image Ii acquired in step S20 (step S40). The extraction of transparent regions involves extracting transparent regions of the object through which the patterned light (in this example, a grid-shaped patterned light) passes, based on change information about the illumination light reaching the screen 40. If the object is opaque or includes an opaque region, the patterned light does not pass through the opaque region. On the other hand, the patterned light passes through the transparent region of the object 50 and reaches the screen 40. However, the patterned light that is directly projected onto the screen 40 and the patterned light that passes through the object 50 and is projected onto the screen 40 differ in terms of misalignment, deformation, brightness, color, etc. This change information about the patterned light can be used to extract transparent regions of the object 50, making it possible to detect the identity of the transparent region, including refractive index information.

[0081] Therefore, the processor 110 can separate and extract the transparent region through which the pattern light passes through the object 50 from the background, based on the image showing the pattern light captured in the first captured image Ii. In the case of the lattice-shaped pattern light of this example, it is easy to see the change in the lattice due to the refraction of the pattern light at the boundary between the pattern light that passes through the transparent region of the object 50 and reaches the screen 40 and the pattern light that is directly projected onto the screen 40, and the like, and therefore it is possible to extract the transparent region including the outline of the transparent region of the object 50 with greater accuracy.

[0082] Next, the processor 110 detects at least one of the transmittance and the transmitted color of the transparent region of the object 50 extracted in step S40. In this example, the processor 110 detects the transmittance and the transmitted color of the transparent region. That is, the processor 110 detects the transmittance and the transmitted color of the object 50 based on image information of the transparent region of the object 50 through which the pattern light has passed, which is included in the first captured image Ii, and image information of the region of the screen 40 onto which the pattern light is directly incident without passing through (transmitting) the object 50. For example, the processor 110 can detect the transmittance of the transparent region of the object 50 based on the ratio between the luminance value of an image representing the pattern light that has reached the screen 40 without passing through the object 50 and the luminance value of an image representing the pattern light that has passed through the object 50 and reached the screen 40. The processor 110 can also detect the transmitted color of the transparent region of the object 50 based on the difference in color information between the image representing the pattern light that has passed through the object 50 and the image representing the pattern light that is directly projected onto the screen 40.

[0083] The transmittance and color information of the transparent region of the object 50 may be calculated for each local region of the transparent region, or may be calculated as an average value for the entire transparent region. When the transmittance and transmitted color of the transparent region of the object 50 are detected based on the first captured image Ii, the detection is performed taking into consideration that the first captured image contains an image of pattern light that has passed through (made a round trip through) the transparent region of the object 50 twice.

[0084] The processor 110 stores information on the transparent region of the object 50 detected based on the first captured image Ii, the transmittance, and the transmitted color in the memory 120 in association with the first captured image Ii (step S52).

[0085] Next, the parameter i is incremented by 1 (step S60), and it is determined whether the parameter i exceeds n (i>n) (step S70). If the parameter i does not exceed n (if i≦n), the process proceeds to step S30, and the processes from step S30 to step S70 are repeatedly executed. If the parameter i exceeds n (if i>n), the process proceeds to step S80.

[0086] In step S80, the processor 110 uses information (silhouettes) of transparent regions included in the plurality of first captured images (I1, I2, ..., In) stored in the memory 120 to generate a 3D model of the transparent region of the object 50 by a volume intersection method based on the silhouettes of the transparent regions of the object 50. In addition to generating the 3D model by the volume intersection method, or instead of generating the 3D model by the volume intersection method, the processor 110 generates a 3D model of the transparent region of the object 50 by photogrammetry based on information of the transparent region (images extracted corresponding to the transparent region) included in the plurality of first captured images (I1, I2, ..., In).

[0087] It is preferable to map a texture corresponding to at least one of the transmittance and the transparent color of the transparent region onto the surface of the 3D model of the transparent region of the object 50 generated by the volume intersection method, and / or to add at least one of the transmittance and the transparent color of the transparent region as additional information of the 3D model of the transparent region.

[0088] When generating a 3D model of the transparent area of ​​the object 50 using photogrammetry, it is preferable to use an image of the transparent area extracted from an image captured when the object 50 is irradiated with uniform illumination light from the lighting device 20 instead of the pattern light by erasing the projected pattern corresponding to the pattern light from the extracted image of the transparent area using image processing, or to use an image of the transparent area extracted from an image captured when the object 50 is irradiated with uniform illumination light from the lighting device 20 instead of the pattern light.

[0089] The processor 110 stores the 3D model of the transparent area of ​​the object 50 thus generated in the memory 120 .

[0090] FIG. 6 is an external view of a main part showing a second embodiment of a scanning system according to the present invention.

[0091] In FIG. 6, the same parts as those in the first embodiment shown in FIG. 1 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0092] The second embodiment shown in FIG. 6 differs from the first embodiment shown in FIG. 1 in that a second imaging device 70 and a third imaging device 80 are added.

[0093] The second imaging device 70 is an imaging device that images the screen 40 from the same side as the object 50 with respect to the screen 40 on the opposite side from the lighting device 20 across the object 50 .

[0094] The second imaging device 70 captures an image of the screen 40 from its front side, and can capture images of the grid-like pattern light that has passed through the object 50 and reached the screen 40, the pattern light that is irradiated directly onto the screen 40 without passing through the object 50, and, if the object 50 has an opaque area, an image showing the shadow (silhouette) of that opaque area.

[0095] The second imaging device 70 does not directly image the object 50, but images the screen 40 onto which pattern light, etc., is projected, which passes through the object 50 as it rotates as the turntable 30 rotates, and therefore indirectly captures multiple images of the object 50 with different imaging directions.

[0096] Therefore, similar to the plurality of first captured images captured by the first imaging device 10, the processor 110 can generate a 3D model of the transparent region of the object 50 based on a plurality of captured images of the object 50 captured by the second imaging device 70 in different imaging directions, and can also detect the transmittance and transmitted color of the transparent region of the object 50. When generating a 3D model of the object 50 from a plurality of captured images of the pattern light that has reached the screen 40 in a case where the irradiation directions of the pattern light on the object 50 are substantially different, it is preferable to generate the 3D model of the transparent region of the object 50 by a volume intersection method using information (silhouette) of the transparent region of the object 50.

[0097] The third imaging device 80 is an imaging device that images the screen from the opposite side of the screen 40 to the object 50. In this case, a rear projection screen that diffuses and transmits light incident on the screen 40 is used as the screen 40.

[0098] The third imaging device 80 captures an image of the screen 40 from its back side, and can capture an image showing light that has been diffused and transmitted by the screen 40, such as a grid-like pattern light that has passed through the object 50 and reached the screen 40, or a pattern light that is incident directly on the screen 40 without passing through the object 50.

[0099] The processor 110 generates a 3D model of the transparent area of ​​the object 50 based on the multiple captured images taken by the third imaging device 80, as well as the multiple captured images taken by the second imaging device 70, and can also detect the transmittance and transmitted color of the transparent area of ​​the object 50.

[0100] In addition, the scanning system of the second embodiment shown in Figure 6 is equipped with a second imaging device 70 and a third imaging device 80 in addition to the first imaging device 10, but is not limited to this and may be a system equipped with only the second imaging device 70 or only the third imaging device 80.

[0101] FIG. 7 is a diagram showing another example of the first captured image captured by the first imaging device.

[0102] The first captured image shown in FIG. 7 is an image of a spherical object having an opaque region 52A and a transparent region 52B captured by the first imaging device 10 (see FIG. 1).

[0103] The processor 110 acquires a plurality of first captured images of the sphere captured by the first imaging device 10 in different imaging directions, and separates and extracts the opaque region 52A and the transparent region 52B of the sphere.

[0104] Furthermore, for the spherical transparent region 52B, the processor 110 detects at least one of the transmittance and the transmission color of the transparent region 53B.

[0105] The processor 110 then generates a 3D model of the opaque region 52A of the sphere by photogrammetry based on multiple images captured from different angles that show the opaque region 52A of the sphere. Textures corresponding to the polygons of minute regions on the surface of the opaque region 52A of this 3D model are mapped onto the polygons.

[0106] The processor 110 also generates a 3D model of the spherical transparent region 52B by a volume intersection method based on a plurality of silhouettes of the spherical transparent region 52B captured in different directions. The processor 110 can map a texture corresponding to at least one of the transmittance and the transparent color of the transparent region 52B onto the surface of the 3D model of the transparent region 52B.

[0107] These 3D models can be combined. Alternatively, a 3D model of the opaque region 52A of the sphere may be generated, and the portions of the sphere surface that do not have an opaque texture may be made transparent, and a texture corresponding to at least one of the transmittance and the transparent color of the transparent region 52B may be mapped onto the 3D model.

[0108] It is also preferable to erase the projected pattern corresponding to the grid-like pattern light contained in the texture of the 3D model by image processing, or to use an image extracted from the first captured image when uniform illumination light is irradiated onto the object 50 from the lighting device 20 instead of the pattern light.

[0109] [Others] In the present embodiment, the object 50 is placed on the turntable 30, and the turntable 30 is rotated to capture an image of the object 50 using the first imaging device 10, thereby obtaining a plurality of first captured images with different imaging directions relative to the object 50. However, this is not limited to this, and a plurality of first captured images with different imaging directions may be obtained by capturing an image of a stationary object 50 while rotating the first imaging device 10 and the lighting device 20. In this case, the user may hold the first imaging device 10 in his / her hand and capture images while changing the imaging direction, shooting position, etc.

[0110] In this embodiment, the hardware structure of a processing unit that executes various processes, such as a CPU (Central Processing Unit), is the following various processors: The various processors include a CPU, which is a general-purpose processor that executes software (programs) to function as various processing units, a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), whose circuit configuration can be changed after manufacture, and a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing specific processes.

[0111] A single processing unit may be configured with one of these various processors, or may be configured with two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). Multiple processing units may also be configured with a single processor. Examples of multiple processing units configured with a single processor include: a first configuration, as typified by client or server computers, in which a single processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units; and a second configuration, as typified by system-on-chip (SoC), in which a processor is used to realize the functions of an entire system including multiple processing units on a single IC (Integrated Circuit) chip. In this way, the various processing units are configured with one or more of the above-mentioned various processors as a hardware structure.

[0112] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit made up of a combination of circuit elements such as semiconductor elements.

[0113] The present invention also includes a scanning program that, when installed on a computer, causes the computer to execute the scanning method of the present invention, and a non-transitory computer-readable recording medium on which the program is recorded.

[0114] Furthermore, the present invention is not limited to the above-described embodiment, and it goes without saying that various modifications are possible without departing from the spirit of the present invention.

[0115] REFERENCE SIGNS LIST 10... First imaging device 20... Illumination device 30... Rotary table 32... Point cloud pattern 40... Screen 50... Object 60... Floor surface 60A... Area 70... Second imaging device 80... Third imaging device 100... Information processing device 110... Processor 120... Memory 130... Display 140... Input / output interface 150... Operation unit L1, L2... Optical axis S10 to S80... Steps

Claims

1. A scanning system comprising an illumination device that irradiates an object with illumination light, a first imaging device that images the object illuminated by the illumination device, and a processor, wherein the processor obtains a plurality of first captured images from the first imaging device having different imaging directions with respect to the object, and detects at least one of the transmittance and transmitted color of a region of the object through which the illumination light passes based on the plurality of first captured images.

2. The scanning system according to claim 1, wherein the processor extracts the area of ​​the object through which the illumination light passes based on information about changes in the illumination light that reaches a screen on the opposite side of the object from the illumination device.

3. The scanning system of claim 2, wherein the processor detects at least one of the transmittance and the transmitted color based on image information of a region of the object through which the illumination light in the first captured image has passed and image information of a region of the screen where the illumination light is directly incident without passing through the object.

4. The scanning system of claim 1, wherein the illumination device includes a projection device that projects a projection pattern, and the processor extracts an area of ​​the object through which the projection pattern passes based on information about changes in the projection pattern that reaches a screen on the opposite side of the object from the projection device.

5. The scanning system of claim 1, wherein the processor generates a 3D model of the object by a volume intersection method based on silhouettes of the object contained in the first captured images captured in different directions.

6. The scanning system of claim 5, wherein the processor generates a 3D model of the object by photogrammetry based on a plurality of the first captured images captured in different directions.

7. The scanning system according to claim 6, wherein a texture generated using a plurality of the first captured images is mapped onto a surface of the 3D model.

8. The scanning system of claim 5, wherein a texture corresponding to at least one of the transmittance and the transmitted color is mapped onto the surface of the region of the object of the 3D model through which the illumination light passes, or at least one of the transmittance and the transmitted color is added as auxiliary information of the 3D model.

9. The scanning system of claim 1, wherein the illumination device includes a projection device that projects a projection pattern, and the processor extracts an area of ​​the object through which the projection pattern passes based on the projection pattern contained in the first captured image captured by the first imaging device and which reaches a screen on the opposite side of the object from the projection device, or extracts an area of ​​the object through which the projection pattern passes based on the projection pattern on the screen and the projection pattern on the object.

10. The scanning system of claim 1, further comprising a second imaging device that images the screen from the same side as the object with respect to the screen opposite the illumination device across the object, or a third imaging device that images the screen from the opposite side of the screen to the object, and the processor detects at least one of the transmittance and transmitted color of a region of the object through which the illumination light that reaches the screen passes based on an image captured by the second imaging device or the third imaging device.

11. The scanning system of claim 1, wherein the illumination device includes a projection device that projects a projection pattern, and the processor acquires, as a reference projection pattern, the projection pattern that reaches a screen on the opposite side of the object from the projection device and that is projected onto the screen when the object is not present, and extracts an area of ​​the object through which the projection pattern passes based on at least one of the first captured image and the projection pattern that reaches the screen and the reference projection pattern.

12. The scanning system according to claim 1, further comprising a rotating table on which the object is placed, and the plurality of first captured images are images captured by the first imaging device at different rotational positions of the rotating table.

13. The scanning system of claim 12, wherein the surface of the rotating table functions as part of a screen on the opposite side of the object from the illumination device.

14. The scanning system according to claim 13, wherein a pattern usable for photogrammetry based on a plurality of the first captured images captured in different directions is provided on a surface of the rotating table.

15. A scanning method in a scanning system including an illumination device that irradiates an object with illumination light, a first imaging device that images the object illuminated by the illumination device, and a processor, the scanning method including the steps of: the processor acquiring a plurality of first captured images from the first imaging device having different imaging directions with respect to the object; and the processor detecting at least one of the transmittance and transmitted color of a region of the object through which the illumination light passes, based on the plurality of first captured images.

16. The scanning method according to claim 15, further comprising a step of the processor generating a 3D model of the object by a volume intersection method based on silhouettes of the object included in the plurality of first captured images having different imaging directions.

17. The scanning method according to claim 15 or 16, further comprising a step in which the processor generates a 3D model of the object by photogrammetry based on the plurality of first captured images having different imaging directions.

18. A scanning program that causes a computer to realize the following functions: a function of imaging an object illuminated with illumination light from a lighting device using a first imaging device, the function of imaging a plurality of first captured images having different imaging directions with respect to the object; a function of acquiring the plurality of first captured images captured by the first imaging device; and a function of detecting at least one of the transmittance and transmitted color of a region of the object through which the illumination light passes based on the plurality of first captured images.

19. The scanning program according to claim 18, comprising at least one of a function for generating a 3D model of the object by a volume intersection method based on the silhouette of the object contained in the plurality of first captured images having different imaging directions, and a function for generating a 3D model of the object by photogrammetry based on the plurality of first captured images having different imaging directions.

20. A non-transitory computer-readable recording medium having the program according to claim 18 or 19 recorded thereon.

Images