Scanning system, method, and program

Abstract

Provided are a scanning system, a method, and a program for detecting at least one of the transmittance and the transmission color of a region of an object that is at least partially transparent. A scanning system is provided with: an illumination device (20) that irradiates an object (50) with illumination light; a first imaging device (10) that captures an image of the object (50) illuminated by the illumination device (20); and a processor. The processor acquires, from the first imaging device (10), a plurality of first captured images having different imaging directions with respect to the object (50), and detects, on the basis of the plurality of first captured images, the transmittance and / or transmission color of a region of the object (50) through which the illumination light passes.

Description

Scanning systems, methods and procedures Technical Field

[0001] This invention relates to a scanning system, method, and procedure, and more particularly to a technique for scanning objects that are at least partially transparent. Background Technology

[0002] Previously, a method for measuring the height (height profile) of a near-transparent object with a refractive index, such as a film or coating, has been proposed (Patent Document 1).

[0003] The method described in Patent Document 1 is based on high-speed moiré fringe interferometry, which acquires an image of an object corresponding to an intensity pattern projected onto a mask, and uses the acquired image, the phase of the intensity pattern associated with the object, the refractive index of the object, and the reference phase of the intensity pattern corresponding to the reference surface to determine the height of the object from the reference surface.

[0004] Furthermore, Patent Document 2 describes a three-dimensional position measurement system comprising: multiple photographic mechanisms that photograph an object from different directions; and a processing mechanism that acquires images photographed by the multiple photographic mechanisms and measures the three-dimensional position of the object point through image processing.

[0005] The three-dimensional position measurement system described in Patent Document 2 measures the three-dimensional position of the object point, specifically when ice is attached to the surface of the object.

[0006] Previous technical documents

[0007] Patent documents

[0008] Patent Document 1: Japanese Patent Publication No. 2008-506957

[0009] Patent Document 2: Japanese Patent Application Publication No. 2018-146363 Summary of the Invention

[0010] -The technical problem that the invention aims to solve-

[0011] One embodiment of the present invention provides a scanning system, method, and procedure for detecting at least one of the transmittance and transmittance color of a region of at least a partially transparent object.

[0012] -Means used to solve technical problems-

[0013] The invention involved in the first aspect is a scanning system comprising: an illumination device for illuminating an object with illumination light; a first imaging device for capturing images of the object illuminated by the illumination device; and a processor, wherein the processor performs the following processing: acquiring a plurality of first imaging images from the first imaging device with different shooting directions relative to the object; and detecting at least one of the transmittance and the transmitted color of the area of ​​the object through which the illumination light is transmitted based on the plurality of first imaging images.

[0014] In the first embodiment, the scanning system according to the second aspect of the present invention preferably extracts the area of ​​the object through which the illumination light is transmitted based on the change information of the illumination light reaching the screen on the side opposite to the illumination device that is separated from the object.

[0015] In the third aspect of the present invention, the scanning system in the second aspect involves a processor detecting at least one of transmittance and transmitted color based on image information of the area of ​​the object through which illumination light is transmitted in the first image and image information of the area of ​​the screen through which illumination light is directly incident without passing through the object.

[0016] The scanning system according to the fourth aspect of the present invention is preferably in any one of the first to third aspects, wherein the illumination device includes a projection device for projecting a projection pattern, and the processor extracts the area of ​​the object through which the projection pattern is transmitted based on the change information of the projection pattern reaching the screen on the side opposite to the projection device across the object.

[0017] The scanning system involved in the fifth aspect of the present invention is preferably in any of the first to fourth aspects, wherein the processor generates a 3D model of the object by means of a visual volume cross method based on the outline of the object contained in a plurality of first camera images with different shooting directions.

[0018] The scanning system involved in the sixth aspect of the present invention is preferably in any of the first to fifth aspects, wherein the processor generates a 3D model of the object by photogrammetry based on multiple first images with different shooting directions.

[0019] The scanning system involved in the seventh aspect of the present invention is preferably in the fifth or sixth aspect, in which a texture generated using a plurality of first camera images is mapped on the surface of the 3D model.

[0020] The scanning system according to the eighth aspect of the present invention preferably has, in the fifth or sixth aspect, a texture corresponding to at least one of transmissivity and transmissive color mapped on the surface of the 3D model in the area of ​​the object through which the illumination light is transmitted, or at least one of transmissivity and transmissive color is added as supplementary information of the 3D model.

[0021] The scanning system according to the ninth aspect of the present invention is preferably in any one of the first to eighth aspects, wherein the illumination device includes a projection device for projecting a projection pattern, and the processor performs the following processing: based on the projection pattern contained in the first image captured by the first camera device and reaching the screen on the side opposite to the projection device across the object, extracting the area of ​​the object through which the projection pattern is transmitted, or based on the projection pattern on the screen and the projection pattern on the object, extracting the area of ​​the object through which the projection pattern is transmitted.

[0022] The scanning system according to the 10th aspect of the present invention preferably includes a second or a third camera device in any of the 1st to 9th aspects. The second camera device captures images of the screen from the same side as the object, relative to the screen on the side opposite to the lighting device across the object. The third camera device captures images of the screen from the side opposite to the object across the screen. The processor detects at least one of the transmittance and the transmitted color of the area of ​​the object through which the illumination light reaching the screen is transmitted, based on the image captured by the second or third camera device.

[0023] The scanning system according to the 11th aspect of the present invention is preferably in any one of the 1st to 10th aspects, wherein the illumination device includes a projection device for projecting a projection pattern, and the processor performs the following processing: acquiring a projection pattern that reaches a screen on the side opposite to the projection device across the object and is projected onto the screen when the object is not present, as a reference projection pattern; and extracting the area of ​​the object transmitted by the projection pattern based on at least one of the first camera image and the projection pattern reaching the screen and the reference projection pattern.

[0024] The scanning system according to the 12th aspect of the present invention is preferably in any one of the 1st to 11th aspects, which includes a rotating stage for placing an object, and the plurality of first imaging images are images captured by the first imaging device at different rotational positions of each rotating stage.

[0025] In the 12th embodiment of the scanning system, the surface of the rotary table functions as part of a screen on the side opposite to the lighting device, separated by the object.

[0026] The scanning system according to the 14th aspect of the present invention preferably has, in the 12th or 13th aspect, a pattern provided on the surface of the rotary table that can be used for photogrammetry of multiple first images with different shooting directions.

[0027] The invention involved in the 15th aspect is a scanning method, which is a scanning method in a scanning system. The scanning system includes: an illumination device for illuminating an object with illumination light; a first imaging device for capturing images of the object illuminated by the illumination device; and a processor, wherein the scanning method includes the following steps: the processor acquires a plurality of first imaging images from the first imaging device that are in different shooting directions relative to the object; and the processor detects at least one of the transmittance and the transmitted color of the area of ​​the object through which the illumination light is transmitted based on the plurality of first imaging images.

[0028] The scanning method according to the 16th aspect of the present invention preferably includes the following steps in the 15th aspect: the processor generates a 3D model of the object by using a visual volume cross method based on the outline of the object contained in a plurality of first camera images with different shooting directions.

[0029] The scanning method according to the 17th aspect of the present invention preferably includes the following steps in the 15th or 16th aspect: the processor generates a 3D model of the object by photogrammetry based on multiple first images with different shooting directions.

[0030] The invention involved in the 18th method is a scanning program that enables a computer to perform the following functions: to enable a first camera device to capture multiple first camera images of an object illuminated by a lighting device and to capture multiple first camera images with different shooting directions relative to the object; to acquire multiple first camera images captured by the first camera device; and to detect at least one of the transmittance and the transmitted color of the area of ​​the object through which the lighting light is transmitted, based on the multiple first camera images.

[0031] The scanning program involved in the 19th aspect of the present invention preferably includes at least one of the following functions in the 18th aspect: the function of generating a 3D model of an object by using a visual volume cross method based on the outline of the object contained in a plurality of first camera images with different shooting directions; and the function of generating a 3D model of an object by using photogrammetry based on a plurality of first camera images with different shooting directions. Attached Figure Description

[0032] Figure 1 is a view showing the main parts of the first embodiment of the scanning system according to the present invention.

[0033] Figure 2 is an example of a first photographic image obtained by photographing an object or the like using the camera device shown in Figure 1.

[0034] Figure 3 is a perspective view of the rotary table shown in Figure 1.

[0035] Figure 4 is a block diagram illustrating an embodiment of the hardware structure of the information processing device constituting the scanning system according to the present invention.

[0036] Figure 5 is a flowchart illustrating an embodiment of the scanning method involved in this invention.

[0037] Figure 6 is a partial view showing the main parts of the scanning system according to the second embodiment of the present invention.

[0038] Figure 7 is another example of a first image captured by the first camera device. Detailed Implementation

[0039] Hereinafter, preferred embodiments of the scanning system, method and program involved in the present invention will be described with reference to the accompanying drawings.

[0040] [Summary of the Invention]

[0041] Regarding the generation of 3D models of objects based on three-dimensional (3D) scans, in cases where the object is not transparent, good modeling can often be achieved by using photogrammetry of multiple images taken from different camera directions relative to the object.

[0042] On the other hand, there are also the following problems: when the object is a transparent or semi-transparent material, there are many cases where the 3D model cannot be generated correctly, and even in object detection on 2D camera images, transparent areas cannot be detected.

[0043] Therefore, the object to be scanned is illuminated with lighting light (preferably a projected pattern), a screen is placed on the background and the object is photographed together with the screen, and the area of ​​the object through which the illumination light is transmitted (transparent area) is extracted based on the photographed image. Furthermore, at least one of the transmittance and transmitted color of the transparent area is detected. In this example, the transparent area includes a semi-transparent area.

[0044] Figure 1 is a view showing the main parts of the first embodiment of the scanning system according to the present invention.

[0045] The scanning system shown in Figure 1 consists of a first camera device 10, an illumination device 20, a rotating stage 30, and a screen 40.

[0046] An object 50 is placed on a rotating platform 30. In this example, the object 50 is a transparent cup. A screen 40 is positioned on the opposite side of the lighting device 20, across the object 50 on the rotating platform 30.

[0047] In Figure 1, the first camera device 10 is fixed to a tripod mounted on the ground 60, and the lighting device 20 can be fixed to the bracket of the first camera device 10, for example. However, the lighting device 20 is not limited to a lighting device fixed to the first camera device 10.

[0048] Furthermore, in Figure 1, L1 represents the optical axis of the first camera device 10, and L2 represents the optical axis of the illumination device 20. The optical axes L1 and L2 shown in Figure 1 are parallel in the vertical direction, but they can also be parallel in the horizontal direction or in the vertical and horizontal directions. In addition, the directions of each optical axis L1 and L2 can be different.

[0049] The first camera device 10 can be a general camera device, but it can also be a camera device equipped with a 3D model shooting mode such as scanning camera involved in the present invention.

[0050] The lighting device 20 illuminates the object 50 with light, but the light forms an image (outline) of the object 50 on the screen 40.

[0051] The first imaging device 10 and the illumination device 20 have parallax, so when the object 50 is opaque, the first imaging device 10 can capture an image of the object 50 on the screen 40 (an image of a portion of the object 50). Furthermore, when the object 50 is partially or entirely transparent, the first imaging device 10 can acquire (capture) information about changes in the illumination light transmitted through the object 50 to the screen 40. Details regarding the changes in illumination light will be described later.

[0052] Furthermore, the rotating platform 30 and the screen 40 are mounted on the ground 60 in the same manner as the tripod. The screen 40 is not limited to a projection screen; it can also be a wall, screen, table, etc. with uniform density.

[0053] The rotating platform 30 rotates together with the object 50 at a predetermined rotational speed, and the first camera device 10 takes multiple pictures at least once during the rotation of the object 50. Thus, the first camera device 10 takes pictures of the object 50 at different rotational positions of each rotating platform 30, resulting in the acquisition of multiple camera images (first camera images) with different shooting directions relative to the object 50.

[0054] Furthermore, it is preferable to change the height and / or pitch angle of the first camera device 10 each time the rotating stage 30 rotates once, so as to photograph the object at various heights and angles.

[0055] Furthermore, the first camera device 10 preferably reduces the aperture (increases the aperture value (F-number)) and uses a wide-angle lens as the photographic lens to focus on the entire object 50 and the screen 40 (so that the object 50 and the screen 40 are within the depth of field). This is to enable a clear capture of the outline of the image on the screen 40. Therefore, the screen 40 is preferably positioned directly behind the object 50.

[0056] Figure 2 is an example of a first photographic image obtained by photographing an object or the like using the camera device shown in Figure 1.

[0057] The lighting device 20 shown in Figure 1 includes a projection device for projecting a projection pattern. The lighting device 20 is capable of projecting a projection pattern (e.g., a grid-like pattern light) by illuminating (projecting) lighting light or by projecting lighting light instead of lighting light.

[0058] The first camera image I shown in Figure 2 is a camera image captured by the first camera device 10 when the lighting device 20 projects a grid-shaped pattern of light. Furthermore, the grid-shaped pattern shown in the first camera image I in Figure 2 represents the grid-shaped pattern of light projected by the lighting device 20, and is indicated by black lines for convenience.

[0059] In Figure 2, the first camera image I includes area 30A corresponding to the rotating platform 30, area 40A corresponding to the screen 40, area 50A corresponding to the object 50, and area 60A corresponding to the ground 60.

[0060] In this example, the object 50 is a transparent cup, so the patterned light transmitted through the object 50 reaches the screen 40. The first camera device 10 captures the reflected light of the patterned light reaching the screen 40 and the patterned light reflected outside the area corresponding to the object 50 (for example, the patterned light reflected from the area that directly enters the screen 40 without passing through the object 50). In addition, other light rays besides the grid-shaped patterned light can also be captured.

[0061] The patterned light that passes through the object 50 and reaches the screen 40 is reduced in intensity according to the transmittance of the object 50, and if the object 50 has a transmitted color, the color corresponding to that transmitted color changes. Furthermore, the patterned light that reaches the screen 40 is reflected on the screen 40 and passes through the object 50 again, thereby being captured by the first camera device 10.

[0062] Pattern light that is transmitted through object 50 to screen 40 and reflected on screen 40 and then transmitted through object 50 again is reduced in color according to the transmittance of object 50 compared to pattern light that is reflected directly on screen 40, and changes color accordingly if object 50 has a transmitted color.

[0063] Furthermore, as shown in this example, when the object 50 is a transparent cup, the transmittance of the pattern light becomes lower in the area at the end or edge of the cup, and the pattern light diffuses due to the lens effect caused by the thickness change.

[0064] Furthermore, if the information of the projected pattern light (shape, color, etc.) is known, the information of the changes in the pattern light projected directly onto the screen 40 and the pattern light projected onto the screen 40 through the object 50 (the shift or deformation of the pattern light) can be used to extract the transparent area of ​​the object 50, and it is possible to know the properties of the transparent area containing information about the refractive index.

[0065] Therefore, based on the image representing the patterned light reflected in the first photographic image I, it is possible to separate and extract the area (transparent area) of the object 50 through which the patterned light is transmitted from the background.

[0066] Furthermore, the transmittance of the transparent area of ​​the object 50 can be detected based on the ratio of the brightness value of an image representing patterned light transmitted through the object 50 to an image representing patterned light directly incident on the screen 40 without transmitting through the object 50. Moreover, the transmitted color of the transparent area of ​​the object 50 can be detected based on the difference in color information between an image representing patterned light transmitted through the object 50 and an image representing patterned light directly projected onto the screen 40.

[0067] Alternatively, the object 50 may not be placed on the rotary stage 30. In the absence of the object 50, a projection pattern based on patterned light projected onto the screen 40 can be obtained as a reference projection pattern. In this case, the object 50 can be placed on the rotary stage 30, and multiple first camera images with different shooting directions relative to the object 50 can be captured. Based on at least one of the multiple first camera images and the projection pattern reaching the screen 40, and the additionally captured reference projection pattern, the transparent area of ​​the object 50 transmitted through the projection pattern can be extracted, and at least one of the transmittance and transmitted color of the transparent area can be detected.

[0068] Figure 3 is a perspective view of the rotary table shown in Figure 1.

[0069] On the surface 30B of the rotary stage 30 shown in Figure 3, a pattern is provided that can be used for photogrammetry based on multiple first images with different shooting directions. In this example, a random point group pattern 32 with known positional relationships is provided.

[0070] By comparing the matching degree of the dot group patterns 32 projected in multiple first camera images I with different shooting directions, the rotation angle of the rotary table 30 can be detected, and the actual size of the object 50 can be determined based on the spacing of each point in the dot group pattern 32 projected in the first camera image I. Furthermore, the surface 30B of the rotary table 30 on which the object 50 is placed can be used as part of a screen (see Figure 2).

[0071] [Information processing device]

[0072] Figure 4 is a block diagram illustrating an embodiment of the hardware structure of the information processing device constituting the scanning system according to the present invention.

[0073] The information processing device 100 shown in Figure 4 can be built into the first camera device 10, and can be composed of a personal computer, workstation, etc., which are separate from the first camera device 10.

[0074] The information processing device 100 includes a processor 110, a memory 120, a display 130, an input / output interface 140, and an operation unit 150.

[0075] The processor 110, comprised of a CPU (Central Processing Unit) and the like, centrally controls all parts of the information processing device 100. It executes a scanning program to perform scanning imaging by the first imaging device 10, acquiring multiple first-camera images captured by the first imaging device 10 with different shooting directions relative to the object 50. The processor 110 performs information processing such as generating a 3D model of the object 50 based on the acquired multiple first-camera images, and detecting at least one of the transmittance and transmitted color of the transparent areas of the object 50 transmitted through illumination light (or a projection pattern such as a grid-like patterned light) from the illumination device 20. Further details regarding the information processing based on the processor 110 will be described later.

[0076] The memory 120 includes flash memory, ROM (Read-only Memory), RAM (Random Access Memory), hard disk drive, etc. The flash memory, ROM, or hard disk drive is a non-volatile memory that stores an operating system, various programs including the scanning program involved in this invention, etc. The scanning program may include a program that generates a 3D model of the object based on multiple first camera images with different shooting directions relative to the object.

[0077] Furthermore, non-volatile memory devices such as flash memory and hard disk drives can store multiple first camera images captured by the first camera device 10 with different shooting directions relative to the object, 3D models of the object generated from the multiple first camera images, and information about the projection pattern (in the case of a grid-shaped pattern light projected by the lighting device 20, information related to the shape of the pattern light).

[0078] RAM functions as the working area for processing based on processor 110. It also temporarily stores various programs and first camera images stored in flash memory or similar sources. Furthermore, processor 110 may have a portion of memory 120 (RAM) built into it.

[0079] In addition to displaying the operation screen of the information processing device 100, the display 130 can also display the first camera image and 3D model read from the memory 120.

[0080] The input / output interface 140 includes a connection section for connecting to external devices and a communication section for connecting to a network. The connection section for connecting to external devices can be compatible with USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface) (HDMI is a registered trademark), etc.

[0081] The information processing device 100 can be configured as a device independent of the first camera device 10. In this case, the processor 110 acquires multiple first camera images from the first camera device 10 via the input / output interface 140, each with a different shooting direction relative to the object. Alternatively, if the multiple first camera images are stored in a cloud drive, the processor 110 can acquire the multiple first camera images from the cloud drive via the input / output interface 140. Furthermore, the processor 110 can store the multiple first camera images thus acquired in the memory 120.

[0082] The operation unit 150 includes pointing devices such as a mouse and a keyboard, and uses the display screen of the display 130 to function as part of a GUI (Graphical User Interface) that accepts instruction input based on user operation.

[0083] In addition, when the information processing device 100 is included in the first camera device 10, the display 130 and the operation unit 150 correspond to the monitor, operation buttons, etc. on the back of the first camera device 10.

[0084] When the processor 110 of the information processing device 100 generates a 3D model of the object 50, the lighting device 20 illuminates the object 50 with grid-shaped patterned light and causes the turntable 30 to rotate at a predetermined speed, thereby causing the object 50 on the turntable 30 to rotate.

[0085] Furthermore, during the period when the rotary table 30 rotates once, the processor 110 captures multiple images of the object 50 by the first camera device 10. Since the object 50 is rotating, the first camera device 10 is able to capture multiple first camera images with different shooting directions relative to the object 50.

[0086] Furthermore, the driving of the rotary table 30 and the illumination of the patterned light from the lighting device 20 are not limited to being carried out by instructions issued by the processor 110, but can also be carried out by instructions issued by the user.

[0087] Furthermore, the scanning and imaging of the object 50 based on the first camera device 10 is not limited to being performed by receiving instructions from the processor 110, but can also be performed by instructions from the user. For example, the first camera device 10 can be set to continuous shooting (burst shooting), and the user can press and hold the shutter button during the rotation of the turntable 30 once.

[0088] If scanning and imaging of the object 50 based on the first imaging device 10 is performed, and multiple first images with different shooting directions relative to the object 50 are captured, the processor 110 acquires the multiple first images with different shooting directions relative to the object 50 from the first imaging device 10 via the input / output interface 140. If the multiple first images are stored in a cloud drive, the multiple first images can be acquired from the cloud drive via the input / output interface 140, or via the recording medium of the first imaging device 10.

[0089] [Scanning Method]

[0090] Figure 5 is a flowchart illustrating an embodiment of the scanning method according to the present invention, and in particular a flowchart illustrating the process from acquiring a plurality of first camera images by the processor 110 to generating a 3D model of the object 50.

[0091] In Figure 5, the processor 110 sets the parameter i, which represents a specific first camera image among a plurality of first camera images, to 1 (step S10).

[0092] Next, the processor 110 acquires multiple first camera images I with different shooting directions relative to the object 50 from the first camera device 10, and stores the acquired multiple first camera images I in the memory 120 (step S20). At this time, the processor 110 assigns numbers 1 to n corresponding to the shooting sequence to the multiple first camera images I, as first camera images (I1, I2, ..., In). In addition, n is a value corresponding to the number of multiple first camera images I.

[0093] The processor 110 obtains the first camera image Ii, represented by parameter i, from the first camera image (I1, I2, ..., In) (step S30). In addition, when the first camera image Ii is initially obtained, since i is set to 1 (refer to step S10), the first camera image I1 is obtained.

[0094] Next, the processor 110 extracts the transparent region of the object 50 based on the first camera image Ii acquired in step S20 (step S40). The extraction of the transparent region is based on the variation information of the illumination light (in this example, a grid-like patterned light) reaching the screen 40 to extract the transparent region of the object through which the patterned light passes. If the object is opaque or contains opaque areas, the patterned light does not pass through those opaque areas (opaque regions). On the other hand, regarding the patterned light, it passes through the transparent region of the object 50 and reaches the screen 40, but the patterned light directly projected onto the screen 40 and the patterned light that passes through the object 50 and is projected onto the screen 40 have different shifts or distortions, brightness, color, etc. This variation information of the patterned light can be used to extract the transparent region of the object 50, and the properties of the transparent region, including information about the refractive index, can be detected.

[0095] Therefore, the processor 110 can separate and extract the transparent area of ​​the object 50 through which the patterned light is transmitted from the background based on the image representing the patterned light reflected in the first captured image Ii. Furthermore, in the case of the grid-like patterned light in this example, it is easy to understand the grid-like changes caused by refraction of the patterned light at the boundary between the patterned light that transmits through the transparent area of ​​the object 50 and reaches the screen 40, and the patterned light that is directly projected onto the screen 40. This allows for more accurate and better extraction of the transparent area containing the outline of the transparent area of ​​the object 50.

[0096] Next, the processor 110 detects at least one of the transmittance and the transmitted color of the transparent region of the object 50 extracted in step S40. In this example, the transmittance and the transmitted color of the transparent region are detected. That is, the processor 110 detects the transmittance and the transmitted color of the object 50 based on the image information of the transparent region of the object 50 through which the patterned light is transmitted, contained in the first image Ii, and the image information of the region of the screen 40 through which the patterned light is directly incident without passing through the object 50 (not transmitted). For example, the transmittance of the transparent region of the object 50 can be detected based on the ratio of the brightness value of the image representing the patterned light that does not transmit through the object 50 to the brightness value of the image representing the patterned light that transmits through the object 50 to the screen 40, and the transmitted color of the transparent region of the object 50 can be detected based on the difference between the color information of the image representing the patterned light that transmits through the object 50 and the image representing the patterned light directly projected onto the screen 40.

[0097] Furthermore, the transmittance and color information of the transparent region of the object 50 can be calculated for each local area of ​​the transparent region, or as an average value of the entire transparent region. Moreover, when detecting the transmittance and transmitted color of the transparent region of the object 50 based on the first camera image Ii, the detection is performed considering an image captured in the first camera image containing patterned light that transmits (round trips) through the transparent region of the object 50 twice.

[0098] The processor 110 associates the information of the transparent area of ​​the object 50 detected by the first camera image Ii, the transmittance and the transmittance color with the first camera image Ii and stores it in the memory 120 (step S52).

[0099] Next, increment parameter i by 1 (step S60) and determine whether parameter i exceeds n (i > n) (step S70). If parameter i does not exceed n (i ≤ n), proceed to step S30 and repeat the process from step S30 to step S70. If parameter i exceeds n (i > n), proceed to step S80.

[0100] In step S80, the processor 110 uses information (contours) of transparent regions contained in a plurality of first camera images (I1, I2, ..., In) stored in the memory 120 to generate a 3D model of the transparent regions of the object 50 using a visual volume cross method based on the contours of the transparent regions of the object 50. Furthermore, either concurrently with or instead of generating a 3D model based on the visual volume cross method, a 3D model of the transparent regions of the object 50 is generated by photogrammetry based on information of the transparent regions contained in the plurality of first camera images (I1, I2, ..., In) (images extracted corresponding to the transparent regions).

[0101] Preferably, a texture corresponding to at least one of the transmittance and transmittance color of the transparent region is mapped onto the surface of the 3D model of the transparent region of the object 50 generated by the apparent volume cross method, and / or at least one of the transmittance and transmittance color of the transparent region is attached as auxiliary information of the 3D model of the transparent region.

[0102] Preferably, in the case where a 3D model of the transparent area of ​​the object 50 is generated by photogrammetry, the projection pattern corresponding to the pattern light is removed from the image of the extracted transparent area by image processing, or the image of the transparent area extracted from the photographic image when uniform illumination light is shone on the object 50 by the illumination device 20 instead of the pattern light is used.

[0103] The processor 110 stores the 3D model of the transparent area of ​​the object 50 thus generated in the memory 120.

[0104] Figure 6 is a partial view showing the main parts of the scanning system according to the second embodiment of the present invention.

[0105] In addition, in Figure 6, the same symbols are used to mark the same parts as in the first embodiment shown in Figure 1, and their detailed descriptions are omitted.

[0106] In the second embodiment shown in FIG6, the addition of a second camera device 70 and a third camera device 80 differs from the first embodiment shown in FIG1.

[0107] The second camera device 70 is a camera device that takes pictures of a screen 40 on the side opposite to the lighting device 20 of the object 50 from the same side as the object 50.

[0108] The second camera device 70 can capture images of grid-like patterned light that reaches the screen 40 through the object 50, patterned light that directly illuminates the screen 40 without being irradiated by the object 50, and images (outlines) representing the opaque areas when the object 50 has opaque areas, by photographing the screen 40 from its surface side.

[0109] The second camera device 70 does not directly photograph the object 50, but indirectly photographs multiple images with different shooting directions relative to the object 50 in order to photograph the screen 40 projected by the pattern light of the object 50 that rotates along with the rotation of the rotating stage 30.

[0110] Therefore, the processor 110 can generate a 3D model of the transparent area of ​​the object 50 based on multiple camera images captured by the second camera device 70, which have different shooting directions relative to the object 50, similar to the multiple first camera images captured by the first camera device 10, and detect the transmittance and transmitted color of the transparent area of ​​the object 50. Furthermore, preferably in the case where the 3D model of the object 50 is generated based on multiple camera images of patterned light reaching the screen 40, where the patterned light irradiation direction relative to the object 50 is substantially different, a 3D model of the transparent area of ​​the object 50 using information (outline) of the transparent area of ​​the object 50 is generated by the apparent volume cross method.

[0111] The third camera device 80 is a camera device that captures images of the screen 40 from the side opposite to the object 50. In this case, the screen 40 is a rear-projection type screen that diffuses and transmits light incident on the screen 40.

[0112] The third camera device 80 can capture an image of light diffused and transmitted through the screen 40 from its rear side, which may be grid-like pattern light that reaches the screen 40 through the object 50, pattern light that directly enters the screen 40 without passing through the object 50, etc.

[0113] The processor 110 is able to generate a 3D model of the transparent area of ​​the object 50 based on the multiple images captured by the third camera device 80, just as the multiple images captured by the second camera device 70, and detect the transmittance and transmittance color of the transparent area of ​​the object 50.

[0114] In addition, the scanning system of the second embodiment shown in FIG6 includes a second camera device 70 and a third camera device 80 in addition to the first camera device 10, but it is not limited to this. It may also be a system that only has a second camera device 70 or only has a third camera device 80.

[0115] Figure 7 is another example of a first image captured by the first camera device.

[0116] The first image shown in Figure 7 is an image obtained by the first camera device 10 (refer to Figure 1) capturing an object that is a sphere with an opaque region 52A and a transparent region 52B.

[0117] The processor 110 acquires multiple first camera images with different shooting directions relative to the sphere captured by the first camera device 10, and extracts the opaque region 52A and transparent region 52B of the sphere by separating them.

[0118] Furthermore, the processor 110 detects at least one of the transmittance and the transmitted color of the transparent region 53B of the sphere.

[0119] Then, the processor 110 generates a 3D model of the opaque region 52A of the sphere through photogrammetry based on multiple images taken from different shooting directions representing the opaque region 52A of the sphere. Additionally, a texture corresponding to the polygon is mapped onto a small region of the surface of the opaque region 52A of the 3D model.

[0120] Furthermore, the processor 110 generates a 3D model of the transparent region 52B of the sphere using a visual volume cross-referencing method based on multiple contours representing the transparent region 52B of the sphere with different shooting directions. The processor 110 is able to map a texture on the surface of the 3D model of the transparent region 52B that corresponds to at least one of the transmittance and transmittance color of the transparent region 52B.

[0121] These 3D models can be synthesized. Furthermore, a 3D model of the opaque region 52A of a sphere can be generated, the parts of the sphere surface without opaque texture can be set to transparent, and the texture can be mapped to at least one of the transmittance and transmittance color of the transparent region 52B.

[0122] Furthermore, it is preferable to remove the projection pattern corresponding to the grid-like pattern light contained in the texture of the 3D model through image processing, or to use an image extracted from the first photographic image when the pattern light is replaced by uniform illumination light irradiated onto the object 50 by the lighting device 20.

[0123] [other]

[0124] In this embodiment, multiple first camera images with different shooting directions relative to the object 50 are obtained by placing the object 50 on the rotating platform 30, rotating the rotating platform 30, and shooting the object 50 with the first camera device 10. However, it is not limited to this. Multiple first camera images with different shooting directions can also be obtained by rotating the first camera device 10 and the lighting device 20 relative to the stationary object 50 while shooting. In this case, the user can also hold the first camera device 10 and shoot while changing the shooting direction, shooting position, etc.

[0125] Furthermore, in this embodiment, for example, the hardware structure of the processing unit (processing unit) that performs various processes, such as a CPU (Central Processing Unit), is as shown below. These various processors include general-purpose processors that execute software (programs) to function as various processing units, such as CPUs; processors such as FPGAs (Field Programmable Gate Arrays) whose circuit structure can be changed after manufacturing, i.e., Programmable Logic Devices (PLDs); and processors such as ASICs (Application Specific Integrated Circuits) that have circuit structures specifically designed for performing specific processes, i.e., dedicated circuits.

[0126] A processing unit can be composed of one of these various processors, or it can be composed of two or more processors of the same or different types (e.g., multiple FPGAs or a combination of CPU and FPGA). Furthermore, multiple processing units can also be composed of a single processor. Examples of multiple processing units composed of a single processor include: first, in computers such as client or server computers, a single processor is composed of a combination of one or more CPUs and software, and this processor functions as multiple processing units. Second, in systems-on-a-chip (SoC), a processor is used to implement the functions of the entire system containing multiple processing units using a single IC (Integrated Circuit) chip. Thus, various processing units are constructed as hardware structures using one or more of the aforementioned processors.

[0127] Furthermore, more specifically, the hardware structure of these various processors is a circuit composed of circuit elements such as semiconductor components.

[0128] Furthermore, the present invention includes a scanning program that, when installed on a computer, enables the computer to execute the scanning method involved in the present invention, and a non-transitory and computer-readable recording medium on which the program is recorded.

[0129] Moreover, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention.

[0130] Symbol Explanation

[0131] 10-First camera device, 20-Lighting device, 30-Rotating platform, 32-Dot pattern, 40-Screen, 50-Object, 60-Ground, 60A-Area, 70-Second camera device, 80-Third camera device, 100-Information processing device, 110-Processor, 120-Memory, 130-Display, 140-Input / output interface, 150-Operation unit, L1, L2-Optical axes, S10~S80-Steps.

Claims

1. A scanning system comprising: an illumination device for illuminating an object with illumination light; a first imaging device for capturing images of the object illuminated by the illumination device; and a processor, wherein, The processor performs the following processing: acquiring multiple first camera images from the first camera device with different shooting directions relative to the object; and detecting at least one of the transmittance and transmitted color of the area of ​​the object through which the illumination light is transmitted, based on the multiple first camera images.

2. The scanning system according to claim 1, wherein, The processor extracts the area of ​​the object through which the illumination light is transmitted, based on information about the changes in the illumination light reaching the screen on the side opposite to the object and the illumination device.

3. The scanning system according to claim 2, wherein, The processor detects at least one of the transmittance and the transmitted color based on image information of the area of ​​the object through which the illumination light is transmitted in the first camera image and image information of the area of ​​the screen directly incident on without passing through the object.

4. The scanning system according to claim 1, wherein, The lighting device includes a projection device for projecting a projection pattern, and the processor extracts the area of ​​the object through which the projection pattern is transmitted, based on the change information of the projection pattern reaching a screen on the side opposite to the projection device across the object.

5. The scanning system according to claim 1, wherein, The processor generates a 3D model of the object by using a visual volume cross method based on the outline of the object contained in multiple first camera images with different shooting directions.

6. The scanning system according to claim 5, wherein, The processor performs the following processing: generating a 3D model of the object by photogrammetry based on multiple first camera images with different shooting directions.

7. The scanning system according to claim 6, wherein, The surface of the 3D model is mapped with textures generated using multiple of the first camera images.

8. The scanning system according to claim 5, wherein, The surface of the object region of the 3D model to which the illumination light is transmitted is mapped with a texture corresponding to at least one of the transmittance and the transmitted color, or at least one of the transmittance and the transmitted color is attached as supplementary information of the 3D model.

9. The scanning system according to claim 1, wherein, The lighting device includes a projection device for projecting a projection pattern, and the processor performs the following processing: based on the projection pattern contained in the first camera image captured by the first camera device and reaching the screen on the side opposite to the projection device across the object, extract the area of ​​the object transmitted by the projection pattern, or based on the projection pattern on the screen and the projection pattern on the object, extract the area of ​​the object transmitted by the projection pattern.

10. The scanning system according to claim 1, comprising a second camera device or a third camera device, the second camera device capturing images of the screen from the same side as the object relative to a screen on the side opposite to the illumination device across the object, the third camera device capturing images of the screen from the side opposite to the object across the screen, and the processor detecting, based on the captured images by the second camera device or the third camera device, at least one of the transmittance and the transmitted color of the area of ​​the object through which the illumination light reaching the screen is transmitted.

11. The scanning system according to claim 1, wherein, The lighting device includes a projection device for projecting a projection pattern. The processor performs the following processing: acquiring the projection pattern that reaches the screen on the side opposite to the projection device across the object and is projected onto the screen when the object is not present, as a reference projection pattern; and extracting the area of ​​the object transmitted by the projection pattern based on at least one of the first camera image and the projection pattern reaching the screen, and the reference projection pattern.

12. The scanning system according to claim 1, comprising a rotating stage for placing the object, wherein the plurality of first camera images are images captured by the first camera device at different rotational positions of each of the rotating stages.

13. The scanning system according to claim 12, wherein, The surface of the rotating platform functions as part of a screen on the side opposite to the lighting device, separated by the object.

14. The scanning system according to claim 13, wherein, The surface of the rotary table is provided with a pattern that can be used for photogrammetry of multiple first camera images based on different shooting directions.

15. A scanning method, which is a scanning method in a scanning system, the scanning system comprising: an illumination device for illuminating an object with illumination light; a first camera device for capturing an image of the object illuminated by the illumination device; and a processor, wherein... The scanning method includes the following steps: the processor acquires multiple first camera images from the first camera device with different shooting directions relative to the object; and the processor detects at least one of the transmittance and transmitted color of the area of ​​the object through which the illumination light is transmitted, based on the multiple first camera images.

16. The scanning method according to claim 15, comprising the step of: the processor generating a 3D model of the object by means of a visual volume cross method based on the outline of the object contained in a plurality of the first camera images with different shooting directions.

17. The scanning method according to claim 15 or 16, comprising the step of: the processor generating a 3D model of the object by photogrammetry based on a plurality of the first camera images with different shooting directions.

18. A scanning program that enables a computer to perform the following functions: to enable a first camera device to capture multiple first camera images of an object illuminated by an illumination device and to capture multiple first camera images with different shooting directions relative to the object; to acquire the multiple first camera images captured by the first camera device; and to detect, based on the multiple first camera images, at least one of the transmittance and the transmitted color of the area of ​​the object through which the illumination light is transmitted.

19. The scanning procedure of claim 18, comprising at least one of the following functions: generating a 3D model of the object by means of a visual volume cross method based on the outline of the object contained in a plurality of the first camera images with different shooting directions; and generating a 3D model of the object by means of photogrammetry based on a plurality of the first camera images with different shooting directions.

20. A recording medium that is non-transitory and computer-readable, and which records the program as described in claim 18 or 19.

Images