Three-dimensional model generating device, three-dimensional model generating method, and three-dimensional model generating program

Abstract

To generate a three-dimensional model by appropriately processing optical surfaces included in a plurality of images.SOLUTION: A three-dimensional model generation apparatus comprises: an image acquisition unit for acquiring a plurality of images captured from a plurality of shooting positions; an optical surface detection unit for detecting an optical surface area in which at least one of a reflected image that is visually recognized by reflection of light and a transmissive visual object that is visually recognized through a transparent member is reflected among the plurality of acquired images; and a model generation unit for placing a mask on the optical surface area and generating a three-dimensional model based on a plurality of images in which the mask is placed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a three-dimensional model generating device, a three-dimensional model generating method, and a three-dimensional model generating program. [Background technology]

[0002] BACKGROUND ART Photogrammetry is a known technique in which a plurality of images are taken while changing the photographing position relative to a subject, and a three-dimensional model is generated based on the plurality of pieces of image data taken (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special table 2006-528381 publication Summary of the Invention [Problem to be solved by the invention]

[0004] When a subject has optical surfaces that reflect the surrounding image, such as reflective surfaces like mirrors or transparent surfaces like windows, a 3D model may be generated as if there is a space behind the optical surfaces, or the part corresponding to the optical surfaces may be broken. Thus, photogrammetry requires that the optical surfaces included in the image be appropriately processed to generate a 3D model.

[0005] The present invention has been made in consideration of the above, and aims to provide a three-dimensional model generation device, a three-dimensional model generation method, and a three-dimensional model generation program that are capable of appropriately processing optical surfaces included in an image to generate a three-dimensional model. [Means for solving the problem]

[0006] The three-dimensional model generation device of the present invention includes an image acquisition unit that acquires multiple images taken from multiple shooting positions, an optical surface detection unit that detects an optical surface area in the multiple acquired images that reflects at least one of a reflected image visible due to the reflection of light and a transmitted visual object visible through a transparent member, and a model generation unit that places a mask in the optical surface area and generates a three-dimensional model based on the multiple images with the mask placed on them.

[0007] The three-dimensional model generation method according to the present invention includes acquiring a plurality of images taken from a plurality of shooting positions, detecting an optical surface area among the acquired images that reflects at least one of a reflected image visible due to the reflection of light and a transmitted visible object visible through a transparent member, placing a mask on the optical surface area, and generating a three-dimensional model based on the plurality of images with the mask placed on them.

[0008] The three-dimensional model generation program of the present invention includes a process of acquiring a plurality of images taken from a plurality of shooting positions, a process of detecting an optical surface area from the acquired plurality of images that reflects at least one of a reflected image visible due to the reflection of light and a transmitted visible object visible through a transparent member, and a process of placing a mask on the optical surface area and generating a three-dimensional model based on the plurality of images with the mask placed on it. [Effects of the Invention]

[0009] According to the present invention, optical surfaces included in a plurality of images can be appropriately processed to generate a three-dimensional model. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram schematically illustrating an example of a three-dimensional model generating device according to this embodiment. [Figure 2] FIG. 2 is a functional block diagram illustrating an example of a three-dimensional model generating device. [Figure 3] FIG. 3 is an explanatory diagram showing the positional relationship between two images to which the principles of photogrammetry are applied. [Figure 4] FIG. 4 is an explanatory diagram showing the positional relationship between the two images. [Figure 5] FIG. 5 is a diagram showing how a three-dimensional space is photographed. [Figure 6] FIG. 6 is a diagram showing an example of a plurality of images captured in a three-dimensional space. [Figure 7] FIG. 7 is a diagram showing an example of a state in which masks are arranged on a plurality of images. [Figure 8] FIG. 8 is a flowchart showing an example of a three-dimensional model generating method according to this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of a 3D model generation device, a 3D model generation method, and a 3D model generation program according to the present invention will be described with reference to the accompanying drawings. Note that the present invention is not limited to these embodiments. Furthermore, the components in the following embodiments include those that are easily replaceable by those skilled in the art, or those that are substantially identical.

[0012] FIG. 1 is a diagram schematically illustrating an example of a three-dimensional model generation device 100 according to this embodiment. FIG. 2 is a functional block diagram illustrating an example of the three-dimensional model generation device 100. The three-dimensional model generation device 100 illustrated in FIGS. 1 and 2 generates a three-dimensional model based on the principles of photogrammetry. As illustrated in FIGS. 1 and 2, the three-dimensional model generation device 100 includes a processing unit 10 and a storage unit 20.

[0013] The processing unit 10 includes a processing device such as a CPU (Central Processing Unit) and a storage device such as a RAM (Random Access Memory) or a ROM (Read Only Memory), etc. The processing unit 10 also includes an image acquisition unit 11, an optical surface detection unit 12, a color attribute detection unit 13, an area detection unit 14, and a model generation unit 15.

[0014] The image acquisition unit 11 acquires a plurality of images I captured from a plurality of shooting positions. Each image I is an image captured by a shooting device such as a camera CR (C1, C2, etc.).

[0015] The optical surface detection unit 12 detects optical surface regions included in the multiple acquired images. In this embodiment, the optical surface region is, for example, a region in an image where an image of the surroundings is reflected, and includes at least one of a reflective surface region where a reflected image visible due to light reflection is reflected and a transparent surface region where a transmitted object visible through a transparent member is reflected. The optical surface detection unit 12 can detect optical surface regions included in the image using a known method. For example, a predetermined pattern is displayed toward the three-dimensional space K using a display device, and the pattern is moved in one direction, and the three-dimensional space is captured in this state. The optical surface detection unit 12 detects whether there is a region in the captured image where the pattern movement is inverted or where the pattern movement is not uniform. If the optical surface detection unit 12 detects a region where the pattern movement is inverted, the optical surface detection unit 12 can determine the region as a reflective surface region. Furthermore, if the optical surface detection unit 12 detects a region where the pattern movement is not uniform, the optical surface detection unit 12 can determine the region as a transparent surface region. The specific method by which the optical surface detection unit 12 detects the optical surface region is not limited to the above and may be other methods.

[0016] The reflective surface region in this embodiment includes a region where the color of the underlying material appears to be superimposed on the color of the image reflected on the mirror surface, such as a non-metallic material with a mirror-finished surface or a chromatic metal material such as gold or copper, and a region where the color of the image reflected on the mirror surface appears as is, such as a colorless metal material with a mirror-finished surface. The transmissive surface region in this embodiment includes, for example, the surface of a light-transmitting material that transmits light, such as a glass plate. The transmissive surface region includes a chromatic light-transmitting material and an achromatic light-transmitting material.

[0017] The color attribute detection unit 13 detects the color attributes of the optical surface region. In this embodiment, the color attributes include the three so-called color attributes: hue, saturation, and lightness. The color attribute detection unit 13 detects the color attributes of the optical surface region, for example, by image processing. By detecting the color attributes of the optical surface region, it is possible to determine the tendency of the color attributes of the optical surface region. The color attribute detection unit 13 can detect the hue, saturation, and lightness that constitute the color attributes of the optical surface region as numerical values ​​such as coordinates in a color space.

[0018] The area detection unit 14 detects the area of ​​the optical surface region. The area detection unit 14 can detect, for example, the number of pixels corresponding to the optical surface region detected in the image I as the area of ​​the optical surface region.

[0019] The model generation unit 15 generates a three-dimensional model based on multiple images acquired by the image acquisition unit 11. The model generation unit 15 can generate a three-dimensional model based on, for example, the principle of photogrammetry. Here, the principle of photogrammetry will be explained. Below, a case where three-dimensional image data is generated from two pieces of image data will be explained. FIG. 3 is an explanatory diagram showing the positional relationship between two images to which the principle of photogrammetry is applied, and FIG. 4 is an explanatory diagram showing the positional relationship between the two images.

[0020] The model generating unit 15 extracts, for example, two pieces of image data whose positions indicated by the position data are the same. Note that the positions being the same do not necessarily have to be exactly the same, and pieces whose positions are shifted by a predetermined amount may also be considered to be the same.

[0021] First, two sets of image data of the object are obtained by a camera C1 for an image of the field of view and a camera C2 for an image of the field of view (see FIG. 3 for both). Next, the model generation unit 15 searches for corresponding points of feature points based on the two sets of image data. The model generation unit 15, for example, performs correspondence for each pixel and searches for the position where the difference is smallest. Here, as shown in FIG. 3, the cameras C1 and C2, which are assumed to exist simultaneously at two viewpoints, are arranged in a relationship of Yl = Yr so that the optical axes Ol and Or are included on the same XZ coordinate plane. Using the corresponding points searched for by the model generation unit 15, a disparity vector corresponding to the angle difference for each pixel is calculated.

[0022] Since the obtained disparity vector corresponds to the distance from cameras C1 and C2 in the depth direction, the model generation unit 15 calculates the distance in proportion to the magnitude of the disparity using perspective. Assuming that the photographer's cameras C1 and C2 only move horizontally, by positioning cameras C1 and C2 so that their optical axes Ol and Or are included in the same XZ coordinate plane, the search for corresponding points can be performed only on the scanning lines, which are epipolar lines Epl and Epr. The model generation unit 15 generates three-dimensional image data of the object using two pieces of image data of the object and the respective distances from cameras C1 and C2 to the object. The model generation unit 15 may store the generated three-dimensional image data in, for example, a storage unit 20, or may output or transmit it to an external device via an output unit or communication unit (not shown).

[0023] On the other hand, when point Ql (Xl, Yl) on the left image corresponds to point Qr (Xr, Yr) on the right image, the disparity vector at point Ql (Xl, Yl) is Vp (Xl-Xr, Yl-Yr). Here, since the two points Ql and Qr are on the same scanning line (epipolar line), Yl = Yr, and the disparity vector is expressed as Vp (Xl-Xr, 0). The model generation unit 15 obtains such disparity vector Vp for all pixel points on the image and creates a group of disparity vectors to obtain information on the depth direction of the image. Incidentally, for a set in which the epipolar line is not horizontal, the height of one of the camera positions may be different (with a low probability). In this case, the model generation unit 15 searches within a rectangle in the epipolar line direction and in a direction perpendicular to the epipolar line, which is approximately the deviation from the horizontal, compared to a case where a large search range is searched for corresponding points in a large two-dimensional area without considering corresponding point matching. The amount of calculation required for the minimum rectangle is reduced, making the process more rational. As shown in FIG. 4, the model generation unit 15 shows a search range in which the epipolar line direction search range for the minimum rectangle is a-b = c-d, and the perpendicular direction search range is b-c = d-a. In this case, the search width in the epipolar line direction is ΔE, and the search width in the direction T perpendicular to the epipolar line is ΔT. The smallest non-inclined rectangle ABCD that contains the minimum inclined rectangle abcd is the desired region.

[0024] In this way, the model generation unit 15 calculates the parallax vector from the corresponding feature points of the multiple cameras C1 and C2 under the epipolar constraint condition, obtains the depth direction information of each point, maps the texture on the surface of the three-dimensional shape, and generates three-dimensional image data. As a result, the model of the part in the image data used for calculation can reproduce the space as seen from the front hemisphere. Furthermore, if there is a part not captured in the image data of the three-dimensional image data, and if the part is connected by extending the lines or surfaces of the surrounding texture, the part in between is interpolated using the same texture.

[0025] The method for generating the three-dimensional image data is not limited to the above, and other methods may be used.

[0026] When an optical surface region is included in a plurality of images I, the model generation unit 15 places a mask covering the optical surface region on the optical surface region, and generates a three-dimensional model based on the plurality of images I on which the mask is placed. The model generation unit 15 can place a mask of a color corresponding to the color of the optical surface region on the optical surface region. For example, when the color attribute of the optical surface region detected by the color attribute detection unit 13 has a predetermined tendency, the model generation unit 15 can generate and place a mask (hereinafter referred to as a corresponding mask) having a color attribute corresponding to the color attribute of the optical surface region. On the other hand, when the color attribute of the optical surface region detected by the color attribute detection unit 13 does not have a predetermined tendency, the model generation unit 15 can place a preset mask (hereinafter referred to as a standard mask).

[0027] For example, if the values ​​indicating the hue, saturation, and brightness of the optical surface region are distributed within a predetermined range in the color space at a rate equal to or greater than a threshold, the model generation unit 15 can determine that the color attributes of the optical surface region have a predetermined tendency. In this embodiment, if the color attributes of the optical surface region have a predetermined tendency, the optical surface region is chromatic. If the color attributes of the optical surface region do not have a predetermined tendency, the optical surface region is achromatic or close to achromatic.

[0028] When generating a corresponding mask corresponding to the color attribute of the optical surface region, the model generation unit 15 can set the color attribute of the corresponding mask to, for example, a color attribute corresponding to a peak value in a distribution of values ​​indicating the color attribute of the optical surface region. This allows a mask of a color corresponding to the color of the optical surface region to be placed on the optical surface region, thereby reducing the sense of incongruity felt by the observer. In this case, the model generation unit 15 may, for example, add a mark indicating gloss to the corresponding mask. In this case, the observer can recognize that the region of the corresponding mask is an optical surface region.

[0029] If the color attributes of the optical surface region do not conform to a predetermined trend, the model generation unit 15 places a preset standard mask. In this case, the standard mask may be, for example, a mask that resembles the reflective surface of an achromatic mirror.

[0030] Furthermore, if the area of ​​the optical surface region detected by the area detection unit 14 is less than a predetermined value, the model generation unit 15 may place a standard mask on the optical surface region regardless of the color attribute of the optical surface region. If the area of ​​the optical surface region is small, it is estimated that the observer will not feel uncomfortable even if a corresponding mask of a color corresponding to the color of the optical surface region is not placed. In this case, the process of setting the color attribute, etc. of the corresponding mask can be omitted.

[0031] The model generating unit 15 may always place the standard mask on the optical surface region regardless of the color attribute of the optical surface region.

[0032] The storage unit 20 stores various types of information. The storage unit 20 stores information about a preset standard mask. The storage unit 20 has a storage such as a hard disk drive or a solid state drive. Note that an external storage medium such as a removable disk may be used as the storage unit 20.

[0033] The memory unit 20 stores a three-dimensional model generation program that causes a computer to execute the following processes: acquiring multiple images I taken from multiple shooting positions; detecting an optical surface area, from the acquired multiple images I, that reflects at least one of a reflected image visible due to the reflection of light and a transmitted visible object visible through a transparent member; and placing a mask on the optical surface area and generating a three-dimensional model based on the multiple images with the mask placed on it.

[0034] Next, the operation of the three-dimensional model generating device 100 configured as described above will be described. Fig. 5 is a diagram showing how a three-dimensional space K is photographed. Fig. 6 is a diagram showing an example of a plurality of images photographed from the three-dimensional space K. Fig. 7 is a diagram showing an example of a state in which masks are arranged on a plurality of images.

[0035] First, as shown in FIG. 5, a three-dimensional space K is photographed from different photographing positions. The image acquisition unit 11 acquires multiple photographed images. Here, an example in which two images I1 and I2 are acquired as shown in FIG. 6 will be described. However, the number of images may be three or more. Assume that objects that reflect surrounding images are arranged in the three-dimensional space K, such as chromatic (e.g., black) resin members 41 and 42 constituting a home appliance such as a television or a rice cooker, an achromatic metal member 43 constituting the reflective surface of a back-surface mirror, and an achromatic, transparent glass member 44 constituting a window, as shown in FIG. 5. For example, reflected images 41r and 42r are reflected in the resin members 41 and 42. Furthermore, a reflected image 43r is reflected in the metal member 43. A reflected image is an image visually recognized by the reflection of light. Furthermore, a through-view object 44t, such as a cloud or a building located in the background, is reflected in the glass member 44. A through-view object is an object visually recognized through a transparent member such as a glass member.

[0036] By capturing an image of this three-dimensional space K, the resin members 41 and 42, the metal member 43, and the glass member 44 appear as optical surface areas 51a, 52a, 53a, and 54a in the captured image I1, as shown in Fig. 6. Furthermore, the resin members 41 and 42, the metal member 43, and the glass member 44 appear as optical surface areas 51b, 52b, 53b, and 54b in the captured image I2.

[0037] When an optical surface area is included in the acquired images I1 and I2, the optical surface detection unit 12 detects the optical surface area. In this embodiment, the optical surface detection unit 12 can detect optical surface areas 51a, 52a, 53a, and 54a included in image I1 and optical surface areas 51b, 52b, 53b, and 54b included in image I2.

[0038] The optical surface regions 51a-54a and 51b-54b reflect reflected images 51r-53r or a through-view object 54t, respectively. If a three-dimensional model is generated in this state, the three-dimensional model may be generated as if the reflected images 41r-43r and the through-view object 44t were actually structures located deep inside the optical surface regions, or the three-dimensional model may be generated in such a way that the portion corresponding to the optical surface region is broken. Therefore, in this embodiment, the optical surface regions are appropriately processed by performing the following processing to generate a three-dimensional model.

[0039] The color attribute detection unit 13 detects the color attributes of the optical surface regions 51a, 52a, 53a, and 54a included in the image I1 and the optical surface regions 51b, 52b, 53b, and 54b included in the image I2. The area detection unit 14 detects the areas of the optical surface regions 51a, 52a, 53a, and 54a included in the image I1 and the optical surface regions 51b, 52b, 53b, and 54b included in the image I2.

[0040] The model generation unit 15 determines whether the areas of the optical surface regions 51a, 52a, 53a, and 54a and the optical surface regions 51b, 52b, 53b, and 54b are less than a predetermined value. In this embodiment, the model generation unit 15 determines that the areas of the optical surface regions 51a, 53a, and 54a in image I1 and the optical surface regions 51b, 53b, and 54b in image I2 are equal to or greater than a predetermined value. The model generation unit 15 also determines that the areas of the optical surface region 52a in image I1 and the optical surface region 52b in image I2 are less than a predetermined value. For the optical surface region 52a in image I1 and the optical surface region 52b in image I2 whose areas are determined to be less than the predetermined value, the model generation unit 15 places a standard mask M2 as shown in FIG. 7 , regardless of the color attribute determined below.

[0041] The model generation unit 15 determines whether the color attributes of the optical surface regions 51a, 53a, and 54a and the optical surface regions 51b, 53b, and 54b have a predetermined tendency. In this embodiment, the model generation unit 15 determines that the color attributes of the optical surface region 51a of image I1 and the optical surface region 51b of image I2, for example, have a predetermined tendency. For the optical surface regions 51a and 51b whose color attributes have been determined to have a predetermined tendency, the model generation unit 15 places a corresponding mask M1 corresponding to the color attribute, as shown in FIG. 7 .

[0042] Furthermore, the model generation unit 15 determines that the color attributes of the optical surface regions 53a and 54a of the image I1 and the optical surface regions 53b and 54b of the image I2 do not conform to a predetermined trend, for example. The model generation unit 15 arranges standard masks M3 and M4, as shown in FIG. 7, for the optical surface regions 53a, 53b, 54a, and 54b whose color attributes are determined not to conform to a predetermined trend.

[0043] Fig. 8 is a flowchart showing an example of a three-dimensional model generation method according to this embodiment. As shown in Fig. 8, the image acquisition unit 11 acquires a plurality of images obtained by capturing a three-dimensional space K from different capturing positions (step S10). The optical surface detection unit 12 detects an optical surface area included in the acquired plurality of images (step S20). The color attribute detection unit 13 detects a color attribute of the optical surface area (step S30). The area detection unit 14 detects the area of ​​the optical surface area (step S40).

[0044] The model generation unit 15 determines whether the area of ​​the optical surface region is less than a predetermined value (step S50). If it is determined that the area of ​​the optical surface region is less than the predetermined value (Yes in step S50), the model generation unit 15 arranges a standard mask for the optical surface region (step S60).

[0045] If the model generation unit 15 determines that the area of ​​the optical surface region is not less than a predetermined value (No in step S50), it determines whether the color attribute of the optical surface region has a predetermined tendency (step S70). If the model generation unit 15 determines that the color attribute of the optical surface region has a predetermined tendency (Yes in step S70), it generates a corresponding mask having a color attribute corresponding to the color attribute of the optical surface region and places it on the optical surface region (step S80). On the other hand, if the model generation unit 15 determines that the color attribute of the optical surface region does not have a predetermined tendency (No in step S70), it places a standard mask on the optical surface region (step S60).

[0046] After the mask is placed in step S60 or step S80, a three-dimensional model is generated based on the multiple images in which the mask is placed (step S90).

[0047] As described above, the three-dimensional model generation device 100 according to this embodiment includes an image acquisition unit 11 that acquires multiple images taken from multiple shooting positions, an optical surface detection unit 12 that detects, from the multiple acquired images, an optical surface area that contains at least one of a reflected image visible due to the reflection of light and a transmitted visual object visible through a transparent member, and a model generation unit 15 that places a mask in the optical surface area and generates a three-dimensional model based on the multiple images with the mask placed on it.

[0048] In addition, the three-dimensional model generation method according to this embodiment includes acquiring a plurality of images taken from a plurality of shooting positions, detecting an optical surface area from the acquired plurality of images that reflects at least one of a reflected image visible due to the reflection of light and a transmitted visible object visible through a transparent member, placing a mask on the optical surface area, and generating a three-dimensional model based on the plurality of images in which the mask is placed.

[0049] In addition, the three-dimensional model generation program according to this embodiment causes a computer to execute the following processes: acquiring multiple images taken from multiple shooting positions; detecting an optical surface area from the acquired multiple images that reflects at least one of a reflected image visible due to the reflection of light and a transmitted visible object visible through a transparent member; and placing a mask on the optical surface area and generating a three-dimensional model based on the multiple images with the mask placed on it.

[0050] According to this configuration, optical surface areas contained in multiple images are detected, a mask is placed, and a three-dimensional model is generated based on the multiple images in which the mask is placed.Therefore, even if an optical surface area is contained in multiple images, the optical surface area can be appropriately processed to generate a three-dimensional model.

[0051] In the three-dimensional model generating device 100 according to this embodiment, the model generating unit 15 places a mask corresponding to the color of the optical surface region on the optical surface region. With this configuration, for optical surface regions of chromatic colors, a mask corresponding to the color of the optical surface region is placed, thereby reducing the sense of discomfort felt by the viewer.

[0052] The three-dimensional model generating device 100 according to this embodiment further includes a color attribute detection unit 13 that detects the color attributes of the optical surface region, and the model generating unit 15 generates and places a corresponding mask having color attributes corresponding to the color attributes of the optical surface region when the color attributes of the optical surface region follow a predetermined trend, and places a preset standard mask when the color attributes of the optical surface region do not follow the predetermined trend. With this configuration, the corresponding mask and the standard mask can be used selectively depending on the color attributes of the optical surface region, thereby more reliably reducing the sense of discomfort felt by the viewer.

[0053] The 3D model generating device 100 according to this embodiment further includes an area detection unit 14 that detects the area of ​​the optical surface region, and when the area of ​​the optical surface region is less than a predetermined value, a preset standard mask is placed regardless of the color attribute of the optical surface region. With this configuration, when the area of ​​the optical surface region is less than the predetermined value, the process of setting the color attribute, etc. of the corresponding mask can be omitted.

[0054] The technical scope of the present invention is not limited to the above-described embodiment, and appropriate modifications can be made without departing from the spirit of the present invention. For example, in the above-described embodiment, the model generation unit 15 has determined that the color attributes of the optical surface regions 54a and 54b corresponding to the glass member 44, such as window glass, do not have a predetermined tendency. However, the present invention is not limited to this. Depending on the scenery (image) behind the light-transmitting region, such as window glass, the color attributes may have a predetermined tendency, for example, when blue sky is visible. In such a case, the model generation unit 15 can determine that the color attributes of the optical surface regions 54a and 54b have a predetermined tendency.

[0055] In the above embodiment, the same standard mask is used for both the reflective surface area and the transmissive surface area, but this is not limiting. The optical surface detection unit 12 may detect the reflective surface area and the transmissive surface area separately. In this case, the model generation unit 15 may apply the standard mask for the reflective surface area and the standard mask for the transmissive surface area separately. [Explanation of symbols]

[0056] CR, C1, C2...camera, I, I1, I2...image, K...three-dimensional space, M1...corresponding mask, M2, M3, M4...standard mask, 10...processing unit, 11...image acquisition unit, 12...optical surface detection unit, 13...color attribute detection unit, 14...area detection unit, 15...model generation unit, 20...storage unit, 41, 42...resin member, 43...metal member, 41r, 42r, 43r...reflected image, 44...glass member, 44t...transmitted visible object, 51a, 51b, 51c, 52a, 52b, 53a, 53b, 53c, 54a, 54b...optical surface area, 100...three-dimensional model generation device

Claims

1. an image acquisition unit that acquires a plurality of images taken from a plurality of shooting positions; an optical surface detection unit that detects an optical surface area in which at least one of a reflected image visually recognized by reflection of light and a transmitted-viewable object visually recognized through a transparent member is reflected, among the plurality of acquired images; a model generation unit that places a mask on the optical surface region and generates a three-dimensional model based on the plurality of images on which the mask is placed; a color attribute detection unit that detects a color attribute of the optical surface region; Equipped with The model generation unit generates and arranges a mask having a color attribute corresponding to the color attribute of the optical surface region when the color attribute of the optical surface region has a predetermined tendency, arranges a mask of a predetermined reflective surface region when the color attribute of the optical surface region does not have the predetermined tendency and the optical surface region is a reflective surface region on which a reflected image visually recognized by reflection of light is projected, and arranges a mask of a predetermined transparent surface region when the color attribute of the optical surface region does not have the predetermined tendency and the optical surface region is a transparent surface region on which a transmitted object visually recognized through a transparent member is projected. 3D model generation device.

2. The color attributes of the optical surface area have a predetermined tendency when the values ​​indicating the hue, saturation, and brightness of the optical surface area are distributed at a rate equal to or greater than a predetermined threshold value. The three-dimensional model generating device according to claim 1 .

3. The mask having the color attribute corresponding to the color attribute of the optical surface region has a color attribute corresponding to a peak value of a distribution of values ​​indicating the color attribute of the optical surface region.

3. The three-dimensional model generating device according to claim 1.

4. further comprising an area detection unit that detects the area of ​​the optical surface region; When the area of ​​the optical surface region is less than a predetermined value, a preset mask is arranged regardless of the color attribute of the optical surface region. The three-dimensional model generating device according to claim 1 .

5. Obtaining a plurality of images taken from a plurality of image positions; Detecting an optical surface area in which at least one of a reflected image visually recognized by reflection of light and a transmitted visual object visually recognized through a transparent member is reflected from the plurality of acquired images; placing a mask on the optical surface region and generating a three-dimensional model based on the plurality of images with the mask placed on them; detecting color attributes of the optical surface region; Including, In generating the three-dimensional model, if the color attribute of the optical surface region has a predetermined tendency, a mask having a color attribute corresponding to the color attribute of the optical surface region is generated and placed; if the color attribute of the optical surface region does not have the predetermined tendency and the optical surface region is a reflective surface region on which a reflected image visually recognized by reflection of light is projected, a mask of a preset reflective surface region is placed; and if the color attribute of the optical surface region does not have the predetermined tendency and the optical surface region is a transmissive surface region on which a transmitted object visually recognized through a transparent member is projected, a mask of a preset transmissive surface region is placed. A method for generating three-dimensional models.

6. acquiring a plurality of images taken from a plurality of positions; A process of detecting an optical surface area in which at least one of a reflected image visually recognized by reflection of light and a transmitted visual object visually recognized through a transparent member is reflected from the plurality of acquired images; placing a mask on the optical surface region and generating a three-dimensional model based on the plurality of images with the mask placed on the optical surface region; detecting color attributes of the optical surface region; on the computer, In the process of generating the three-dimensional model, if the color attribute of the optical surface region has a predetermined tendency, a mask having a color attribute corresponding to the color attribute of the optical surface region is generated and placed; if the color attribute of the optical surface region does not have the predetermined tendency and the optical surface region is a reflective surface region on which a reflected image visually recognized by reflection of light is projected, a mask of a preset reflective surface region is placed; and if the color attribute of the optical surface region does not have the predetermined tendency and the optical surface region is a transmissive surface region on which a transmitted object visually recognized through a transparent member is projected, a mask of a preset transmissive surface region is placed. A three-dimensional model generation program.

Images