Three-dimensional model generation apparatus, three-dimensional model generation method, and three-dimensional model generation program

Abstract

A three-dimensional model is generated by appropriately processing the optical surfaces contained in multiple images. [Solution] The three-dimensional model generation device acquires multiple images taken from multiple shooting positions. An image acquisition unit, and among the multiple acquired images, a reflected image and a transparent image that are visible due to light reflection. Optical detection of an optical surface region on which at least one of the transmitted objects visible through a light-emitting element is reflected. A surface detection unit places a mask on the optical surface area, and based on multiple images with the mask placed on them, a tertiary detection is performed. It includes a model generation unit that generates the original model.

Description

Technical Field

[0001] The present invention relates to a three-dimensional model generation apparatus, a three-dimensional model generation method, and a three-dimensional model generation program.

Background Art

[0002] A technique called photogrammetry is known, in which a plurality of images are taken while changing the shooting position with respect to a subject, and a three-dimensional model is generated based on the plurality of captured image data (for example, see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a case where an optical surface that reflects an image of the surroundings such as a reflecting surface such as a mirror or a transmitting surface such as a window is provided in a subject, a three-dimensional model may be generated as if there is also a space behind the optical surface, or a three-dimensional model may be generated in which a portion corresponding to the optical surface is broken. Thus, in photogrammetry, it is required to appropriately process the optical surface included in the image to generate a three-dimensional model.

[0005] The present invention has been made in view of the above, and an object thereof is to provide a three-dimensional model generation apparatus, a three-dimensional model generation method, and a three-dimensional model generation program capable of appropriately processing an optical surface included in an image to generate a three-dimensional model.

Means for Solving the Problems

[0006] The three-dimensional model generation apparatus according to the present invention acquires multiple images taken from multiple shooting positions. An image acquisition unit, and among the multiple acquired images, a reflected image visible due to light reflection. and detect the optical surface region in which at least one of the transparent objects visible through the transparent member is reflected. An optical surface detection unit, and a plurality of the optical surface regions on which the masks are placed. It includes a model generation unit that generates a three-dimensional model based on an image.

[0007] The three-dimensional model generation method according to the present invention acquires multiple images taken from multiple shooting positions. To do so, among the multiple images acquired, the reflected image and the transmitted image that are visible due to the reflection of light. To detect an optical surface region in which at least one of the transmissive objects visible through a light-emitting element is reflected. Then, a mask is placed in the optical surface region, and based on the plurality of images on which the mask is placed This includes generating a three-dimensional model.

[0008] The three-dimensional model generation program according to the present invention uses multiple images captured from multiple shooting positions. The process of obtaining the image, and the reflected image visible due to light reflection from among the multiple images obtained. and detect the optical surface region in which at least one of the transparent objects visible through the transparent member is reflected. The process involves placing a mask on the optical surface region and using a plurality of images on which the mask is placed. This process includes generating a three-dimensional model. [Effects of the Invention]

[0009] According to the present invention, a three-dimensional model is generated by appropriately processing optical surfaces contained in multiple images. It is possible. [Brief explanation of the drawing]

[0010] [Figure 1] FIG. 1 is a diagram schematically showing an example of a three-dimensional model generation device according to the present embodiment. [Figure 2] FIG. 2 is a functional block diagram showing an example of a three-dimensional model generation device. [Figure 3] FIG. 3 is an explanatory diagram showing the positional relationship between two images to which the principle of photogrammetry is applied. [Figure 4] FIG. 4 is an explanatory diagram showing the positional relationship between two images. [Figure 5] FIG. 5 is a diagram showing a state of photographing a three-dimensional space. [Figure 6] FIG. 6 is a diagram showing an example of a plurality of images obtained by photographing a three-dimensional space. [Figure 7] FIG. 7 is a diagram showing an example of a state where masks are arranged on a plurality of images. [Figure 8] FIG. 8 is a flowchart showing an example of a three-dimensional model generation method according to the present embodiment.

Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments of a three-dimensional model generation device, a three-dimensional model generation method, and a three-dimensional model generation program according to the present invention will be described based on the drawings. Note that the present invention is not limited by this embodiment. Further, the constituent elements in the following embodiments include those that can be replaced by those skilled in the art and are easy to replace, or those that are substantially the same.

[0012] FIG. 1 is a diagram schematically showing an example of a three-dimensional model generation device 100 according to the present embodiment. FIG. 2 is a functional block diagram showing an example of the three-dimensional model generation device 100. The three-dimensional model generation device 100 shown in FIGS. 1 and 2 generates a three-dimensional model based on the principle of photogrammetry. A Dell is generated. As shown in Figures 1 and 2, the three-dimensional model generation apparatus 100 is a processing unit 1. It comprises 0 and a memory unit 20.

[0013] The processing unit 10 is a CPU (Central Processing Unit) and other processing units. The device and RAM (Random Access Memory) or ROM (Read It has a storage device such as Only Memory. The processing unit 10 has an image acquisition unit 11 and an optical It comprises a surface detection unit 12, a color attribute detection unit 13, an area detection unit 14, and a model generation unit 15. ru.

[0014] The image acquisition unit 11 acquires multiple images I taken from multiple shooting positions. Each image I is These are images captured by imaging devices such as camera CR (C1, C2, etc.).

[0015] The optical surface detection unit 12 detects optical surface regions included in the multiple acquired images. In terms of form, the optical surface region is, for example, the region in an image where the surrounding image is projected, and the reflection of light Reflected image visible by light is projected onto the reflective surface area and transmitted image is visible through the transparent material. It includes at least one of the transparent surface regions in which an object is reflected. The optical surface detection unit 12 detects by known methods. The optical surface region included in the image can be detected. For example, a display device can be used to input the region into three-dimensional space K. A predetermined pattern is displayed and the pattern is moved in one direction, and this state The three-dimensional space is captured using the optical surface detection unit 12. Check whether there are areas where the image is inverted or where the movement of the pattern is not uniform. The optical surface detection unit 12 detects an area where the movement of the pattern is reversed and displayed. In addition, the region can be designated as a reflective surface region. Furthermore, the optical surface detection unit 12 determines the pattern If a region where the movement is not uniform is detected, that region can be designated as a transparent surface region. The specific method by which the optical surface detection unit 12 detects the optical surface region is not limited to the above, and other methods are also available. This method is also acceptable.

[0016] In this embodiment, the reflective surface region is, for example, a non-metallic member whose surface has been mirror-finished. Or, the color of the underlying material, such as a colored metal component like gold or copper, overlaps with the color of the image reflected on the mirror surface. The area that is visible, and the image reflected on the mirror surface, like a colorless metal component with a mirror finish. This includes areas where the color is visible as is. In this embodiment, the transparent surface area is, for example, glass. Examples include the surface of a light-transmitting material such as a sheet of wood. The light-transmitting surface area can be a chromatic color. This includes light-transmitting members and achromatic light-transmitting members, etc.

[0017] The color attribute detection unit 13 detects the color attributes of the optical surface area. In this embodiment, the color attributes are: It includes the so-called three attributes of color: hue, saturation, and brightness. The color attribute detection unit 13 detects, for example, an image. The process detects color attributes in the optical surface area. This allows us to determine the trend of color attributes in the optical surface region. The color attribute detection unit 13 detects the optical surface Regarding the hue, saturation, and brightness that constitute the color attributes of a region, for example, the number of coordinates in the color space. It can be detected as a value.

[0018] The area detection unit 14 detects the area of ​​the optical surface region. The area detection unit 14 detects, for example, the area of ​​the optical surface region in image I. The number of pixels corresponding to the detected optical surface area is detected as the area of ​​the optical surface area. It is possible.

[0019] The model generation unit 15 generates a three-dimensional model based on multiple images acquired by the image acquisition unit 11. The model generation unit 15 generates a three-dimensional model based on, for example, the principle of photogrammetry. It is possible to generate a Dell. Here, we will explain the principle of photogrammetry. Below, 2 This section explains how to generate three-dimensional image data from individual image data. Figure 3 shows a photo Figure 4 is an explanatory diagram showing the positional relationship of two images to which the principle of grammetry is applied. This is an explanatory diagram showing the relative positions.

[0020] The model generation unit 15 extracts, for example, two image data where the position indicated by the position data is the same. To release. Note that "same position" does not mean exactly the same, but rather that the position is shifted by a predetermined amount. Objects can be considered to be in the same position.

[0021] First, camera C1 for field of view images and camera C2 for field of view images (i The misalignment (see Figure 3) yields two sets of image data. Next, the model generation unit 15, The model generation unit 15 performs a search for corresponding feature points based on two sets of image data. For example, we perform a pixel-by-pixel correspondence and search for the position where the difference is minimized. Here, as shown in Figure 3... Cameras C1 and C2, which are assumed to exist simultaneously in two viewpoints, have the same X-axis, Ol and Or. Assume that the coordinates are located on the Z-coordinate plane such that Yl = Yr. Using the corresponding points searched by the generation unit 15, a disparity vector corresponding to the angle difference for each pixel is generated. Calculate.

[0022] The model generation unit 15 determines that the obtained disparity vector is the distance from cameras C1 and C2 in the depth direction. Since this corresponds to distance, the distance is calculated proportionally to the magnitude of the parallax using perspective. Photographer If we assume that cameras C1 and C2 move only almost horizontally, then cameras C1 and C2 are By arranging the optical axes Ol and Or so that they lie on the same XZ coordinate plane, The search for the hit point only needs to be performed on the scan lines that are the epipolar lines Epl and Epr. The model generation unit 15 uses two image data of the object and the data from cameras C1 and C2 to the object. Using the respective distances, three-dimensional image data of the object is generated. The model generation unit 15 generates The resulting three-dimensional image data may be stored, for example, in the memory unit 20, or in an output (not shown). The power unit or communication unit may output or transmit data to the outside.

[0023] On the other hand, the point Ql(Xl, Yl) on the left image corresponds to the point Qr(Xr, Yr) on the right image. In this case, the disparity vector at point Ql(Xl, Yl) is Vp(Xl-Xr, Yl-Yr). Here, since the two points Ql and Qr lie on the same scan line (epipolar line), Yl =Yr, and the disparity vector will be expressed as Vp(Xl-Xr, 0). Model The generation unit 15 obtains such a disparity vector Vp for all pixel points on the image, and By creating a set of difference vectors, we can obtain information about the depth direction of the image. By the way, epipolar lines For sets that are not level, one of the camera positions is at a different height (though this is unlikely). This can happen. In this case, the model generation unit 15 uses a large search range and aims to match corresponding points. Compared to searching for corresponding points in a large 2D region without any awareness of the epipolar line direction, By searching within a rectangle perpendicular to the epipolar line, with a deviation from the horizontal that is approximately the same as the line's horizontal displacement. The computational complexity for the minimum rectangle is reduced, making it more rational. Then, the model generation unit 15 is shown in Figure As shown in 4, the minimum epipolar line direction search range for a rectangle is a~b=c~d The search range is shown when the orthogonal search range is b~c = d~a. In this case, Epipo The search width in the direction of the epipolar line is ΔE, and the search width in the direction T perpendicular to the epipolar line is ΔT. The region we are looking for is the smallest unsloped rectangle ABCD that contains the least sloping rectangle abcd. ru.

[0024] In this way, the model generation unit 15 generates an epipo from corresponding feature points of multiple cameras C1 and C2. - Using constraints, the disparity vector is determined, and information in the depth direction of each point is obtained, and the 3D shape is displayed. The texture on the surface is mapped to generate three-dimensional image data. This allows for computation The model of a portion of the image data used reproduces the space as seen from the front hemisphere. This is possible. Also, if there are parts of the three-dimensional image data that are not captured in the image data... If you want to extend and connect the lines or surfaces of the surrounding texture, use the same technique in between. Interpolate using sculpting.

[0025] Furthermore, the method for generating three-dimensional image data is not limited to the one described above, and other methods may also be used. You may use this method.

[0026] When multiple images I include an optical surface region, the model generation unit 15 performs the following for the optical surface region. A mask is placed to cover the optical surface area, and based on the multiple images I on which the mask is placed, three A dimensional model is generated. The model generation unit 15 generates a color mask corresponding to the color of the optical surface region. It can be placed in the learning area. For example, the model generation unit 15 is determined by the color attribute detection unit 13. If the color attributes of the detected optical surface area follow a predetermined trend, the color attributes of the optical surface area are corresponding to the color attributes of the optical surface area. It is possible to generate and place a mask with the corresponding color attribute (hereinafter referred to as the "corresponding mask"). Meanwhile, the model generation unit 15 detects the color attributes of the optical surface region detected by the color attribute detection unit 13. If the result does not follow a predetermined trend, a pre-configured mask (hereinafter referred to as the standard mask) is placed. It is possible.

[0027] The model generation unit 15 generates values ​​in the color space that represent the hue, saturation, and brightness of the optical surface area, for example. If a certain percentage of the optical surface region is distributed within a predetermined range at a threshold or higher, the color attribute of that optical surface region is predetermined It can be determined that there is a tendency. In this embodiment, the color attribute of the optical surface area is predetermined If there is a tendency, the optical surface area is chromatic. If the color attributes of the optical surface area do not follow the predetermined tendency... In total, the optical surface area is achromatic or nearly achromatic.

[0028] When the model generation unit 15 generates a corresponding mask corresponding to the color attributes of the optical surface region, The color attributes of the response mask are determined, for example, according to the peak value in the distribution of values ​​indicating the color attributes of the optical surface area. This allows for a color attribute to be defined as such. This makes the color mask corresponding to the color in the optical surface area optical Because it is placed in a surface area, the observer's sense of unease is reduced. In this case, the model generation unit 15 For example, the corresponding mask may have a glossy marking. In this case, the observer can see the corresponding mask. This allows us to recognize that the region marked "K" is an optical surface region.

[0029] The model generation unit 15, if the color attributes of the optical surface area do not follow a predetermined trend, sets a preset standard. A quasi-mask is placed. In this case, the standard mask can be, for example, one that mimics the reflective surface of a colorless mirror. You can set a mask or other elements to change the appearance.

[0030] Furthermore, the model generation unit 15 checks if the area of ​​the optical surface region detected by the area detection unit 14 is not a predetermined value. If the condition is met, a standard mask will be placed on the optical surface area regardless of the color attributes of the optical surface area. This is also acceptable. When the area of ​​the optical surface region is small, the color corresponding to the color of the optical surface region is used. It is estimated that the discomfort caused to the observer will not be significantly increased even without the mask being placed. This eliminates the need to set the color attributes of the corresponding mask.

[0031] Furthermore, the model generation unit 15 always applies a standard model to the optical surface area regardless of the color attributes of the optical surface area. You could also place a screen there.

[0032] The memory unit 20 stores various information. The memory unit 20 stores information according to a pre-set standard mask. It stores the information. The storage unit 20 is, for example, a hard disk drive, solid state It has storage such as a drive. Note that the storage unit 20 is a removable disk. External storage media such as the following may be used.

[0033] The memory unit 20 processes to acquire multiple images I taken from multiple shooting positions, and the acquired Among multiple images I, the reflected image visible due to light reflection and the image visible through the transparent material A process to detect an optical surface region in which at least one of the transparent objects is reflected, and a mask applied to the optical surface region. The process involves placing elements and generating a three-dimensional model based on multiple images with masks placed on them. Store the 3D model generation program to be executed by the computer.

[0034] Next, the operation of the three-dimensional model generation device 100 configured as described above will be explained. Figure 5 shows This figure shows the process of photographing the three-dimensional space K. Figure 6 shows multiple images of the three-dimensional space K. This figure shows an example of an image. Figure 7 shows an example of a state where a mask is placed on multiple images. That is the case.

[0035] First, as shown in Figure 5, the three-dimensional space K is photographed from different shooting positions. Image acquisition unit 1 Step 1 acquires multiple captured images. Here, as shown in Figure 6, two images I1, We will explain using the example of obtaining I2, but the number of images can be three or more. In the original space K, there are colorful elements that make up home appliances such as the television and rice cooker shown in Figure 5. Colored (for example, black) resin members 41, 42, and achromatic metal parts that constitute the reflective surface of the back mirror. Materials 43, achromatic and transparent glass components 44 that make up the window, and other objects that reflect the surrounding image are arranged. It is assumed that the resin members 41 and 42 have reflected images 41r and 42r. It is there. Also, the metal component 43 has a reflected image 43r. A reflected image is a reflection of light. This is an image that can be seen by the glass component 44. In addition, the glass component 44 can see clouds, buildings, etc. that are in the background. A 44t transparent object is visible. A transparent object is an object that can be seen through a transparent material such as glass. It is an object that can be seen.

[0036] By capturing this three-dimensional space K, the captured image I1, as shown in Figure 6, contains , resin members 41, 42, metal member 43 and glass member 44 are in optical surface areas 51a, 52a, They appear as 53a and 54a. Also, in the captured image I2, resin members 41 and 42, The metal member 43 and the glass member 44 reflect light as optical surface regions 51b, 52b, 53b, and 54b. Insert.

[0037] The optical surface detection unit 12 detects when an optical surface region is included in the multiple images I1 and I2 that it has acquired. The optical surface region is detected. In this embodiment, the optical surface detection unit 12 is included in the image I1. The optical surface regions 51a, 52a, 53a, and 54a, and the optical surface region 51b included in image I2. It can detect 52b, 53b, and 54b.

[0038] In the optical surface regions 51a to 54a and 51b to 54b, the reflected images 51r to 53r or The transparent object 54t is visible. If a three-dimensional model is generated in this state, the reflected image 41r ~43r and the transparent object 44t appear as if they exist deep within the optical plane as actual structures. A three-dimensional model is generated, or a three-dimensional model in which the part corresponding to the optical surface region is torn. In some cases, the following may be generated. Therefore, in this embodiment, by performing the following process The optical surface region is processed appropriately to generate a three-dimensional model.

[0039] The color attribute detection unit 13 detects optical surface regions 51a, 52a, 53a, and 54a included in the image I1. And detect the color attributes of optical surface regions 51b, 52b, 53b, and 54b contained in image I2. Furthermore, the area detection unit 14 detects the optical surface regions 51a, 52a, 53a, 5 included in the image I1. The areas of optical surface regions 51b, 52b, 53b, and 54b included in 4a and image I2 are detected. ru.

[0040] The model generation unit 15 generates optical surface regions 51a, 52a, 53a, 54a and optical surface region 51 In this embodiment, it is determined whether the areas of b, 52b, 53b, and 54b are less than a predetermined value. The model generation unit 15 generates the area of ​​the optical surface regions 51a, 53a, and 54a of the image I1, and the image The area of ​​optical surface regions 51b, 53b, and 54b of I2 is determined to be greater than or equal to a predetermined value. Furthermore, the model generation unit 15 generates the area of ​​the optical surface region 52a of image I1 and the light of image I2. The area of ​​the academic surface region 52b is determined to be less than a predetermined value. The model generation unit 15 then... The optical surface area 52a of image I1 and the optical surface area of ​​image I2 were determined to have an area less than a predetermined value. For region 52b, regardless of the color attributes determined below, a standard mask is used, as shown in Figure 7. Place M2 in the correct location.

[0041] The model generation unit 15 generates optical surface regions 51a, 53a, 54a and optical surface regions 51b, 53 b, it is determined whether the color attributes of 54b follow a predetermined trend. In this embodiment, model The generation unit 15 generates, for example, the color of the optical surface area 51a of image I1 and the optical surface area 51b of image I2. The model generation unit 15 determines that the attribute follows a predetermined trend. For the determined optical surface regions 51a and 51b, as shown in Figure 7, the corresponding color attributes are Position the response mask M1.

[0042] Furthermore, the model generation unit 15 generates, for example, the optical surface regions 53a, 54a of image I1 and image I2 The model generation unit 15 determines that the color attributes of the optical surface regions 53b and 54b do not follow a predetermined trend. These are the optical surface regions 53a, 53b, 54a, and 54 in which it was determined that the color attributes do not follow a predetermined trend. For b, standard masks M3 and M4 should be placed as shown in Figure 7.

[0043] Figure 8 is a flowchart showing an example of a three-dimensional model generation method according to this embodiment. As shown in Figure 8, the image acquisition unit 11 captures multiple images of the three-dimensional space K from different shooting positions. An image is acquired (step S10). The optical surface detection unit 12 is found to be included in the multiple acquired images. Step S20 detects the optical surface region. The color attribute detection unit 13 detects the color attribute of the optical surface region. The property is detected (step S30). The area detection unit 14 detects the area of ​​the optical surface region ( (Tep S40).

[0044] The model generation unit 15 determines whether the area of ​​the optical surface region is less than a predetermined value (step S5). 0). If it is determined that the area of ​​the optical surface region is less than a predetermined value (Yes in step S50), the mod The mask generation unit 15 is configured to place a standard mask in the optical surface area (step S6). 0).

[0045] The model generation unit 15 determines that the area of ​​the optical surface region is not less than a predetermined value (step In step S50, determine whether the color attributes of the optical surface area follow a predetermined trend (Step S50 No.). 70). The model generation unit 15 determines that the color attributes of the optical surface region have a predetermined tendency ( Step S70 (Yes), a corresponding mask having color attributes corresponding to the color attributes of the optical surface area It is generated and placed in the optical surface region (step S80). Meanwhile, the model generation unit 15 generates the optical surface If it is determined that the color attributes of the region do not follow a predetermined trend (No. in step S70), the optical surface region Place the standard mask (step S60).

[0046] After placing the mask in step S60 or step S80, the mask is placed A three-dimensional model is generated based on multiple images (step S90).

[0047] As described above, the three-dimensional model generation apparatus 100 according to this embodiment is capable of taking images from multiple shooting positions. Image acquisition unit 11 acquires multiple images taken, and among the acquired multiple images, light reflection At least one of the reflected image seen by the transparent member and the transmitted object seen through the transparent member. An optical surface detection unit 12 detects the optical surface area in which the image is projected, and a mask is placed on the optical surface area. The system includes a model generation unit 15 that generates a three-dimensional model based on multiple images in which elements are arranged.

[0048] Furthermore, the three-dimensional model generation method according to this embodiment involves multiple images taken from multiple shooting positions. The process involves acquiring an image and then identifying the reflected image, which is visible due to the reflection of light, from among the multiple images acquired. and detect the optical surface region in which at least one of the transparent objects visible through the transparent member is reflected. This involves placing a mask in the optical surface area and creating a three-dimensional image based on multiple images with the mask placed on them. This includes generating a model.

[0049] Furthermore, the three-dimensional model generation program according to this embodiment uses images taken from multiple shooting positions. The process of acquiring multiple images, and the process of determining which of the acquired images is visible based on light reflection. An optical surface region in which at least one of the reflected image and the transmitted object visible through the transparent member is projected. The process involves detection, placing a mask on the optical surface area, and then judging based on multiple images with the mask placed on them. The computer is made to perform the process of generating a three-dimensional model.

[0050] According to this configuration, optical surface regions contained in multiple images are detected and a mask is placed, To generate a three-dimensional model based on multiple images in which the optical surface area is placed, the optical surface area is applied to the multiple images. Even when a region is included, the optical surface region is appropriately processed to generate a three-dimensional model. It is possible.

[0051] In the three-dimensional model generation apparatus 100 according to this embodiment, the model generation unit 15 is an optical surface A mask corresponding to the color of the region is placed in the optical surface region. With this configuration, the chromatic optical surface Regarding the region, a mask corresponding to the color of the optical surface region is placed, thus reducing the observer's discomfort. It can reduce the feeling.

[0052] In the three-dimensional model generation apparatus 100 according to this embodiment, the color attributes of the optical surface region are detected. The system further includes a color attribute detection unit 13, and the model generation unit 15 determines that the color attributes of the optical surface region have a predetermined tendency. If it is located in the optical surface area, a corresponding mask with color attributes corresponding to the color attributes of the optical surface area is generated and placed. Furthermore, if the color attributes of the optical surface area do not follow a predetermined trend, a pre-set standard mask is placed. This configuration allows for the use of corresponding masks and standard masks depending on the color attributes of the optical surface area. Because it can do this, it can more reliably reduce the observer's sense of unease.

[0053] The three-dimensional model generation apparatus 100 according to this embodiment uses an area detector to detect the area of ​​the optical surface region. The device further includes an extension 14, and if the area of ​​the optical surface region is less than a predetermined value, the color attribute of the optical surface region is determined Regardless, a pre-set standard mask is placed. According to this configuration, the area of ​​the optical surface region If the value is less than a predetermined value, the process of setting the color attributes of the corresponding mask can be omitted.

[0054] The technical scope of the present invention is not limited to the embodiments described above, and the invention does not depart from the spirit of the present invention. Modifications can be made as appropriate within a certain range. For example, in the above embodiment, the model generation unit 15 is the color attribute of the optical surface areas 54a and 54b corresponding to the glass component 44 such as window glass. The example given was one where it was determined that there was no established trend, but this is not the only example. In light-transmitting areas such as these, the color attributes vary depending on the scenery (image) in the background, for example, if a blue sky is visible. There may be a certain tendency. In such cases, the model generation unit 15 generates the optical surface region 54a It can be determined that the color attribute of 54b follows a predetermined trend.

[0055] Furthermore, in the above embodiment, the same standard mask is applied to the reflective surface region and the transmissive surface region. The explanation uses the example of the case shown, but is not limited to this. The optical surface detection unit 12 detects the reflective surface region The reflective surface region may be detected separately. In this case, the model generation unit 15 detects the reflective surface region. The standard mask for the transparent area and the standard mask for the transparent area may be applied separately. [Explanation of Symbols]

[0056] CR, C1, C2…Camera, I, I1, I2…Image, K…Three-dimensional space, M1…Corresponding mass K, M2, M3, M4... Standard mask, 10... Processing unit, 11... Image acquisition unit, 12... Optical surface inspection Output unit, 13...Color attribute detection unit, 14...Area detection unit, 15...Model generation unit, 20...Storage unit, 4 1, 42... Resin component, 43... Metal component, 41r, 42r, 43r... Reflected image, 44... Glass Components, 44t... Transparent visible objects, 51a, 51b, 51c, 52a, 52b, 53a, 53b ,53c,54a,54b...Optical surface area, 100...Three-dimensional model generation device

Claims

1. An image acquisition unit that acquires multiple images taken from multiple shooting positions, An optical surface detection unit detects an optical surface region from among the multiple images acquired, A model generation unit that places a mask on the optical surface region and generates a three-dimensional model based on a plurality of images on which the mask is placed, A color attribute detection unit for detecting the color attributes of the optical surface area and Equipped with, The model generation unit generates and places a mask having color attributes corresponding to the color attributes of the optical surface area if the values ​​representing the hue, saturation, and brightness of the optical surface area are distributed at a rate equal to or greater than a predetermined threshold among the color attributes of the optical surface area. Three-dimensional model generation device.

2. A mask having the color attributes corresponding to the color attributes of the optical surface region has color attributes corresponding to the peak value of the distribution of values ​​indicating the color attributes of the optical surface region. A three-dimensional model generation apparatus according to claim 1.

3. An image acquisition unit that acquires multiple images taken from multiple shooting positions, An optical surface detection unit detects an optical surface region from among the multiple images acquired, A model generation unit that places a mask on the optical surface region and generates a three-dimensional model based on a plurality of images on which the mask is placed, A color attribute detection unit for detecting the color attributes of the optical surface area and Equipped with, The model generation unit places a pre-set mask for the reflective surface area when the values ​​representing the hue, saturation, and brightness of the optical surface area are not distributed at a rate equal to or greater than a predetermined threshold, and the optical surface area is a reflective surface area on which a reflected image visible due to light reflection is projected. The model generation unit also places a pre-set mask for the transparent surface area when the values ​​representing the hue, saturation, and brightness of the optical surface area are not distributed at a rate equal to or greater than a predetermined threshold, and the optical surface area is a transparent surface area on which a transmissive object visible through a transparent member is projected. Three-dimensional model generation device.

Images