Three-dimensional geometric shape measuring device and three-dimensional geometric shape measuring method

Abstract

A 3D geometry measuring device includes: a projection part that projects a projection image including a stripe pattern extending in multiple directions onto an object to be measured; a capture part that generates captured images including the stripe pattern projected by the projection part onto the object to be measured; a feature quantity recognition part that recognizes a feature quantity indicating the stripe pattern included in the captured image; a defective pixel detection part that detects a defective pixel in the captured image based on each feature quantity recognized from the captured image of the stripe pattern extending in the corresponding direction; and a geometry recognition part that recognizes a correspondence between pixels of the captured image other than the defective pixel and pixels of the projected image to recognize the 3D geometry of the object to be measured based on the correspondence.

Description

[0001] This application is a divisional application of the invention patent application with the application date of June 20, 2018, application number 201810637645.4, and invention name "Three-dimensional geometric shape measuring device and three-dimensional geometric shape measuring method". Technical Field

[0002] The present invention relates to a three-dimensional geometric shape measuring device and a three-dimensional geometric shape measuring method for measuring the three-dimensional geometric shape of an object to be measured. Background Art

[0003] Methods for measuring an object to be measured without any contact can be divided into two types: passive techniques such as stereo methods, and active techniques such as triangulation methods, time-of-flight methods, and confocal methods. Among them, triangulation methods are increasing in various fields such as product quality management and reverse engineering.

[0004] The light pattern projection method uses the principle of the triangulation method and performs three-dimensional (3D) geometry measurement by projecting a stripe pattern from a projector onto an object to be measured and then by capturing the pattern that changes along the geometry of the object to be measured with a camera. Japanese Unexamined Patent Application Publication No. 2009-094295 discloses a measuring device for measuring the height of an electronic component based on a captured image obtained by imaging an optical cutting line when line light is projected onto the electronic component.

[0005] Since the light pattern projection method can simultaneously measure a larger area when an image including a plurality of fringe patterns is projected onto an object to be measured, it can measure 3D geometry faster. Summary of the invention

[0006] Problems to be solved by the present invention

[0007] In the light pattern projection method, when the surface of the object to be measured is smooth, multiple reflection occurs, that is, the projection light from the projector is repeatedly reflected on the surface of the object to be measured. Due to the multiple reflection, there is a problem of reduced measurement accuracy.

[0008] As a method of preventing multiple reflections, a method of applying an anti-multiple reflection spray on the surface of an object to be measured, a mask that cuts a portion of the projection light from a projector in its optical path, etc. has been adopted. However, in the method of applying the anti-multiple reflection spray on the surface of the object to be measured, there is a problem that the number of man-hours for washing increases. There is also another problem that the anti-multiple reflection spray cannot be applied in an environment that needs to maintain high cleanliness.

[0009] In addition, since the number of times a pattern is projected onto an object to be measured needs to increase in order to cut a portion of the projection light from the projector, the method using a mask is associated with a problem related to an increase in measurement time. In addition, in this method, there is another problem that a different mask needs to be created for each individual object to be measured.

[0010] The present invention focuses on these points, and an object of the present invention is to provide a three-dimensional geometry measuring device and a three-dimensional geometry measuring method that can prevent a decrease in measurement accuracy caused by multiple reflections.

[0011] Means of solving the problem

[0012] According to a three-dimensional (3D) geometry measuring device of a first embodiment of the present invention, wherein the 3D geometry measuring device measures the 3D geometry of the object to be measured by projecting a projection image including a stripe pattern onto the object to be measured, the device comprises: a projection part, which projects the projection image onto the object to be measured; a capture part, which generates a captured image of the object to be measured on which the projection image is projected; an analysis part, which obtains a correspondence between a projection pixel position as a pixel position of the projection image and a capture pixel position as a pixel position of the capture image; a line recognition part, which recognizes a first core line of the capture part corresponding to the capture pixel position or a second core line of the projection part corresponding to the projection pixel position; a defective pixel detection part, which detects defective pixels based on a positional relationship between the projection pixel position and the first core line or a positional relationship between the projection pixel position and the second core line; and a geometry recognition part, which recognizes the 3D geometry of the object to be measured based on a correspondence of pixel positions other than the position of the defective pixel detected by the defective pixel detection part from among the correspondences obtained by the analysis part.

[0013] When the distance between the capture pixel position and the first epipolar line corresponding to the projection pixel position having a corresponding relationship with the capture pixel position and the distance between the projection pixel position and the second epipolar line corresponding to the capture pixel position having a corresponding relationship with the projection pixel position are greater than a threshold value, the defective pixel detection section may detect the pixel at the capture pixel position as a defective pixel or detect the pixel at the projection pixel position as a defective pixel. When the distance between the capture pixel position and the first epipolar line corresponding to the projection pixel position having a corresponding relationship with the capture pixel position is equal to or less than a threshold value, the analysis section may estimate that the position on the first epipolar line closest to the capture pixel position and the projection pixel position have a corresponding relationship with each other.

[0014] The projection part can project a projection image including pixels other than the defective pixels detected by the defective pixel detection part among a plurality of pixels included in the projection image onto the object to be measured, and the analysis part can estimate the correspondence between (i) the pixel positions of the projection image other than the defective pixels projected by the projection part and (ii) the captured image positions of the captured image of the object to be measured onto which the projection image other than the defective pixels is projected.

[0015] The projection part can project an image including defective pixels detected by the defective pixel detection part among multiple pixels included in the projected image onto the object to be measured, and the analysis part can estimate the correspondence between (i) the projection pixel position of the image including the defective pixels projected by the projection part and (ii) the captured pixel position of the captured image of the object to be measured onto which the image including the defective pixels is projected.

[0016] If the analysis part estimates that a projected pixel position of the projected image corresponds to multiple captured image pixel positions, the defective pixel detection part may detect a pixel of at least one captured image pixel position among the multiple captured image pixel positions corresponding to the one projected pixel position as a defective pixel.

[0017] If the analysis part estimates that a projected pixel position of the projected image corresponds to a plurality of captured image pixel positions on the first core line of the capture part identified by the line identification part, the defective pixel detection part may detect a pixel of at least one of the plurality of captured image pixel positions corresponding to the one projected pixel position as a defective pixel.

[0018] The projection part may project a projection image including a stripe pattern extending in a first direction orthogonal to the optical axis of the capture part and the optical axis of the projection part, and a projection image including a stripe pattern extending in a second direction parallel to a plane including the optical axis of the projection part and the optical axis of the capture part. The projection part may project the projection image including the stripe pattern extending in the second direction parallel to the plane including the optical axis of the projection part and the optical axis of the capture part and having a sinusoidal brightness distribution onto the object to be measured, and when the phase of the pixel or the phase change amount between pixels is different from a standard value, the defective pixel detection part may detect a pixel of the stripe pattern extending in the second direction in the captured image of the object to be measured onto which the projection image is projected as a defective pixel.

[0019] The capturing part and the projection part may be arranged so that the first core lines of the capturing part identified by the line recognition part do not cross each other in the captured image. The capturing part and the projection part may be arranged so that the second core lines of the projection part identified by the line recognition part do not cross each other in the projected image.

[0020] The projection part may project a projection image including a fringe pattern having a binary image and a projection image including a fringe pattern having a sinusoidal brightness distribution onto the object to be measured. The projection part may sequentially project projection images including fringe patterns having fringe periods different from each other.

[0021] The projection part may project the fringe pattern with a sinusoidal brightness distribution, and the analysis part may estimate that a pixel closest to the second epipolar line of the projection part corresponding to the capture pixel position is a pixel having correspondence with the capture pixel position among a plurality of pixels presenting brightness of the same phase as the capture pixel position in the fringe image. The 3D geometry measuring device may further include a determination part that determines that the alignment state of the projection part and the capture part is inappropriate when a statistic of (i) the distance between the capture pixel position and the first epipolar line or (ii) the distance between the projection pixel position and the second epipolar line exceeds an allowable value.

[0022] According to a second embodiment of the present invention, a 3D geometry measurement method is provided, wherein the 3D geometry measurement method measures the 3D geometry of the object to be measured by projecting a projection image including a stripe pattern onto the object to be measured, and the method comprises: a step of projecting the projection image onto the object to be measured; a step of generating a captured image of the object to be measured onto which the projection image is projected; a step of obtaining a correspondence between a projection pixel position as a pixel position of the projection image and a capture pixel position as a pixel position of the capture image; a step of identifying a first core line of a capture part corresponding to the capture pixel position or a second core line of a projection part corresponding to the projection pixel position, and the capture part generates the captured image of the object to be measured and the projection part projects the projection image onto the object to be measured; a step of detecting defective pixels based on a positional relationship between the projection pixel position and the first core line or a positional relationship between the projection pixel position and the second core line; and a step of identifying the 3D geometry of the object to be measured based on a correspondence of pixel positions other than the position of the detected defective pixel from among the obtained correspondences.

[0023] According to another embodiment of the present invention, a 3D geometry measuring device includes: a projection part, which projects a projection image including a stripe pattern extending in multiple directions onto an object to be measured; a capture part, which generates captured images including the stripe pattern projected by the projection part onto the object to be measured; a feature quantity recognition part, which recognizes feature quantities indicating the stripe pattern included in the captured image; a defective pixel detection part, which detects defective pixels in the captured image based on each feature quantity recognized from the captured image of the stripe pattern extending in the corresponding direction; and a geometry recognition part, which recognizes the correspondence between pixels of the captured image other than the defective pixels and pixels of the projection image to recognize the 3D geometry of the object to be measured based on the correspondence.

[0024] According to another embodiment of the present invention, a three-dimensional 3D geometric shape measurement method includes the following steps: projecting a projection image including a stripe pattern extending in multiple directions onto an object to be measured; respectively generating captured images including the stripe pattern projected onto the object to be measured; identifying feature quantities indicating the stripe pattern included in the captured image; detecting defective pixels in the captured image based on each feature quantity identified from the captured image of the stripe pattern extending in the corresponding direction; and identifying the correspondence between pixels of the captured image other than the defective pixels and pixels of the projection image to identify the 3D geometric shape of the object to be measured based on the correspondence.

[0025] Effects of the Invention

[0026] According to the present invention, an effect of preventing a decrease in measurement accuracy caused by multiple reflections is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figures 1A to 1C An overview of a 3D geometry measuring device according to a first embodiment is shown.

[0028] Figure 2A and Figure 2B Each displays a projection image projected by the projection section onto the object to be measured.

[0029] Figure 3 Shows the configuration of the 3D geometry measurement device.

[0030] FIG. 4A to FIG. 4F Examples of the types of projection images projected by the projection control section are respectively displayed.

[0031] FIG. 5A to FIG. 5D Examples of gradient fringe patterns with sinusoidal brightness distribution are shown respectively.

[0032] Figure 6The feature amount to be recognized by the feature amount recognition section is shown.

[0033] Fig. 7A and Figure 7B Each shows multiple reflections.

[0034] Fig. 8A and Figure 8B Each shows multiple reflections of light on a groove of an object to be measured.

[0035] Fig.9A and Fig. 9B Each shows a change in a feature amount of a gradation fringe pattern on a captured image due to multiple reflected light.

[0036] Fig.10 A method of calculating a feature amount based on a captured image is shown.

[0037] Fig.11 Display corresponds to FIG. 4C to FIG. 4F An example of a Gray code for a binary stripe pattern is shown in .

[0038] Fig.12 is a flowchart showing an example of the operation of the 3D geometry measuring device.

[0039] Fig.13 The principle of detecting a defective pixel according to the second embodiment is shown.

[0040] Fig.14 The principle of detecting a defective pixel according to the second embodiment is shown.

[0041] Fig.15 Shows the configuration of the 3D geometry measurement device.

[0042] Fig.16 The defective pixel detection section shows a defective pixel detection operation.

[0043] Fig.17 The core points are shown.

[0044] Fig.18 Detection of defective pixels on an epipolar line by a defective pixel detection section is shown.

[0045] Fig.19A and Fig.19B Each shows a method of detecting a defective pixel by the defective pixel detection section in the case of projecting a stripe pattern extending in the second direction.

[0046] Fig. 20 A method of correcting the correspondence relationship by the defective pixel detection section is shown.

[0047] Fig.21is a flowchart showing an example of the operation of the 3D geometry measuring device according to the second embodiment.

[0048] Fig. 22 A method of estimating the correspondence relationship between the captured pixel position and the projected pixel position by the analysis section according to the third embodiment is shown. DETAILED DESCRIPTION

[0049] <First Embodiment>

[0050] <Overview of 3D Geometry Measurement Device 100>

[0051] Figures 1A to 1C An overview of a 3D geometry measuring device 100 according to the present embodiment is shown. Figure 1A The configuration of the 3D geometry measuring device 100 is shown. The 3D geometry measuring device 100 has a projection section 1 , a capture section 2 , and a control section 3 .

[0052] The projection part 1 is a projection device having a light source such as a light emitting diode, a laser, etc., and projects a plurality of respectively different projection images including a stripe pattern onto a measurement surface of an object to be measured. The capture part 2 is a capture device having a lens 21 and an imaging element 22. The capture part 2 generates a plurality of captured images by respectively capturing the object to be measured while the projection images are sequentially projected onto the object to be measured by the projection part 1. The capture part 2 is placed in such a way that the optical axis of the capture part 2 forms a predetermined angle with the optical axis of the projection part 1. The control part 3 measures the geometric shape of the object to be measured based on the captured images generated by the capture part 2. The control part 3 can be realized by a computer, for example.

[0053] Figure 1B and Figure 1C Each shows an example of a captured image generated by the capturing section 2 while the projecting section 1 projects a projection image onto an object to be measured. Figure 1B and 1C As shown, the projection section 1 projects a projection image including a binary fringe pattern onto a target for measurement. Figure 1B The captured image generated by the capturing section 2 when the projection section 1 projects a projection image including a binary fringe pattern onto a respective uniform measurement surface is shown. The white area represents a light projection area, and the black area represents a non-projection area. When the measurement surface has no irregularities, the binary fringe pattern of the captured image generated by the capturing section 2 matches the binary fringe pattern of the projection image.

[0054] Figure 1C The captured image generated by the capturing section 2 when the projection section 1 projects a binary fringe pattern onto a measurement surface having a convex portion is shown. Figure 1CIn the captured image, a portion of the binary fringe pattern image is deformed. In the captured image, the image of the binary fringe pattern is deformed by a certain amount according to the height of the protruding portion. Therefore, the 3D geometry measuring device 100 can measure the geometry of the object to be measured by identifying the height of each position of the protruding portion based on the amount of deformation in the binary fringe pattern image in the captured image.

[0055] Figure 2A and Figure 2B Each displays a projection image projected by the projection section 1 onto the object to be measured. Figure 2A An example of a binary stripe pattern extending along a first direction is shown, and Figure 2B An example of a binary stripe pattern extending along the second direction is shown. Figure 2A As shown, the projection part 1 projects a binary stripe pattern extending in a first direction (hereinafter this may be referred to as a vertical pattern). The first direction is a direction orthogonal to the optical axis of the projection part 1 and is a direction orthogonal to the optical axis of the capture part 2. Figure 2B As shown, the projection part 1 projects a binary stripe pattern extending in a second direction (hereinafter this may be referred to as a horizontal pattern). The second direction is a direction parallel to a plane including the optical axis of the projection part 1 and the optical axis of the capture part 2.

[0056] When the fringe pattern is projected onto the object to be measured, the fringe pattern deviates in the width direction according to the 3D geometry of the object to be measured, such as Figure 1C shown.

[0057] Moreover, the width of the stripe pattern fluctuates according to the 3D geometry of the object to be measured. In the first captured image with the stripe pattern extending in the first direction, the direction of the orientation deviation of the optical axis of the projection part 1 and the optical axis of the capture part 2 is matched with the direction in which the deviation occurs in the width direction of the stripe pattern. That is, (i) the direction of the image of the line segment generated by projecting the line segment connecting the projection part 1 and the capture part 2 onto the plane on which the object to be measured is placed matches (ii) the direction in which the deviation occurs in the width direction of the stripe pattern. Therefore, in the first captured image, the sensitivity of detecting the deviation of the stripe pattern in the width direction is high. For this reason, the resolution is improved in the measurement of the 3D geometry of the object to be measured.

[0058] Meanwhile, in the second captured image with the stripe pattern extending in the second direction, (i) the direction in which the orientations of the optical axis of the projection section 1 and the optical axis of the capture section 2 are deviated and (ii) the direction in which the orientations of the optical axis of the projection section 1 and the capture section 2 are deviated are orthogonal to each other. That is, the direction of the image of the line segment generated by projecting the line segment connecting the projection section 1 and the capture section 2 onto the plane on which the object to be measured is placed is orthogonal to the direction in which the orientations of the optical axis of the projection section 1 and the capture section 2 are deviated. Therefore, the measurement resolution of the second captured image is significantly reduced when measuring the 3D geometry of the object to be measured, as compared to the first captured image, and the 3D geometry measuring device 100 cannot accurately measure the geometry.

[0059] The 3D geometry measuring device 100 obtains the 3D geometry of the object to be measured by analyzing the fringe pattern projected onto the object to be measured. However, when the surface of the object to be measured is smooth, there is a problem of reduced measurement accuracy due to multiple reflections caused by multiple reflections of the projection light from the projection part 1. Therefore, the 3D geometry measuring device 100 captures the object to be measured while projecting the fringe pattern extending in the first direction and the fringe pattern extending in the second direction onto the object to be measured, respectively, and detects defective pixels affected by the multiple reflections by analyzing the captured images.

[0060] Figure 3 The configuration of the 3D geometry measuring device 100 is shown. The 3D geometry measuring device 100 has a projection section 1, a capture section 2, a control section 3, and a memory section 4. The memory section 4 includes a storage medium including a hard disk, a read-only memory (ROM), a random access memory (RAM), and the like. The memory section 4 stores a program to be executed by the control section 3. The control section 3 is, for example, a central processing unit (CPU), and functions as a projection control section 301, an acquisition section 302, a feature quantity recognition section 303, a defective pixel detection section 304, an analysis section 305, and a geometry recognition section 306 by executing the program stored in the memory section 4.

[0061] The projection control section 301 generates a control signal for projecting a projection image including a fringe pattern onto an object to be measured, and inputs the generated control signal into the projection section 1. FIG. 4A to FIG. 4F and FIG. 5A to FIG. 5D The types of stripe patterns projected by the projection control section 301 will be described.

[0062] [Type of stripe pattern]

[0063] FIG. 4A to FIG. 4F Examples of the types of projection images projected by the projection control section 301 are respectively shown. FIG. 4A to FIG. 4FIn the figure, the black area represents an area where the projection part 1 does not project light (hereinafter referred to as a "non-projection area"), and the white area represents an area where the projection part 1 projects light (hereinafter referred to as a "light projection area").

[0064] Figure 4A A standard projection pattern in which no light is projected onto any part of the object to be measured (ie, a completely black pattern) is displayed. Figure 4B Displays a standard projection pattern (i.e., a full white pattern) that projects light onto the entire object to be measured. FIG. 4C to FIG. 4F Displays a binary fringe pattern consisting of a light projection area and a non-projection area and in which stripes having different widths for each projected image are arranged in the same direction. FIG. 4A to FIG. 4F The stripe pattern shown in corresponds to a Gray code and is used to identify the location of pixels in a captured image. The details will be described below.

[0065] FIG. 5A to FIG. 5D Examples of gradient stripe patterns with sinusoidal brightness distribution are shown respectively. FIG. 4C to FIG. 4F The binary fringe pattern shown in FIG. 5A to FIG. 5D The gradient stripe pattern shown in the figure is a binary image consisting of black and white areas, but the brightness changes sinusoidally from white to black along the width of the stripe. FIG. 5A to FIG. 5D The intervals between the stripes in the gradient stripe pattern are constant, and the spatial frequency of the stripes in the gradient stripe pattern is, for example, Figure 4F Four times the spatial frequency of the binary fringe pattern.

[0066] Figures 5A to 5D The gradient stripe patterns are different from each other in that the phases of the sine waves indicating the brightness distribution are different from each other by 90 degrees, and their brightness distributions are otherwise the same. In this embodiment, the projection control section 301 projects a total of 10 projection images: Figure 4A and Figure 4B The two standard patterns shown in FIG. 4C to FIG. 4F The four binary fringe patterns shown in FIG. 5A to FIG. 5D Four gradient stripe patterns are shown in . Figures 5A to 5D The gradient stripe pattern shown in FIG. 4A to FIG. 4F The fringe pattern shown in is used together to identify the location of pixels in the captured image.

[0067] Figure 3 The acquisition section 302 shown in FIG. 1 acquires a captured image generated by the capture section 2. The feature amount recognition section 303 recognizes a feature amount indicating a feature of a stripe pattern included in the captured image acquired by the acquisition section 302. Figure 6 , the feature amount identified by the feature amount identifying section 303 will be described. Figure 6 Display as Figure 5A As shown, the brightness distribution of the captured image generated by the capturing section 2 while the gradient stripe pattern is projected onto the object to be observed. Figure 6 In FIG. 6 , the horizontal axis 6 indicates the distance in the width direction of the gradient stripe pattern, and the vertical axis indicates the brightness of the gradient stripe pattern. Figure 6 As shown, Figure 5A The brightness distribution of the gradational stripe pattern in the image is a sine wave. The feature quantity identification section 303 calculates the amplitude A of the sine wave of the brightness distribution of the gradational stripe pattern. m , as the offset O of the sum of the brightness amplitude and the minimum value f , as the amplitude A m With offset O f The ratio (A m / O f ) or the distortion of the sine waveform, etc., as a feature quantity. Amplitude is a feature quantity for determining the absolute value of brightness. In addition, by using the normalized value of contrast and the normalized value of distortion, the feature quantity identification part 303 can make these values ​​become quantities that are almost independent of the exposure time of the camera, etc.

[0068] [About multiple reflections]

[0069] The defective pixel detection section 304 detects defective pixels of the captured image affected by multiple reflections. Specifically, the defective pixel detection section 304 detects defective pixels by determining whether the feature quantity identified by the feature quantity identification section 303 is within a predetermined range. The defective pixel is a pixel having a brightness affected by multiple reflections, which is different from the brightness of the directly reflected light.

[0070] By reference Fig. 7A and Figure 7B , multiple reflections will be described. Fig. 7A and 7B When the object to be measured is smooth and has a complex shape, the light emitted by the projection part 1 may enter the capture part 2 after being repeatedly reflected multiple times on the surface to be measured. In this case, Fig. 7A As shown, light emitted from the projection section 1 enters one pixel of the imaging element 22 via two or more paths.

[0071] Specifically, the light entering the imaging element 22 includes direct light, which is light emitted by the projection section 1 and directly enters the capturing section 2 after being diffused and reflected on the surface to be measured, and multiple reflected light, which enters the capturing section 2 after being subjected to multiple reflections. As a result, in the captured image captured by the capturing section 2, a pixel having a brightness value corresponding to black when there is no multiple reflected light may have a brightness value corresponding to white. In particular, when the object to be measured contains metal or the like that may cause random reflection, multiple reflections may occur.

[0072] Figure 7B An example of a captured image affected by multiple reflections is shown. Figure 7B Corresponds to Figure 1C , but due to the influence of light, the shadow part has Figure 1C In addition, due to the influence of light, the brightness of the display Figure 6 Distortion or the like may occur in the sinusoidal waveform of the brightness distribution shown.

[0073] The defective pixel detection section 304 detects defective pixels based on the feature amount of the stripe pattern recognized by the feature amount recognition section 303. For example, the defective pixel detection section 304 determines whether the feature amount is within a predetermined range and detects a pixel whose feature amount falls outside the predetermined range as a defective pixel. The predetermined range is, for example, a feature amount (such as amplitude A) assuming that there is no influence of multiple reflected light. m , contrast, and waveform distortion, etc.)

[0074] Fig. 8A and Figure 8B Each shows multiple reflected light in a groove of an object to be measured. Fig. 8A Displays multiple paths of projected light. Figure 8B Displays the brightness distribution of direct light and multiple reflected light on the plane P of the object to be measured. Fig. 8A In FIG. 1 , the solid line indicates direct light, and the dotted line indicates multi-reflected light. On the depth side of the groove, the distance over which projection light that reaches another position of the object to be measured is reflected from one position of the object to be measured is relatively small.

[0075] For example, in Fig. 8A and Figure 8B In the embodiment of the present invention, the difference between (i) the optical path of the direct light directly reaching the position A or the position B on the plane P of the object to be measured and (ii) the optical path of the multi-reflected light reaching the position A or the position B after being reflected on the plane O of the object to be measured is relatively small on the depth side of the groove. When the difference between the optical paths of the direct light and the multi-reflected light is small, the influence of the multi-reflected light is large because the difference between the intensities of the direct light and the multi-reflected light is large. In addition, when the difference between the optical paths of the direct light and the multi-reflected light is small, the multi-reflected light has the same period as the direct light. In this case, distortion is unlikely to occur in the synthetic light in which the direct light and the multi-reflected light are combined. For this reason, even if the pixel has been affected by the multi-reflected light, the defective pixel detection part 304 may not be able to detect the pixel as a defective pixel.

[0076] Here, when the defective pixel detection part 304 detects a pixel as a defective pixel, even if the pixels near the pixel are not detected as defective pixels, it is considered that the peripheral pixels are also likely to be affected by the multiple reflected light. In view of this, the defective pixel detection part 304 can assume that the pixels included in the distance range from the defective pixel to one cycle of the stripes on both sides of the defective pixel are defective pixels. This reduces the possibility that the defective pixel detection part 304 fails to detect the defective pixel. In this way, the influence of multiple reflected light in the geometric shape recognition result of the geometric shape recognition part 306 can be further reduced. For example, if the geometric shape recognition part 306 recognizes a groove of the object to be measured, the defective pixel detection part 304 detects the pixels included in the distance range from the defective pixel to one cycle of the stripes on both sides of the defective pixel as estimated defective pixels. Then, the geometric shape recognition part 306 re-recognizes the geometric shape based on the estimated defective pixel. Accordingly, the recognition accuracy of the geometric shape can be improved.

[0077] [Changes in feature quantities due to multiple reflected light]

[0078] Fig.9A and Fig. 9B Each shows a change in the feature amount of the gradation stripe pattern on the captured image due to multiple reflected light. Fig.9A and Fig. 9B , changes in the feature amount of a gradational fringe pattern on a captured image due to multiple reflected light will be described. Fig.9A and 9B Graph showing changes in direct light brightness distribution D and multi-reflected light brightness distribution M. Fig.9A and Fig. 9B The horizontal axes each indicate a distance in the width direction of the gradient stripe pattern of the captured image, and Fig.9A and 9B The vertical axis indicates brightness. The solid line indicates direct light brightness distribution D, and the one-dot chain line indicates multiple reflected light brightness distribution M. The dotted line indicates synthetic light brightness distribution C, that is, the distribution of synthetic light in which direct light and multiple reflected light are combined.

[0079] The characteristic quantities such as amplitude, contrast, etc. of the gradation stripe pattern are represented by a composite light brightness distribution C. This brightness distribution C is a brightness distribution obtained by combining the direct light brightness distribution D and the multi-reflected light brightness distribution M.

[0080] like Fig.9A As shown in the figure, when the phases of the direct light brightness distribution D and the multi-reflected light brightness distribution M are close, the amplitude A of the composite light brightness distribution C is m Greater than the amplitude A of the direct light brightness distribution D m On the other hand, if Fig. 9BAs shown in the figure, when the phase difference between the direct light brightness distribution D and the multi-reflected light brightness distribution M is large, the amplitude A of the composite light brightness distribution C m Smaller than the amplitude A of the direct light brightness distribution D m Therefore, the characteristic amount of the composite light brightness distribution C fluctuates according to the phase relationship between the direct light and the multi-reflected light. For this reason, the defective pixel detection section 304 determines the amplitude A by using only one gradation stripe pattern. m Whether it is within the predetermined range and the existence of multiple reflected light cannot be detected.

[0081] In addition, when the amplitude A of the gradient fringe pattern used to eliminate the influence of multiple reflected light is m When the lower limit value of is set high, there is a problem that the defective pixel detection section 304 may detect even a pixel in which subsurface scattering occurs as a defective pixel. In subsurface scattering, when the object to be measured contains ceramics, plastics, etc., after the projection light from the projection section 1 enters and is scattered inside the object to be measured, the light passes through the surface of the object to be measured. When subsurface scattering occurs, the amplitude is attenuated. The same problem also occurs with contrast, but this problem rarely occurs in the case of waveform distortion.

[0082] In addition, when the difference in phase and period between the direct light brightness distribution D and the multiple reflected light brightness distribution M is relatively small, it is difficult for the defective pixel detection section 304 to detect waveform distortion. For this reason, the defective pixel detection section 304 cannot fully detect multiple reflected light through waveform distortion of one gradient stripe pattern.

[0083] Therefore, when the feature quantity recognition section 303 sets the feature quantity recognized from the first captured image with the stripe pattern extending in the first direction as the first feature quantity, and sets the feature quantity recognized from the second captured image with the stripe pattern extending in the second direction as the second feature quantity, the defective pixel detection section 304 detects the defective pixel of the first captured image affected by the multiple reflections based on the first feature quantity and the second feature quantity. For example, the defective pixel detection section 304 first detects the pixel of the first captured image whose first feature quantity falls outside the predetermined range as a defective pixel. The predetermined range is a pixel such as the amplitude A when it is assumed that there is no influence of the multiple reflected light. m , contrast, waveform distortion, etc., and the range of values ​​of the first feature quantity and the second feature quantity.

[0084] Because the pixels corresponding to each other in the first captured image and the second captured image are pixels of different stripe patterns projected onto the same position of the object to be measured, the influence of the multiple reflected light is more likely to occur in the first image at the position of the second captured image where the influence of the multiple reflected light has occurred. Therefore, the defective pixel detection section 304 identifies the pixel of the first captured image corresponding to the pixel of the second captured image whose second feature amount falls outside the predetermined range, and detects the identified pixel of the first captured image as a defective pixel.

[0085] In addition, the first range for detecting defective pixels based on the first feature quantity and the second range for detecting defective pixels based on the second feature quantity may be different. For example, the defective pixel detection section 304 detects pixels of the first captured image whose first feature quantity falls outside the first range as defective pixels, and detects pixels of the second captured image whose second feature quantity falls outside the second range as defective pixels. In addition, the defective pixel detection section 304 identifies pixels of the first captured image corresponding to pixels of the second captured image identified as defective pixels, and detects these pixels as defective pixels. The first range and the second range are the first feature quantity and the second feature quantity (such as amplitude A) assuming that there is no influence of multiple reflected light. m , contrast and offset O f etc.) ranges of values.

[0086] In the second captured image, as Figure 2B As shown, the width of the stripe pattern of the captured image is unlikely to fluctuate due to the 3D geometry of the object to be measured because (i) the direction in which the orientation of the optical axes of the projection section 1 and the capture section 2 is deviated and (ii) the direction in which the width of the stripe pattern fluctuates are orthogonal to each other. For this reason, the second range for the defective pixel detection section 304 to determine whether the second feature amount of the second captured image is a defective pixel can be set to be narrower than the first range. By adopting this configuration, the accuracy of detecting defective pixels can be improved.

[0087] In addition, the defective pixel detection section 304 can detect a pixel whose difference between the first feature quantity and the second feature quantity falls outside a predetermined range as a defective pixel by comparing the first feature quantity and the second feature quantity. More specifically, the defective pixel detection section 304 compares the first feature quantity and the second feature quantity with respect to multiple types of feature quantities such as amplitude and contrast. For example, the defective pixel detection section 304 detects a pixel whose difference between the first feature quantity and the second feature quantity falls outside a predetermined range with respect to any one of amplitude and contrast, etc., as a defective pixel.

[0088] In addition, the defective pixel detection part 304 can specify a predetermined area in the captured image of the stripe pattern extending in the first direction and the second direction for each pixel, obtain the similarity between the first feature quantity and the second feature quantity of the pixels in the specified area, and compare the obtained similarity with a threshold. For example, the defective pixel detection part 304 can set the difference between the average value of the first feature quantity and the average value of the second feature quantity in the specified area as the similarity. In addition, the defective pixel detection part 304 can calculate the spatial variation (difference amount) of the feature quantity for the first feature quantity and the second feature quantity in the specified area, and set the obtained difference as the similarity. In this way, the defective pixel detection part 304 can more significantly detect the influence of multiple reflections. The threshold is the amplitude A as the similarity between the first feature quantity and the second feature quantity when it is assumed that there is no influence of multiple reflected light. m When the similarity between the first feature quantity and the second feature quantity is lower than the threshold value, the defective pixel detection section 304 detects the pixels in the designated area as defective pixels. On the other hand, when the similarity between the first feature quantity and the second feature quantity is equal to or greater than the threshold value, the defective pixel detection section 304 does not detect the pixels in the designated area as defective pixels.

[0089] [Calculation method of feature quantity based on captured image]

[0090] Fig.10 A method of calculating a feature amount (waveform distortion) based on a captured image is shown. Fig.10 The vertical axis indicates FIG. 5A to FIG. 5D The brightness of the same pixel obtained in the captured image while a gradient fringe pattern is projected onto the object to be observed. Fig.10 The horizontal axis indicates FIG. 5A to FIG. 5D The phase corresponding to the gradient fringe pattern. Fig.10 Each black circle in the FIG. 5A to FIG. 5D The observed values ​​of the brightness of the same pixel when the gradient fringe pattern is sequentially projected onto the object to be measured. Fig.10 The dotted line in indicates a sine wave fitted to the observed value represented by each black circle. The difference between the brightness corresponding to each black circle and the brightness of the sine wave in the same phase is equal to the deviation from the ideal value of the observed value. The feature quantity identification section 303 can normalize the magnitude of the waveform distortion by accumulating the deviation corresponding to each observed value and by normalizing the accumulated deviation with the amplitude.

[0091] The projection control section 301 sequentially FIG. 5A to FIG. 5D The gradual stripe pattern is projected onto the object to be measured. Then, the capture part 2 generates the projection control part 301 in sequence. FIG. 5A to FIG. 5D A captured image of an object to be measured is captured when a gradient fringe pattern is captured. FIG. 5A to FIG. 5DThe phases of the gradient stripe patterns are different from each other by 90 degrees. For this reason, the feature quantity identification section 303 will FIG. 5A to FIG. 5D The gradient fringe pattern corresponding to the brightness of the identified pixels of the captured image is plotted on Fig.10 On the graph, each displacement is 90 degrees.

[0092] To create Fig.10 The feature quantity identification section 303 calculates the brightness value l as the observed value when each gradation stripe pattern is projected by the following expression: n (x, y) is normalized, where n represents a step number (n=1 to N), and N represents a step number indicating the number of projection images including a gradation fringe pattern extending in the same direction.

[0093]

[0094] Furthermore, the feature quantity identification section 303 identifies the sine wave having the highest matching rate with each plotted point, such as Fig.10 The feature quantity identification section 303 uses the following expression to identify the brightness value l in each pixel (x, y). n (x, y) to obtain the phase ψ(x, y) to obtain a sine wave:

[0095]

[0096] In this case, the feature quantity identification section 303 obtains the deviation between the drawing brightness of each gradation stripe pattern and the identified sine wave by the following expression: This deviation is a feature quantity for detecting distortion or the like of the sine waveform.

[0097]

[0098] The defective pixel detection section 304 compares the deviation with a threshold value, and detects a defective pixel when the deviation is greater than the threshold value. The threshold value is a maximum value of the deviation assumed to occur in a pixel not affected by the multi-reflected light. On the other hand, when the deviation is equal to or less than the threshold value, the defective pixel detection section 304 does not detect the defective pixel.

[0099] [Identify the location of the pixels that captured the image]

[0100] The analysis section 305 identifies the correspondence between the pixels of the projection image and the pixels of the captured image acquired by the acquisition section 302 by analyzing the gradient information of the gradient stripe pattern of the captured image. The correspondence is represented by information showing the position (or coordinates) of the projected pixel corresponding to the pixel of the captured image. If pixel A of the captured image is a pixel obtained by capturing pixel B of the projection image, pixel A and pixel B have a correspondence with each other. A method for obtaining the correspondence between the pixels of the projection image and the pixels of the captured image will be described below.

[0101] As mentioned above, FIG. 4C to FIG. 4F The binary stripe pattern shown in corresponds to Gray code. Fig.11 In FIG. 4C to FIG. 4F An example of a binary fringe pattern is shown in . By associating 0 in the Gray code with non-projected areas and 1 with light projected areas, Figures 4C to 4F The binary fringe pattern shown.

[0102] FIG. 4A to FIG. 4F and Fig.11 Each position in the x direction in is represented by a code value, which is a combination of the numbers 0 or 1 at each position in the Gray code. Fig.11 Position 0 in corresponds to the code value "0000", position 1 corresponds to the code value "0001", and position 15 corresponds to the code value "1000".

[0103] In the projection control section 301, the capture section 2 Figure 4A and Figure 4B The standard patterns shown in FIG. 1 are projected onto the object to be measured and the object to be measured is captured at the same time. The analysis part 305 obtains the average value of the two captured standard patterns as the median for each pixel. Similarly, FIG. 4C to FIG. 4F The analysis section 305 identifies the code value of each pixel by comparing the brightness value of each pixel in the four captured images with the corresponding median value. By identifying the code value, the analysis section 305 can identify which binary fringe is reflected at each pixel position in the binary fringe pattern projected toward different positions.

[0104] In addition, the analyzing section 305 identifies Fig.10 1 and 2 show the pixel positions of the projected image that match the phase of the sine wave identified by the feature quantity identification section 303. Since the gradation stripe pattern of the projected image has periodicity, there are a plurality of pixel positions of the projected image that match the phase of the identified sine wave.

[0105] Therefore, the analyzing section 305 obtains FIG. 4C to FIG. 4FThe analysis section 305 obtains the correspondence between the pixels of the projected image and the pixel positions of the captured image by selecting the correspondence closest to the correspondence obtained based on the binary stripe pattern from the multiple correspondences obtained by analyzing the gradient information of the gradient stripe pattern.

[0106] It should be noted that the analysis section 305 may use a projection image including a gradient fringe pattern instead of using a projection image including a binary fringe pattern. The analysis section 305 may be configured to have FIG. 5A to FIG. 5D In addition to the projected image of the gradient stripe pattern, by sequentially projecting a gradient stripe pattern with a sinusoidal brightness distribution (where the stripe width is different from FIG. 5A to FIG. 5D The corresponding relationship between the pixels of the captured image and the pixels of the projected image can be obtained by projecting multiple projection images having a first periodic gradient stripe pattern (a stripe width of a certain value). For example, the projection part 1 can be configured to project multiple projection images having a first periodic gradient stripe pattern, project multiple projection images having a second periodic gradient stripe pattern, and project multiple projection images having a third periodic gradient stripe pattern. In this case, the projection part 1 can identify the geometric shape of the object to be measured by projecting a projection image having a sinusoidal brightness distribution. In addition, multiple projection images having the first to third periodic gradient stripe patterns can be projected as gradient stripe patterns extending along the first and second directions.

[0107] Furthermore, the defective pixel detected by the defective pixel detection section 304 does not present a brightness that matches the stripe pattern of the projection image due to the influence of multiple reflections. Therefore, the analysis section 305 obtains the correspondence between (i) the pixels of the captured image other than the defective pixel detected by the defective pixel detection section 304 among the pixels of the captured image and (ii) the pixels of the projection image. In this way, the analysis section 305 can reduce the probability of erroneously obtaining the correspondence between the pixels of the captured image and the pixels of the projection image.

[0108] The geometric shape recognition section 306 obtains the 3D geometric shape of the object to be measured based on the correspondence between the pixels of the captured image obtained by the analysis section 305 and the projection image. The arrangement of the projection section 1 and the capture section 2 and the directions of the optical axes of the projection section 1 and the capture section 2 are known. For this reason, by using the principle of the triangulation method, the geometric shape recognition section 306 obtains the 3D geometric shape of the object to be measured by, for example, obtaining the 3D position of each pixel of the captured image from the correspondence between the captured image obtained by the analysis section 305 and the projection image.

[0109] [Processing of 3D Geometry Measurement Device 100]

[0110] Fig.121 is a flowchart showing an example of the operation of the 3D geometry measuring device. When the user issues an instruction to measure the object to be measured by the 3D geometry measuring device 100 through, for example, an operation key not shown, the processing starts. First, the projection control section 301 projects a standard pattern (full black pattern) in which light is not projected onto any part of the object to be measured, and while the projection control section 301 projects the aforementioned standard pattern, the capture section 2 captures the object to be measured. Then, the projection control section 301 projects a standard projection pattern (full white pattern) in which light is projected onto the entire object to be measured, and while the projection section 1 projects the aforementioned standard pattern, the capture section 2 captures the object to be measured (S101).

[0111] In addition, the projection control section 301 sequentially projects a plurality of projection images including a binary fringe pattern extending in a first direction, and while the projection control section 301 projects each projection image, the capture section 2 captures the object to be measured. In addition, the projection control section 301 sequentially projects a plurality of projection images including a binary fringe pattern extending in a second direction, and while the projection control section 301 projects each projection image, the capture section 2 captures the object to be measured (S102).

[0112] Next, the projection control section 301 sequentially projects a plurality of projection images including a gradient stripe pattern having a sinusoidal brightness distribution and extending in a first direction, and while the projection section 1 projects each projection image, the capture section 2 captures the object to be measured. In addition, the projection control section 301 sequentially projects a plurality of projection images including a gradient stripe pattern having a sinusoidal brightness distribution and extending in a second direction, and while the projection control section 301 projects each projection image, the capture section 2 captures the object to be measured (S103).

[0113] The analysis section 305 identifies the code value of each pixel based on the brightness value of each pixel of the captured image projecting the binary stripe pattern extending in the first direction and the second direction (S104). In addition, the analysis section 305 analyzes the gradient information of the captured image projecting the gradient stripe pattern extending in the first direction and the second direction (S105).

[0114] The feature amount identification section 303 identifies feature amounts indicating features of stripe patterns included in (i) a captured image projecting a gradation stripe pattern extending in the first and second directions and (ii) a captured image projecting a binary stripe pattern extending in the first and second directions ( S106 ).

[0115] Next, the defective pixel detection section 304 detects defective pixels by identifying pixels whose feature quantities identified by the feature quantity identification section 303 are not within a predetermined range (S107). First, the defective pixel detection section 304 detects a pixel of the first captured image whose first feature quantity falls outside the predetermined range as a defective pixel, the first feature quantity being a feature quantity identified by the feature quantity identification section 303 based on a stripe pattern extending in a first direction in the first captured image. In addition, the defective pixel detection section 304 identifies a pixel of the first captured image corresponding to a pixel of the second captured image whose second feature quantity falls outside the predetermined range, the second feature quantity being a feature quantity identified by the feature quantity identification section 303 based on a stripe pattern extending in a second direction in the second captured image. The defective pixel detection section 304 detects the identified pixel of the first captured image as a defective pixel.

[0116] The defective pixel detection section 304 compares the first feature amount and the second feature amount ( S108 ), and detects a pixel, in which the difference between the first feature amount and the second feature amount falls outside a predetermined range, as a defective pixel.

[0117] The analysis part 305 obtains the correspondence between the pixels of the defective pixels of the captured image and the pixels of the projected image other than the defective pixels by using the identified code value. In addition, the analysis part 305 obtains the correspondence between the pixels of the captured image other than the defective pixels and the pixels of the projected image by analyzing the gradient information of the gradient stripe pattern (S109). The analysis part 305 identifies the correspondence between the pixels of the captured image and the projected image by selecting the correspondence closest to the correspondence obtained using the code value from the multiple correspondences obtained by analyzing the gradient information. The geometric shape recognition part 306 obtains the 3D geometric shape of the object to be measured based on the correspondence between the pixels of the captured image and the projected image obtained by the analysis part 305 (S110).

[0118] According to the present embodiment, the defective pixel detection section 304 detects defective pixels of a captured image affected by multiple reflections using a first feature quantity of a first captured image identified from a captured image of a stripe pattern extending in a first direction and a second feature quantity identified from a second captured image of a stripe pattern extending in a second direction. The second captured image is different from the first captured image in terms of brightness distribution and in terms of the way in which multiple reflected light and direct light overlap. Therefore, even when the defective pixel detection section 304 fails to detect multiple reflected light due to reasons such as overlap of multiple reflected light and direct light in the first captured image, it can be expected that the defective pixel detection section 304 detects multiple reflected light in the second captured image.

[0119] Moreover, if multiple reflections occur on the measurement target, the feature quantity changes depending on the direction of the projection pattern. However, even when the direction of the projection pattern changes, if multiple reflections do not occur on the measurement target, the feature quantity is unlikely to change significantly. Therefore, by comparing the first feature quantity and the second feature quantity, the multiple reflected light can be detected more accurately. Therefore, the defective pixel detection part 304 can improve the detection accuracy of the defective pixel by detecting the defective pixel based on both the first feature quantity and the second feature quantity.

[0120] In addition, since the 3D geometry measuring device 100 uses (i) a first feature quantity recognized from a captured image of a stripe pattern extending in a first direction and (ii) a second feature quantity recognized from a captured image of a stripe pattern extending in a second direction, the administrator of the 3D geometry measuring device 100 does not need to strictly set the thresholds of the respective feature quantities. For example, the administrator of the 3D geometry measuring device 100 may set the range of feature quantities used to determine whether a pixel is a defective pixel to be larger than the range in the case where only one of the first feature quantity and the second feature quantity is used. Moreover, if the defective pixel detection section 304 uses a plurality of feature quantities, the same effect can be achieved. Therefore, while excluding the influence of multiple reflections, the geometry recognition section 306 is able to measure a target with large subsurface scattering that is prone to erroneously detecting a defective pixel when a strict threshold is set.

[0121] In addition, according to the present embodiment, the defective pixel detection section 304 detects pixels within a range of a distance from a defective pixel to one cycle of stripes corresponding to both sides of the defective pixel as defective pixels. Therefore, due to the fact that the cycle of direct light and the cycle of the stripe pattern are close and the detection of defective pixels is difficult, even when distortion is unlikely to occur in the synthetic light of the combination of direct light and multiple reflected light, the defective pixel detection section 304 can accurately remove pixels affected by multiple reflected light.

[0122] It should be noted that in the present embodiment, an example of a case where the projection control section 301 sequentially projects a stripe pattern extending in a first direction and a second direction onto an object to be measured is described. However, the present invention is not limited to the example of a case where a stripe pattern extending in multiple directions is continuously projected onto an object to be measured. For example, the projection control section 301 may be configured to project a stripe pattern extending in a first direction onto an object to be measured and project a stripe pattern extending in a second direction when the amount of defective pixels detected by the defective pixel detection section 304 based on the first feature amount is outside a standard range. The standard range may be obtained statistically, for example, using a range of defective pixels and a desired measurement accuracy. In addition, the standard range may be obtained using, for example, a ratio of a range of pixels that do not correspond to defective pixels to a range of defective pixels.

[0123] In addition, in the present embodiment, an example is described in which the projection control section 301 projects a projection image including a stripe pattern extending in a first direction and a projection image including a stripe pattern extending in a second direction onto an object to be measured. However, the present invention is not limited to the example of the case where a stripe pattern extending in the first direction and the second direction is projected. For example, the projection control section 301 may also be configured to project a projection image including a stripe pattern that does not extend in each of the first direction and the second direction onto the object to be measured when the amount of defective pixels detected by the defective pixel detection section 304 is outside a predetermined range. The predetermined range may be statistically obtained, for example, using the range of defective pixels and the desired measurement accuracy. In addition, the predetermined range may be obtained using, for example, a ratio of a range of pixels that do not correspond to defective pixels to a range of defective pixels. On the other hand, when the amount of defective pixels detected by the defective pixel detection section 304 is within the predetermined range, the projection control section 301 does not project a projection image including a stripe pattern extending in different directions onto the object to be measured. By adopting such a configuration, the defective pixel detection section 304 can improve the accuracy of detecting defective pixels.

[0124] In addition, the projection control part 301 may be configured to repeat the process of projecting a projection image including a stripe pattern extending in another direction onto the object to be measured. For example, the projection control part 301 sequentially projects a stripe pattern extending from a first direction to an Nth direction (N is a natural number). The projection control part 301 may be configured to stop projecting the stripe pattern based on the range of defective pixels newly detected from the captured image projecting the stripe pattern extending in the Nth direction. That is, the projection control part 301 may be configured to stop projecting the stripe pattern when the range of defective pixels not detected from the captured image captured when the stripe pattern extending in the first direction to the Nth direction is projected is equal to or less than a threshold value among the defective pixels detected in the captured image captured when the stripe pattern extending in the Nth direction is projected. The threshold value is, for example, a value indicating that the influence of multiple reflected light has become sufficiently small.

[0125] In addition, the projection control part 301 may project a projection image including a stripe pattern extending in a first direction onto the object to be measured, and then additionally project a projection image having a stripe pattern extending in the first direction and having different periods onto the object to be measured. In addition, the projection control part 301 may project a projection image including a stripe pattern extending in a second direction onto the object to be measured, and then project a projection image having a stripe pattern extending in the second direction and having different periods onto the object to be measured. When a stripe pattern having different periods is projected, the phase of the multiple reflected light overlapping with the direct light changes. For this reason, the defective pixel detection part 304 can more accurately detect pixels affected by the multiple reflected light.

[0126] Preferably, the defective pixel detection section 304 uses stripe patterns with the same period to compare the feature quantities. As described above, this also applies to the case where multiple gradation stripe patterns with different periods are used instead of binary patterns when obtaining the correspondence between pixels of the captured image and the projected image.

[0127] In addition, in the present embodiment, an example of a case where the projection control section 301 projects each projection image using projection light having the same wavelength is described. However, the present invention is not limited to this case. For example, the projection control section 301 may project more than two projection images including a stripe pattern extending in a first direction onto an object to be measured using light of a first wavelength, and may project a projection image including a stripe pattern extending in a second direction onto an object to be measured using light of a second wavelength. By adopting this configuration, it is possible to project a stripe pattern extending in a first direction and a stripe pattern extending in a second direction onto an object to be measured, and it is possible to shorten the measurement time. The threshold value may be changed for each wavelength.

[0128] In addition, in the present embodiment, an example is described in which the projection control section 301 projects a projection image including a stripe pattern extending in the second direction onto an object to be measured for both a gradual stripe pattern and a binary pattern having a sinusoidal brightness distribution. However, the present invention is not limited thereto. For example, the projection control section 301 may adopt a configuration in which a gradual stripe pattern extending in the second direction is projected onto an object to be measured, but a binary stripe pattern extending in the second direction does not need to be projected onto the object to be measured. Thus, the projection control section 301 can reduce the amount of measurement time.

[0129] Moreover, the projection control section 301 does not need to project the same number of projection images of the gradient stripe pattern or the binary stripe pattern to be projected in the first direction and the second direction. For example, the projection control section 301 can make the number of projection images of the gradient stripe pattern in the second direction less than that in the first direction (for example: for the first direction, 6 phase shifts per 60 degrees; for the second direction, 4 phase shifts per 90 degrees; etc.).

[0130] In addition, in the present embodiment, an example is described in which the analysis section 305 obtains the corresponding relationship between the pixels of the projected image and the captured image by respectively analyzing the captured image of the object to be measured while the gradient stripe pattern and the binary stripe pattern are projected. However, the present invention is not limited to this case. For example, the following configuration may be adopted, so that the analysis section 305 obtains the corresponding relationship between the pixels of the projected image and the captured image by analyzing the captured image of the object to be measured while the binary stripe pattern is projected, and the projection control section 301 does not project the gradient stripe pattern. In this case, the feature quantity recognition section 303 may obtain information such as the offset O from the captured image projected with the binary stripe pattern. f The characteristic quantity of .

[0131] In addition, in the present embodiment, an example is described in which the projection control section 301 projects a projection image including a stripe pattern extending in a first direction and a projection image including a stripe pattern extending in a second direction onto an object to be measured, wherein the first direction and the second direction are orthogonal to each other. However, the present invention is not limited to this case. For example, the first direction and the second direction may be approximately orthogonal to each other. The first direction may be approximately orthogonal to each other. Figure 2B The second direction may be angled to any degree other than that shown. The second direction may be angled to any degree different from the first direction.

[0132] In addition, in the present embodiment, an example is described in which the defective pixel detection section 304 detects a pixel as a defective pixel when the difference between the first feature quantity and the second feature quantity falls outside a predetermined range. However, the present invention is not limited to this case. For example, the defective pixel detection section 304 may detect a pixel as a defective pixel when the ratio between the first feature quantity and the second feature quantity falls outside a predetermined range.

[0133] The characteristics of the 3D geometry measuring device and the 3D geometry measuring method according to the first embodiment will be summarized.

[0134] The 3D geometry measurement device comprises:

[0135] (1) a projection section that sequentially projects two or more projection images including a fringe pattern extending in a first direction, and projects at least one projection image including a fringe pattern extending in a second direction onto an object to be measured;

[0136] a capturing part, respectively generating captured images including the fringe pattern projected by the projecting part onto the object to be measured;

[0137] a feature amount recognition section that recognizes a feature amount indicative of a stripe pattern included in a captured image;

[0138] a defective pixel detecting section that detects a defective pixel in a captured image affected by multiple reflections based on (i) a first feature quantity recognized from a captured image of a stripe pattern extending in a first direction and (ii) a second feature quantity recognized from a captured image of a stripe pattern extending in a second direction, both of which are recognized by the feature quantity recognition section;

[0139] an analyzing portion, obtaining a correspondence between (i) pixels of a captured image defective pixel and (ii) pixels of a projected image excluding the defective pixel; and

[0140] The geometric shape recognition section recognizes the 3D geometric shape of the object to be measured based on the correspondence relationship obtained by the analysis section.

[0141] (2) The 3D geometry measuring device according to (1), wherein the defective pixel detection section detects a pixel whose first feature amount or second feature amount in the captured image falls outside a predetermined range as a defective pixel.

[0142] (3) The 3D geometry measuring device according to (1) or (2), wherein the defective pixel detection section detects a pixel in which a difference between the first feature amount and the second feature amount in the captured image is equal to or greater than a threshold value as a defective pixel.

[0143] (4) A 3D geometric shape measuring device according to any one of (1) to (3), wherein the defective pixel detection part detects pixels in the captured image whose first feature quantity falls outside the first range as defective pixels, and detects pixels in the captured image whose second feature quantity falls outside the second range as defective pixels.

[0144] (5) A 3D geometry measuring device according to any one of (1) to (4), wherein the projection part sequentially projects a binary image and an image with a sinusoidal brightness distribution as the above-mentioned two or more projection images, and the projection image includes a stripe pattern extending along the first direction.

[0145] (6) A 3D geometry measuring device according to any one of (1) to (5), wherein the first direction is a direction orthogonal to a plane including an optical axis of the projection part and a capture part, and the second direction is a direction parallel to a plane including an optical axis of the projection part and a capture part.

[0146] (7) A 3D geometry measuring device according to any one of (1) to (6), wherein the defective pixel detection portion detects pixels within a range of distances from the defective pixel to one cycle of stripes on both sides of the detected defective pixel as defective pixels.

[0147] (8) A 3D geometry measuring device according to any one of (1) to (7), wherein the projection part projects the two or more projection images including a stripe pattern extending in a first direction onto an object to be measured using light of a first wavelength, and projects the projection image including a stripe pattern extending in a second direction onto the object to be measured using light of a second wavelength.

[0148] (9) A 3D geometric shape measuring device according to any one of (1) to (8), wherein, when the range of defective pixels detected by the defective pixel detection part is outside a predetermined range, the projection part projects a projection image including a stripe pattern that does not extend along the first direction or the second direction onto the object to be measured.

[0149] (10) A 3D geometric shape measuring device according to any one of (1) to (9), wherein, when the range of defective pixels detected by the defective pixel detection part based on the first feature quantity is larger than the standard range, the projection part projects a projection image including a stripe pattern extending along the second direction onto the object to be measured.

[0150] (11) A 3D geometry measuring device according to any one of (1) to (10), wherein the projection portion sequentially projects a projection image including a stripe pattern having a sinusoidal brightness distribution with different widths as a projection image including a stripe pattern extending along a first direction and a second direction.

[0151] (12) 3D geometric shape measurement methods, including:

[0152] a projecting step of sequentially projecting two or more projection images including a fringe pattern extending in a first direction, and projecting a projection image including a fringe pattern extending in a second direction onto an object to be measured;

[0153] a capturing step of respectively generating captured images including the fringe pattern projected onto the object to be measured in the projecting step;

[0154] a feature amount identifying step of identifying a feature amount indicative of a stripe pattern included in the captured image;

[0155] a defective pixel detecting step of detecting a defective pixel in a captured image affected by multiple reflections based on a first feature quantity identified from a captured image of a stripe pattern extending in a first direction and a second feature quantity identified from a captured image of a stripe pattern extending in a second direction, both of which are identified in the feature quantity identifying step;

[0156] an analyzing step of obtaining a correspondence between (i) pixels of the captured image defective pixels and (ii) pixels of the projected image excluding the defective pixels; and

[0157] The geometric shape recognition step recognizes the 3D geometric shape of the object to be measured based on the correspondence obtained in the analysis step.

[0158] <Second Embodiment>

[0159] In the first embodiment, an example of a case where the defective pixel detection section 304 detects a defective pixel by determining whether the feature quantity recognized by the feature quantity recognition section 303 is within a predetermined range is described. However, the present invention is not limited to this case. On the other hand, in the second embodiment, an example of a case where a defective pixel is detected based on the positional relationship between a pixel and an epipolar line will be described.

[0160] [Principle of defective pixel detection]

[0161] Fig.13 and Fig.14 The principle of detecting a defective pixel according to the second embodiment is shown. Fig.13 Indicates an example of a case where a measurement point MP on an object to be measured is captured by a pair of capturing portions. Fig.13 The pixel position on the captured image plane of the right side capture portion is the captured pixel position A1, which corresponds to the same measurement position MP. Fig.13 The pixel position on the captured image plane of the captured part on the left is the captured pixel position A2. In this case, the captured pixel positions A1 and A2 have a fixed relationship based on the positional relationship between the two captured parts. The straight line projected onto the captured image plane of the captured part on the left is the first epipolar line EL1, which is a straight line extending from the optical center O1 of the captured part on the right through the captured pixel position A1 to the measurement point MP. The first epipolar line EL1 is indicated by a dotted line in an oblique direction. Due to the nature of geometric shape restrictions, the captured pixel position A2 is at any position on the first epipolar line EL1 projected onto the captured image.

[0162] In addition, the straight line projected onto the captured image plane of the captured portion on the right is the second epipolar line EL2, which is a straight line extending from the optical center O2 of the captured portion 2 on the left through the captured pixel position A2 to the measurement point MP. Due to the nature of the geometric shape restriction, the captured pixel position A1 is at any position on the second epipolar line EL2.

[0163] On the captured image plane on the left, a plurality of first epipolar lines may be drawn. Each first epipolar line intersects with another first epipolar line at one point (i.e., epipolar point EP1). Similarly, on the captured image plane on the right, a plurality of second epipolar lines may be drawn. Each second epipolar line intersects with another second epipolar line at one point (i.e., epipolar point EP2).

[0164] Fig.14An example is shown in which the projection section 1 projects a projection image including a fringe pattern onto an object to be measured, and the capture section 2 captures the projection image projected onto the object to be measured. Fig.14 In the case of , when the projection part 1 is regarded as the capture part by assuming that the direction of the light beam is in the opposite direction, the projection pixel position A1′ and the capture pixel position A2 as the position of the pixel of the projected image are in the same direction as Fig.13 When the optical center of the projection part 1 is O1′, the straight line projected onto the captured image plane of the capture part 2 is the first epipolar line EL1, which is a straight line extending from the optical center O1′ of the projection part 1 through the projection pixel position A1′ to the measurement point MP.

[0165] The capture pixel position A2 is at any position on the first epipolar line EL1. On the other hand, if the capture pixel position A2 is not on the first epipolar line EL1, multiple reflections may occur on the optical path between the projection pixel position A1′ and the capture pixel position A2. Using this principle, the defective pixel detection part 502 detects defective pixels by comparing the positions of the capture pixel position A2 and the first epipolar line EL1.

[0166] Moreover, when the optical center of the capture part 2 is O2, the straight line projected onto the projection image plane of the projection part 1 is the second epipolar line EL2, which is a straight line extending from the optical center O2 of the capture part 2 through the capture pixel position A2 to the measurement point MP. The projection pixel position A1′ is located at any position on the second epipolar line EL2. On the other hand, if the projection pixel position A1′ is not on the second epipolar line EL2, there is a possibility that the pixel at the projection pixel position A1′ is a defective pixel. Using this principle, the defective pixel detection part 502 can detect defective pixels by comparing the positions of the projection pixel position A1′ and the second epipolar line EL2.

[0167] [Configuration of 3D Geometry Measurement Device 200 According to Second Embodiment]

[0168] Fig.15 The configuration of a 3D geometry measuring device 200 according to the second embodiment is shown. The 3D geometry measuring device 200 includes a projection section 1, a capture section 2, a control section 3, and a memory section 4. The control section 3 is, for example, a CPU and functions as a projection control section 301, an acquisition section 302, a line recognition section 501, a defective pixel detection section 502, an analysis section 305, and a geometry recognition section 306 by executing a program stored in the memory section 4. Figure 3 The same functional blocks as those shown in FIG. 1 are denoted by the same reference numerals and the related descriptions are omitted.

[0169] The projection control section 301 includes Figure 3The projection control part 301 has the same function as the function of the projection control part 301. The projection control part 301 controls the circuit of turning on / off the projection part 1 for each pixel, so the projection control part 301 is able to project only a part of the pixels of the projection part 1. Since there are many defective pixels caused by multiple reflected light, if the projection image in which the projection image pixels at the position corresponding to the defective pixel are removed is the projection defective pixel, the influence of the multiple reflected light can be prevented. Therefore, when the defective pixel detection part 502 detects a defective pixel, the projection control part 301 projects the projection image generated by removing the defective pixel detected by the defective pixel detection part 502 from the pixels of the original projection image (hereinafter referred to as the "first selected projection image") onto the object to be measured.

[0170] It can be considered that by projecting a projection image including a plurality of pixels onto the object to be measured at one time, defective pixels due to multiple reflections occur. Therefore, when the defective pixel detection section 502 detects a defective pixel, the projection control section 301 projects an image including the defective pixel detected by the defective pixel detection section 502 (hereinafter referred to as a second selected projection image) onto the object to be measured, and performs measurement again to reduce the number of pixels to be projected at one time. The second selected projection image is, for example, a projection image composed of defective pixels detected by the defective pixel detection section 502.

[0171] The line identification part 501 identifies the first epipolar line of the capture part 2 corresponding to the projection pixel image or the second epipolar line of the projection part 1 corresponding to the capture pixel position. The position of the first epipolar line of the capture part 2 relative to any one projection pixel position of the projection part 1 is predetermined. The relationship between the projection pixel position and the first epipolar line is uniquely determined based on the relationship between the projection part 1 and the capture part 2, and is not affected by the position or size of the object to be measured. Similarly, the position of the second epipolar line of the projection part 1 relative to any one capture pixel position of the capture part 2 is predetermined. The memory part 4 pre-stores the position of the first epipolar line of the capture part 2 relative to each projection pixel position and the position of the second epipolar line of the projection part 1 relative to each capture pixel position of the capture part 2.

[0172] The line recognition section 501 recognizes the position of the first epipolar line relative to the projection pixel position and the position of the second epipolar line relative to the capture pixel position by reading the position of the epipolar line stored in the memory section 4. The line recognition section 501 sends information on the position of the recognized first epipolar line or the second epipolar line to the defective pixel detection section 502.

[0173] The analysis section 305 has a similar function to that in the first embodiment. The analysis section 305 obtains a correspondence relationship between a projection pixel position, which is a position of a projection pixel, and a capture pixel position, which is a position of a capture pixel acquired by the acquisition section 302 .

[0174] The analysis section 305 obtains the absolute phase value of each captured pixel position by analyzing the gradual change of the pattern in the first direction and the second direction having a sinusoidal brightness distribution in the captured image. AP,k (I, j) is represented by the following expression, where i and j respectively indicate the i-th pixel from the left edge in the second direction and the j-th pixel from the top edge in the first direction.

[0175]

[0176] In the above expression, and is the phase value of the fringe in the fringe pattern. In addition, m i and m j The number of stripes from the left edge and the top edge are shown respectively. The projected pixel position (i, j) corresponding to the captured pixel position (i, j) p ,j p ) can be obtained by the following expression. It should be noted that p k is the number of pixels of the projection portion 1 corresponding to one period of the fringes in the fringe pattern.

[0177]

[0178] In the case where the projection control section 301 projects the first selected projection image that does not include the defective pixel, the analysis section 305 estimates the correspondence between the projection pixel position of the first selected projection image and the captured pixel of the captured image of the object to be measured on which the first selected projection image is projected, for example, in the re-measurement. Since the defective pixel may be affected by multiple reflected light, the projection control section 301 prevents the influence of multiple reflected light by projecting the first selected projection image that does not include the defective pixel. Therefore, the analysis section 305 can accurately obtain the correspondence between the projection pixel position and the captured pixel position.

[0179] For example, when the projection control section 301 projects the second selected projection image including the defective pixel during the re-measurement, the analysis section 305 estimates the correspondence between the projection pixel position of the second selected projection image and the captured pixel position of the captured image. Compared with the case where all the projection images are projected, the projection control section 301 reduces the number of pixels simultaneously projected by projecting the second selected projection image. The analysis section 305 can increase the number of pixels used to identify the 3D geometry of the object to be measured by re-estimating the correspondence of the pixels detected as defective pixels by the defective pixel detection section 502.

[0180] The defective pixel detection section 502 detects defective pixels based on the relationship between the captured pixel position and the first epipolar line or the relationship between the projected pixel position and the second epipolar line. If the distance between the captured pixel position and the first epipolar line corresponding to the projected pixel position having a corresponding relationship with the captured pixel position is greater than or equal to a first threshold, or if the distance between the projected pixel position and the second epipolar line corresponding to the captured pixel position having a corresponding relationship with the projected pixel position is greater than or equal to a second threshold, the defective pixel detection section 502 detects the pixel corresponding to the captured pixel position or the projected pixel position as a defective pixel.

[0181] Fig.16 An operation of detecting a defective pixel performed by the defective pixel detecting section 502 is shown. Fig.16 The case where the projection light from the projection part 1 is multiply reflected is shown. The multiply reflected light is indicated by a dotted line. The projection light from the projection pixel position A1′ on the image plane of the projection part 1 reaches the capture pixel position A2 on the image plane of the capture part 2 as the multiply reflected light multiply reflected on the object to be measured.

[0182] like Fig.14 As shown in FIG. 1 , if the projection light reaches the capturing portion 2 after being reflected only once on the object to be measured, the captured image position A2 is located on the first epipolar line EL1 corresponding to the projection pixel position A1′. Fig.16 As shown, if the projection light reaches the capturing part 2 after multiple reflections on the object to be measured, the captured image position A2 does not match the first epipolar line EL1 corresponding to the projected pixel position A1'. In these cases, the defective pixel detection part 502 determines that the pixel at the captured pixel position A2 is a defective pixel.

[0183] [Defect determination of capture part 2]

[0184] The defective pixel detection section 502 obtains the position of the first epipolar line EL1 of the first epipolar line of the capture section 2 identified by the line recognition section 501 as corresponding to the projection pixel position A1′. The defective pixel detection section 502 determines whether the projection light reaching the capture pixel position A2 is multi-reflected light by comparing the positions of the capture pixel position A2 and the first epipolar line EL1. More specifically, the defective pixel detection section 502 obtains the distance D1 between the capture pixel position A2 and the first epipolar line EL1, and compares the obtained distance D with a first threshold value. The first threshold value is a value showing the distance between the capture pixel positions A2 that the projection light can reach after being reflected only once on the object to be measured.

[0185] If the distance between the capture pixel position A2 and the first epipolar line EL1 is greater than or equal to the first threshold value, the defective pixel detection part 502 detects the pixel at the capture pixel position A2 as a defective pixel. In addition, the defective pixel detection part 502 detects the pixel at the projection pixel position A1′ as a defective pixel. The projection pixel position A1′ is a position estimated by the analysis part 305 to have a corresponding relationship with the capture pixel position A2 where the defective pixel is detected. The defective pixel detection part 502 sends information about the position of the detected pixel to the analysis part 305 and the geometric shape recognition part 306. On the other hand, if the distance between the capture pixel position A2 and the first epipolar line EL1 is less than or equal to the first threshold value, the defective pixel detection part 502 does not detect the pixels at the capture pixel position A2 and the projection pixel position A1′ as defective pixels. In this case, the defective pixel detection part 502 sends information about the distance D1 between the capture pixel position A2 and the first epipolar line EL1 to the geometric shape recognition part 306.

[0186] [Defect determination of projection part 1]

[0187] In addition, the defective pixel detection part 502 obtains the position of the second core line EL2 of the second core line of the projection part 1 identified by the line identification part 501 as corresponding to the capture pixel position A2. The defective pixel detection part 502 determines whether the projection light from the projection pixel position A1′ is a multiple reflection light by comparing the positions of the projection pixel position A1′ and the second core line EL2. More specifically, the defective pixel detection part 502 obtains the distance D2 between the projection pixel position A1′ and the second core line EL2, and compares the obtained distance D2 with the second threshold value. The second threshold value is a value showing the distance between the projection pixel position A1′ having a corresponding relationship with the capture pixel position A2 and the second core line EL2 corresponding to the capture pixel position A2 when the projection light is direct light reflected only once on the object to be measured.

[0188] If the distance between the projection pixel position A1′ and the second epipolar line EL2 is greater than or equal to the second threshold value, the defective pixel detection part 502 detects the pixel at the projection pixel position A1′ as a defective pixel. In addition, the defective pixel detection part 502 detects the pixel at the capture pixel position A2 as a defective pixel. The capture pixel position A2 is a position that the analysis part 305 estimates to have a corresponding relationship with the projection pixel position A1′ where the defective pixel is detected. The defective pixel detection part 502 sends information about the position of the detected defective pixel to the analysis part 305 and the geometric shape recognition part 306. On the other hand, if the distance between the projection pixel position A1′ and the second epipolar line EL2 is less than or equal to the second threshold value, the defective pixel detection part 502 does not detect the pixels at the projection pixel position A1′ and the capture pixel position A2 as defective pixels. In this case, the defective pixel detection part 502 sends information about the distance D2 between the projection pixel position A1′ and the second epipolar line EL2 to the geometric shape recognition part 306.

[0189] [Position of the core point]

[0190] Preferably, the capture portion 2 and the projection portion 1 are arranged so that a plurality of first core lines of the capture portion 2 identified by the line recognition portion 501 do not cross each other in the captured image. In addition, preferably, the capture portion 2 and the projection portion 1 are arranged so that a plurality of second core lines of the projection portion 1 identified by the line recognition portion 501 do not cross each other in the captured image. Fig.17 EP1 is a diagram for explaining the epipolar points. Each first epipolar line on the captured image plane identified by the line identification section 501 passes through the epipolar point EP1. Also, each second epipolar line on the projected image plane identified by the line identification section 501 passes through the epipolar point EP2.

[0191] The defective pixel detection part 502 detects defective pixels by comparing the distance between the captured pixel position and the first core line of the captured image with a first threshold value. In addition, the defective pixel detection part 502 detects defective pixels by comparing the distance between the projected pixel position and the second core line of the projected image with a second threshold value. However, if the defective pixel is located at a position close to the core point EP1 or the core point EP2, it is considered that the defective pixel is close to the two core lines. Therefore, there is a possibility that the defective pixel detection part 502 cannot detect the defective pixel at a position near the core point EP1 or the core point EP2. Therefore, it is preferred that the capture part 2 and the projection part 1 are arranged in such a way that each of the core points EP1 and EP2 is located outside the captured image and the projected image, so that the situation where the defective pixel is close to the core line hardly occurs.

[0192] [Detection of overlapping correspondences]

[0193] When the analysis part 305 estimates that a projection pixel position of the projection image corresponds to multiple capture pixel positions on the first core line of the capture part 2 identified by the line identification part 501, the defective pixel detection part 502 detects at least one pixel corresponding to the same projection pixel position as a defective pixel. Fig.18 This is a diagram for explaining detection of defective pixels on an epipolar line.

[0194] like Fig.18 As shown, if the analysis part 305 estimates that a plurality of capture pixel positions A2 and A3 on the first epipolar line correspond to the same projection pixel position A1′ of different projection images, there is a high possibility of multiple reflections occurring due to the projection light from the same projection pixel position A1′ of different projection images being reflected at different measurement points MP1 and MP2, respectively. Therefore, the defective pixel detection part 502 determines whether the analysis part 305 estimates that a plurality of capture pixel positions A2 and A3 on the first epipolar line EP1 correspond to the same projection pixel position A1′ of different projection images. When the analysis part 305 estimates that a plurality of capture pixel positions A2 and A3 on the first epipolar line EL1 correspond to the same projection pixel position A1′ of different projection images, the defective pixel detection part 502 detects the pixels of the capture pixel positions A2 and A3 as defective pixels.

[0195] Similarly, the defective pixel detection section 502 may determine whether the analysis section 305 estimates that a plurality of projection pixel positions on the second epipolar line of the projection image correspond to the same captured pixel position of different captured images. If the analysis section 305 estimates that a plurality of projection pixel positions on the second epipolar line of the projection portion 1 identified by the line identification section 501 correspond to the same captured pixel position of different captured images, the defective pixel detection section 502 detects pixels at the captured pixel position and at the projection pixel position having a corresponding relationship with the captured pixel position as defective pixels.

[0196] As described above, the defective pixel detection section 502 detects defective pixels based on the distance between the captured pixel position corresponding to the projection pixel position and the first epipolar line of the captured image corresponding to the projection pixel position. Therefore, if the defective pixel is located on the epipolar line, there is a possibility that the defective pixel cannot be detected by the defective pixel detection section 502. Therefore, the defective pixel detection section 502 detects whether the analysis section 305 estimates that a plurality of captured pixel positions on the epipolar line correspond to the same projection pixel position of the projection image to prevent failure in detecting defective pixels.

[0197] It should be noted that there is a possibility of multiple reflections occurring when the analysis section 305 estimates that a plurality of captured pixel positions that are not on the same epipolar line correspond to the same projection pixel position of the projection image. Therefore, when the analysis section 305 has estimated that the same projection pixel position of the projection pixel corresponds to a plurality of captured pixel positions, the defective pixel detection section 502 may detect at least any one pixel at the plurality of captured pixel positions corresponding to the same projection pixel position as a defective pixel. The defective pixel detection section 502 may detect all pixels at the plurality of captured pixel positions corresponding to the same projection pixel position as defective pixels, and may detect some pixels at these captured pixel positions as defective pixels.

[0198] [Detect defective pixels using horizontal patterns]

[0199] Preferably, the defective pixel detection section 304 detects the defective pixel by using Figure 2A The horizontal pattern shown and Figure 2B For example, if the defective pixel detection section 304 does not detect a defective pixel when the defective pixel detection section 304 performs measurement using a vertical pattern extending in a first direction on a captured image of the object to be measured on which the projection image is projected, the defective pixel detection section 304 performs measurement using a stripe pattern extending in a second direction orthogonal to the first direction on the captured image of the object to be measured on which the projection image is projected. When the phase of the stripe pattern extending in the second direction or the amount of phase change between pixels is different from a standard value compared to a case where the same projection image is projected onto the object to be measured (such as a plane where multiple reflections do not occur), the defective pixel detection section 304 detects pixels with different phases or phase change amounts as defective pixels. Fig.19A and Fig.19B Each is a diagram for explaining a method of detecting a defective pixel by the defective pixel detection section 304 when a stripe pattern extending in the second direction is projected.

[0200] Fig.19A A captured image is displayed when the projection control section 301 projects a stripe pattern (horizontal pattern) extending in the second direction onto an object to be measured (such as a plane where multiple reflection does not occur). Fig.19B When another object to be measured is placed at a position where measurement is performed, the projection control section 301 projects a projection image similar to the image above. Fig.19A The same fringe pattern is captured. Fig.19A and Fig.19B The fringe pattern shown in has a sinusoidal brightness distribution. Fig.19B In the figure, the phase of the brightness distribution in the area indicated by the arrow has changed from Fig.19A The status changes.

[0201] If the projection control section 301 projects a projection image including a stripe pattern extending in the second direction, the phase tends not to change due to the object to be measured compared to the stripe pattern extending in the first direction, because the direction in which the phase of the stripe pattern changes due to the presence of the object to be measured matches the direction of the optical axis of the capturing section 2. Therefore, if the phase of a pixel in the captured image changes when the projection control section 301 projects a projection image including a stripe pattern extending in the second direction, there is a high possibility that the change is caused by multiple reflected light. Therefore, when the projection control section 301 projects a stripe pattern extending in the second direction, the defective pixel detection section 304 can improve accuracy by detecting pixels whose phase changes exceed a standard value among the pixels in the captured image as defective pixels. The standard value is a value showing the amount of phase change that may occur due to the influence of the 3D geometry of the object to be measured, external light, etc.

[0202] [Modification of the correspondence between the captured pixel position and the projected pixel position]

[0203] The defective pixel detection part 502 may instruct the analysis part 305 to modify the correspondence relationship estimated by the analysis part 305. For example, when the defective pixel detection part 502 detects that the distance between (i) the captured pixel position estimated by the analysis part 305 to have a correspondence relationship with the projected pixel position and (ii) the first epipolar line corresponding to the projected pixel is less than or equal to a first threshold, the defective pixel detection part 502 instructs the analysis part 305 to estimate that the position on the first epipolar line closest to the captured pixel position and the projected pixel position have a correspondence relationship with each other. Fig. 20 is a diagram for explaining a method for modifying the correspondence relationship by the defective pixel detecting section 502.

[0204] exist Fig. 20 It is assumed that (i) the distance D3 between the capture pixel position A4 estimated by the analysis section 305 to have a corresponding relationship with the projection pixel position A1′ and (ii) the first epipolar line EL1 corresponding to the projection pixel position A1′ is less than or equal to the first threshold value. In this case, the defective pixel detection section 502 instructs the analysis section 305 to estimate that the capture pixel position B closest to the capture pixel position A4 on the first epipolar line has a corresponding relationship with the projection pixel position A1′. In this case, the defective pixel detection section 502 causes the analysis section 305 to ignore the original estimation result that the projection pixel position A1′ and the capture pixel position A4 have a corresponding relationship with each other.

[0205] The estimation of the correspondence between the projection pixel position and the capture pixel position tends to have a measurement error due to the influence of external light, etc. On the other hand, the correspondence between the capture pixel position and the first epipolar line of the capture section 2 can be measured in advance, so the reliability of the measurement result is quite high. Therefore, by modifying the correspondence estimated by the analysis section 305 so that the capture pixel position is changed to be on the epipolar line, when the capture pixel position A4 estimated by the analysis section 305 to have a correspondence with the projection pixel position A1′ is offset from the epipolar line by a distance D3, the analysis section 305 can reduce the measurement error caused by the influence of external light, etc.

[0206] Similarly, for example, when the defective pixel detection section 502 detects that the distance between (i) the projected pixel position estimated by the analysis section 305 to have a corresponding relationship with the captured pixel position and (ii) the second epipolar line corresponding to the captured pixel is less than or equal to the second threshold, the defective pixel detection section 502 may instruct the analysis section 305 to estimate that the position based on the projected pixel position and the captured pixel position have a corresponding relationship with each other on the second epipolar line. For example, the defective pixel detection section 502 may instruct the analysis section 305 to estimate that the position closest to the projected pixel position on the second epipolar line has a corresponding relationship with the captured pixel position.

[0207] Similar to the first embodiment, the geometric shape recognition section 306 obtains the 3D geometric shape of the object to be measured based on the correspondence between the captured pixel position and the projected pixel position obtained by the analysis section 305. The geometric shape recognition section 306 obtains the position of the defective pixel detected by the defective pixel detection section 502, and recognizes the 3D geometric shape based on the correspondence after removing the correspondence of the defective pixel detected by the defective pixel detection section 304 from the plurality of correspondences obtained by the analysis section 305.

[0208] [Processing of 3D Geometry Measurement Device 200]

[0209] Fig.21 An exemplary operation of the 3D geometry measuring device 200 according to the second embodiment is shown. This processing starts when, for example, a user gives an instruction to measure an object to be measured with the 3D geometry measuring device 200 by using an operation key not shown. Since the processing of steps S101 to S105 is similar to that of Fig.12 The processing procedures of steps S101 to S105 are the same, so the processing procedures of steps S101 to S105 are not described.

[0210] The analysis section 305 obtains a correspondence between a captured pixel position showing a position of a pixel in a captured image and a projected pixel position showing a position of a pixel in a projected image. In addition, the analysis section 305 obtains a correspondence between the captured pixel position and the projected pixel position by using the analysis result of the gradient information of the gradient stripe pattern obtained in step S105. The analysis section 305 estimates the correspondence between the captured pixel position and the projected pixel position by selecting a correspondence closest to the correspondence estimated by using the code value from the correspondence obtained by the analysis result of the gradient information (S201). The line identification section 501 identifies the first epipolar line of the captured portion 2 corresponding to the projected pixel position (S202).

[0211] The defective pixel detection part 502 obtains the distance between the captured pixel position A2 having a corresponding relationship with the projection pixel position estimated by the analysis part 305 and the first core line of the captured image corresponding to the projection pixel position, and compares the obtained distance with the first threshold. If the distance between the captured pixel position and the first core line is greater than or equal to the first threshold, the defective pixel detection part 502 detects the pixel at the captured pixel position A2 as a defective pixel. The defective pixel detection part 502 compares the distance between the captured pixel position and the first core line of each corresponding relationship estimated by the analysis part 305 with the first threshold, and detects the pixel whose distance from the captured pixel position is greater than or equal to the first threshold as a defective pixel (S203). The geometric shape recognition part 306 recognizes the 3D geometric shape of the object to be measured based on the corresponding relationship of the pixels other than the defective pixels detected by the defective pixel detection part 304, and ends the processing (S204).

[0212] According to the present embodiment, the defective pixel detection section 304 detects defective pixels based on the positional relationship between the captured pixel position and the first epipolar line or the positional relationship between the projected pixel position and the second epipolar line. Since the position of the epipolar line is specified by the arrangement of the capture section 2 and the projection section 1, the position of the epipolar line can be measured in advance. Therefore, the 3D geometry measuring device can simplify the process of detecting defective pixels by detecting defective pixels using epipolar lines.

[0213] It should be noted that in this embodiment, an example is described in which the projection control section 301 projects a projection image including a stripe pattern extending in a first direction and a projection image including a stripe pattern extending in a second direction, and the first direction and the second direction are orthogonal to each other. However, the present invention is not limited to this. For example, the first direction and the second direction may be approximately orthogonal to each other. The first direction may be any direction as long as it is different from the direction of the projection control section 301. Figure 2B The second direction may be any direction different from the first direction.

[0214] [Third embodiment]

[0215] In the first and second embodiments, the projection control section 301 projects the projection patterns for the spatial encoding method and the phase shift method as projection patterns in the horizontal direction and the vertical direction. That is, the projection control section 301 projects the projection image having the stripe pattern extending in the first direction and the projection image having the stripe pattern extending in the second direction as stripe patterns showing sinusoidal brightness distribution. In addition, the projection control section 301 projects the projection image having the stripe pattern extending in the first direction and the projection image having the stripe pattern extending in the second direction as stripe patterns having two values.

[0216] However, the present invention is not limited thereto. For example, the projection control section 301 may also project only a projection pattern of the phase shift method in the horizontal direction of the projection pattern. In this case, the projection control section 301 projects a projection image having a stripe pattern presenting a sinusoidal brightness distribution as a stripe pattern extending in the second direction, and does not project a projection image having a stripe pattern containing two values ​​onto the object to be measured.

[0217] When the projection control section 301 projects the projection pattern in the horizontal direction, the analysis section 305 obtains the phase value I RP,2 (I, j). In this case, the absolute phase value of the projection pattern along the horizontal direction can be expressed as

[0218] I AP,2 =2πm+I RP,2 (i,j)(Equation A),

[0219] And multiple candidates may be considered. There are multiple candidates for the projected pixel position that have a corresponding relationship with the captured pixel position, as shown in the equation shown below:

[0220]

[0221] where i p and j p (m) respectively show the i-th edge starting from the left edge in the second direction p pixels and the jth pixel starting from the top edge in the first direction p (m) pixels.

[0222] The analysis section 305 estimates that the projection pixel position of the second core line of the projection section 1 corresponding to the captured pixel position, among the projection pixel positions of pixels presenting brightness of the same phase in the stripe pattern as the captured image, has a corresponding relationship with the captured pixel position. Fig. 22 3 is a diagram for explaining a method of estimating the correspondence relationship between the captured pixel position and the projected pixel position by the analyzing section 305 of the third embodiment. Fig. 22The multiple black circles in show multiple projected pixels that present the same phase as the pixel at the capture pixel position.

[0223] The analysis section 305 identifies the projection pixel that exhibits the same phase as the pixel at the captured pixel position by analyzing the gradient information of the gradient stripe pattern in the captured image. Fig. 22 As shown by the black circle in , there are a plurality of projected pixels that present brightness having the same phase as the pixel at the captured pixel position. Therefore, the analysis section 305 obtains the projected pixel positions of these pixels as candidates for the projected pixel positions having a corresponding relationship with the captured pixel position. Among these candidates, the analysis section 305 estimates that the candidate closest to the second epipolar line corresponding to the captured pixel position has a corresponding relationship with the captured pixel position.

[0224] The analyzing section 305 selects the pixel corresponding to the pixel closest to the second epipolar line of the corresponding projection part. c The projected pixel position (i p ,j p (m c )), as shown in the following equation:

[0225]

[0226] Since the analysis section 305 estimates the projection pixel position having a corresponding relationship with the captured pixel position by using the epipolar line having such a configuration, the analysis section 305 can reduce the number of projection images projected onto the object to be measured by the projection control section 301. As a result, the analysis section 305 can shorten the measurement time.

[0227] The fringe pattern extending in the second direction does not cause the phase to change significantly depending on the presence of the object to be measured. Therefore, for example, when a projection image including a fringe pattern extending in the second direction is projected onto a flat panel, the position of the fringe extending in the second direction of the captured image is measured in advance, and the approximate position of the fringe pattern extending in the direction of the captured image can be identified. The value (or approximate value) of the integer m in the above equations A and B can be obtained before measuring the object to be measured by using information showing the position of the fringe extending in the second direction. By doing so, the processing when measuring the actual object to be measured can be simplified.

[0228] It should be noted that an example is described in which the pixel estimated by the analysis section 305 to be closest to the second epipolar line of the projection section 1 corresponding to the captured pixel position is a pixel having a corresponding relationship with the pixel at the captured pixel position among the pixels at the projection pixel position presenting a brightness of the same phase as the captured pixel position in the stripe pattern. However, the present invention is not limited to this. Projection Projection Description For example, the analysis section 305 may obtain a plurality of first epipolar lines on each captured image for a plurality of projection pixel positions having the same phase value as the captured pixel position in the stripe pattern, and may estimate that the projection pixel position corresponding to the first epipolar line closest to the captured pixel position has a corresponding relationship with the captured pixel position.

[0229] In addition, the projection control section 301 may project projection images whose periods of the stripe pattern extending in the second direction are different from each other, and the analysis section 305 may estimate the correspondence between the captured pixel position and the projected pixel position for each projection image. In this case, if the analysis section 305 estimates the correspondence between the captured pixel position and the projected pixel position for each projection image, the defective pixel detection section 502 detects the pixel at the captured pixel position and the projected pixel position, for which the correspondence estimated by the analysis section 305 has changed, as a defective pixel.

[0230] In addition, the 3D geometry measuring device may have a configuration including a plurality of capture parts or a plurality of projection parts. Even when the 3D geometry measuring device includes a plurality of capture parts or a plurality of projection parts, if a pair of capture parts and projection parts is specified, the epipolar line of the projection pixel corresponding to a specific capture pixel position is uniquely determined. Therefore, the defective pixel detection part 502 can detect defective pixels based on the positional relationship between the capture pixel position and the first epipolar line or the positional relationship between the projection pixel position and the second epipolar line, similar to the configuration of the 3D geometry measuring device having one capture part and one projection part.

[0231] In addition, if one projection part and a plurality of capture parts are used, a 3D geometric shape can be obtained by obtaining pixels corresponding to each other between a pair of capture parts. In this case, the 3D geometric shape of the object to be measured can be obtained by removing defective pixels identified by the defective pixel detection part according to the first to third embodiments.

[0232] In addition, in the second embodiment, an example of a case where the defective pixel detection section 502 detects defective pixels based on the distance between the projection pixel position and the second epipolar line on the projection image plane of the projection section is described. In addition, an example of a case where the defective pixel detection section 502 detects defective pixels based on the distance between the capture pixel position and the first epipolar line on the capture image plane of the capture section is described. However, the present invention is not limited to the configuration for detecting defective pixels. For example, the 3D geometry measuring device may include a determination section that performs various determinations based on statistics of (i) the distance between the capture pixel position and the first epipolar line or (ii) the distance between the projection pixel position and the second epipolar line.

[0233] When the statistic of the distance between the capture pixel position and the first epipolar line exceeds the allowable value, the determination part determines that the alignment state of the projection part 1 and the capture part 2 is inappropriate. For example, for a plurality of randomly selected projection pixel positions, the determination part obtains (i) the capture pixel position estimated by the analysis part 305 to have a corresponding relationship with the projection pixel position and (ii) the respective distances between the first epipolar line corresponding to the projection pixel position, and calculates the average respective values ​​of the obtained distances. When the calculated average respective values ​​exceed the allowable value, the determination part can self-diagnose that the alignment state of the orientation of the optical axis, etc. of the projection part 1 and the capture part 2 is inappropriate. In this case, the determination part can display a message stored in the storage part 4 on the display part (not shown), indicating that the calibration of the position of the epipolar line, etc. corresponding to each of the capture pixel position and the projection pixel position needs to be made. Taking the desired measurement accuracy into consideration, the appropriate allowable value is specified by those skilled in the art.

[0234] Similarly, when the statistic of the distance between the projection pixel position and the second epipolar line exceeds the allowable value, the determination section determines that the alignment state of the projection section 1 and the capture section 2 is inappropriate. For a plurality of randomly selected capture pixel positions, the determination section obtains (i) the projection pixel position estimated by the analysis section 305 to have a corresponding relationship with the capture pixel position and (ii) the respective distances between the second epipolar line corresponding to the capture pixel position, and calculates the average respective value of the obtained distances. When the calculated average respective value exceeds the allowable value, the determination section can self-diagnose that the alignment state of the orientation of the optical axis, etc. of the projection section 1 and the capture section 2 is inappropriate. In this case, the determination section can display a message stored in the memory section 4 on the display section (not shown), indicating that calibration of the position, etc. of the epipolar line corresponding to each of the capture pixel position and the projection pixel position needs to be made.

[0235] The present invention is described based on exemplary embodiments. The technical scope of the present invention is not limited to the scope described in the above-mentioned embodiments, and various changes and modifications can be made within the scope of the present invention. For example, the specific embodiments of the distribution and integration of the device are not limited to the above-mentioned embodiments, and all or part of them can be configured with any units that are functionally or physically dispersed or integrated. In addition, new exemplary embodiments generated by any combination of them are included in the exemplary embodiments of the present invention. In addition, the effects of the new exemplary embodiments brought about by these combinations also have the effects of the original exemplary embodiments.

[0236] [Description of Reference Signs]

[0237] 1 Projection part

[0238] 2 Capture part

[0239] 3 Control section

[0240] 4 Memory section

[0241] 21 Lenses

[0242] 22 Imaging element

[0243] 100, 200 3D geometry measuring device

[0244] 301 Projection Control Section

[0245] 302 Get Part

[0246] 303 Feature Identification Part

[0247] 304, 502 Defective pixel detection section

[0248] 305 Analysis Section

[0249] 306 Geometric Shape Recognition Section

[0250] 501 Line Identification Section

Claims

1. A 3D geometric shape measuring device, comprising: a projection section that sequentially projects two or more projection images including a stripe pattern extending in a first direction onto an object to be measured, and projects a projection image including a stripe pattern extending in a second direction onto the object to be measured; a capturing section for generating a captured image including the stripe pattern extending along a first direction projected by the projecting section onto the object to be measured, and a captured image including the stripe pattern extending along a second direction; a feature quantity recognition section that recognizes a first feature quantity from the captured image of the stripe pattern extending in the first direction, and recognizes a second feature quantity from the captured image of the stripe pattern extending in the second direction; a defective pixel detection section that identifies a similarity between the first feature quantity and the second feature quantity, and detects a defective pixel in the captured image based on a comparison result of the identified similarity with a threshold value; and A geometric shape recognition section recognizes a correspondence relationship between pixels of the captured image other than the defective pixel and pixels of the projected image to recognize a 3D geometric shape of the object to be measured based on the recognized correspondence relationship.

2. A 3D geometric shape measuring device, comprising: a projection part, projecting a projection image including stripe patterns extending in multiple directions onto an object to be measured, sequentially projecting two or more projection images including stripe patterns extending in a first direction onto the object to be measured, and projecting a projection image including a stripe pattern extending in a second direction onto the object to be measured; a capturing part, respectively generating captured images including the fringe pattern projected by the projecting part onto the object to be measured; a feature amount recognition section that recognizes a feature amount indicating a feature of the stripe pattern included in the captured image, recognizes a first feature amount recognized from the captured image of the stripe pattern extending in the first direction, and recognizes a second feature amount from the captured image of the stripe pattern extending in the second direction; a defective pixel detecting section that detects a defective pixel in the captured image based on the first feature amount and the second feature amount identified by the feature amount identifying section; and a geometric shape recognition section that recognizes a correspondence relationship between pixels of the captured image other than the defective pixel and pixels of the projected image to recognize a 3D geometric shape of the object to be measured based on the recognized correspondence relationship, The defective pixel detection section detects a pixel in which a difference between the first feature amount and the second feature amount in the captured image is equal to or larger than a threshold value as the defective pixel.

3. The 3D geometry measuring device according to claim 1 or 2, wherein: When the range of the defective pixels detected by the defective pixel detection section is outside a predetermined range, the projection section projects a projection image including a fringe pattern extending in a direction different from a projected direction onto the object to be measured.

4. A three-dimensional 3D geometric shape measurement method, comprising the following steps performed by a computer: sequentially projecting two or more projection images including a stripe pattern extending along a first direction onto an object to be measured, and projecting a projection image including a stripe pattern extending along a second direction onto the object to be measured; generating a captured image including the stripe pattern extending along a first direction projected onto the object to be measured, and a captured image including the stripe pattern extending along a second direction, respectively; identifying a first feature quantity from the captured image of the stripe pattern extending along the first direction, and identifying a second feature quantity from the captured image of the stripe pattern extending along the second direction; identifying a similarity between the first feature quantity and the second feature quantity, and detecting a defective pixel in the captured image based on a comparison result of the identified similarity with a threshold; and A correspondence relationship between pixels of the captured image excluding the defective pixel and pixels of the projected image is identified to identify a 3D geometric shape of the object to be measured based on the identified correspondence relationship.

5. A three-dimensional 3D geometric shape measurement method, comprising the following steps performed by a computer: projecting a projection image including stripe patterns extending in multiple directions onto an object to be measured, sequentially projecting two or more projection images including stripe patterns extending in a first direction onto the object to be measured, and projecting a projection image including a stripe pattern extending in a second direction onto the object to be measured; respectively generating captured images including the fringe pattern projected onto the object to be measured; identifying a feature quantity indicative of a feature of the stripe pattern included in the captured image, identifying a first feature quantity identified from the captured image of the stripe pattern extending along the first direction, and identifying a second feature quantity from the captured image of the stripe pattern extending along the second direction; detecting defective pixels in the captured image based on the identified first feature quantity and the second feature quantity; and identifying a correspondence between pixels of the captured image other than the defective pixel and pixels of the projection image to identify a 3D geometric shape of the object to be measured based on the identified correspondence, In the step of detecting the defective pixel, a pixel in which a difference between the first feature amount and the second feature amount in the captured image is equal to or larger than a threshold value is detected as the defective pixel.

Images