Three-dimensional shape measuring device and three-dimensional shape measuring method

Abstract

The invention suppresses the reduction of the measurement accuracy caused by multiple reflected light, etc. The invention comprises: a relationship determination unit (45) for determining a combination of a first image pixel and a second image pixel corresponding to the same projection coordinates as the projection coordinates corresponding to the first image pixel and located on the first epipolar line of the second image corresponding to the first image pixel; a determination unit (46) for determining whether at least one of the first image pixel or the second image pixel is a bad pixel based on the distance between the projection pixel of the projection image having the projection coordinates corresponding to the first image pixel and the projection pixel of the projection image having the projection coordinates corresponding to the second image pixel; and a shape measurement unit (47) for measuring the shape of the measurement object using the first image pixel or the second image pixel corresponding to the combination determined as not being the bad pixel.

Description

Technical Field

[0001] The present invention relates to a three-dimensional shape measuring device and a three-dimensional shape measuring method for measuring the shape of a measurement object. Background Art

[0002] A method has been proposed in which a projection pattern is projected and an image obtained by capturing an object on which the projection pattern is projected is analyzed, thereby measuring the shape of the object without contacting the object (for example, see Japanese Patent Application Laid-Open No. 2017-146298). Summary of the invention

[0003] Problems that the invention will solve

[0004] If the projection pattern is projected onto the object to be measured, there is a situation where reflection of light between multiple surfaces of the object to be measured, i.e., multiple reflection, is caused. If multiple reflections are caused, an area with a brightness different from that without multiple reflections will be generated in the captured image. In the case where there is only one camera, it is necessary to determine the influence of the multiple reflected light by analyzing the captured image obtained from one camera. For example, in the method described in Japanese Patent Gazette No. 2017-146298, it is possible to determine pixels affected by multiple reflected light based on the brightness value of the captured image. However, in the case where the pixels affected by the multiple reflected light show a brightness close to that of the pixels directly irradiated with the reflected light, there is a possibility that the pixels affected by the multiple reflected light cannot be detected.

[0005] Therefore, the present invention has been made in view of the above-mentioned circumstances, and an object of the present invention is to provide a three-dimensional shape measuring device and a three-dimensional shape measuring method capable of suppressing a decrease in measurement accuracy due to multiple reflected light and the like.

[0006] Means for solving problems

[0007] A three-dimensional shape measuring device according to a first embodiment of the present invention comprises: a projection unit that projects a projection image including a pattern for determining projection coordinates onto a measuring object; a first imaging unit that generates a first captured image by imaging the projection image projected onto the measuring object; a second imaging unit that generates a second captured image by imaging the projection image projected onto the measuring object; a first coordinate determining unit that determines the projection coordinates corresponding to a first captured pixel of the first captured image based on the pattern included in the first captured image; a second coordinate determining unit that determines the projection coordinates corresponding to a second captured pixel of the second captured image based on the pattern included in the second captured image; a line determining unit that determines a first core line of the second captured image corresponding to the first captured pixel based on the configuration of the first imaging unit and the second imaging unit; and a relationship determining unit that determines the relationship between the first and second captured pixels. a combination of a first pixel and a second pixel corresponding to the same projection coordinates as the projection coordinates corresponding to the first pixel and located on the first epipolar line of the second image corresponding to the first pixel; a determination unit that determines, for each combination of the first pixel and the second pixel, whether at least one of the first pixel or the second pixel is a defective pixel based on a distance between a projection pixel of the projected image having the projection coordinates corresponding to the first pixel and a projection pixel of the projected image having the projection coordinates corresponding to the second pixel; and a shape measurement unit that measures the shape of the measurement object using at least one of the first pixel and the second pixel corresponding to the combination determined as not being the defective pixel.

[0008] Alternatively, the projection unit projects a plurality of projection images of the pattern including stripes extending in the same direction onto the measurement object, the line determination unit determines a second epipolar line of the projection image corresponding to the second imaging pixel based on a configuration of the second imaging unit and the projection unit, and determines a third epipolar line of the projection image corresponding to the first imaging pixel based on a configuration of the first imaging unit and the projection unit, and the determination unit determines, for each combination of the first imaging pixel and the second imaging pixel, whether at least one of the first imaging pixel or the second imaging pixel is a defective pixel based on a distance between the projection pixel having the projection coordinates corresponding to the first imaging pixel and located on the third epipolar line of the projection image corresponding to the first imaging pixel and the projection pixel having the projection coordinates corresponding to the second imaging pixel and located on the second epipolar line of the projection image corresponding to the second imaging pixel.

[0009] Alternatively, the projection unit projects a plurality of projection images of the pattern including stripes extending in the same direction onto the measurement object, the line determination unit determines a second epipolar line of the projection image corresponding to the second imaging pixel based on a configuration of the second imaging unit and the projection unit, and determines a third epipolar line of the projection image corresponding to the first imaging pixel based on a configuration of the first imaging unit and the projection unit, and the determination unit determines, for each combination of the first imaging pixel and the second imaging pixel, whether at least one of the first imaging pixel or the second imaging pixel is a defective pixel based on a distance between the projection pixel having the projection coordinates corresponding to the first imaging pixel and located on the third epipolar line of the projection image corresponding to the first imaging pixel and the projection pixel located at an intersection of the second epipolar line and the third epipolar line.

[0010] The projection unit may project the pattern including fringes onto the measurement object, wherein the fringes included in the pattern extend in different directions from each other.

[0011] Alternatively, the first coordinate determination unit determines the projection coordinates corresponding to the first imaging pixel for each of the plurality of projection images in which the stripes extend in different directions, and the second coordinate determination unit determines the projection coordinates corresponding to the second imaging pixel for each of the plurality of projection images in which the stripes extend in different directions, and the determination unit obtains a comparison value between the projection coordinates corresponding to the first imaging pixel and the projection coordinates corresponding to the second imaging pixel for each of the plurality of projection images in which the stripes extend in different directions in the combination of the first imaging pixel and the second imaging pixel determined by the relationship determination unit, and determines whether at least one of the first imaging pixel or the second imaging pixel is a defective pixel by using an evaluation value based on the obtained comparison value.

[0012] Alternatively, the first coordinate determination unit determines the projection coordinates corresponding to the first image pixel for each of the plurality of projection images in which the stripes extend in different directions, and the three-dimensional shape measuring device further comprises: an acquisition unit that acquires the combination of the first image pixel and the second image pixel determined by the relationship determination unit, and acquires a plurality of the second image pixels within a predetermined range from the second image pixel in the acquired combination as corresponding candidate pixels; a candidate coordinate determination unit that determines the projection coordinates corresponding to the plurality of corresponding candidate pixels for each of the projection images in which the stripes extend in different directions; and an evaluation unit that obtains a comparison value between the projection coordinates corresponding to the first image pixel and the projection coordinates corresponding to the corresponding candidate pixels for each of the projection images in which the stripes extend in different directions, and obtains an evaluation value based on the plurality of comparison values ​​obtained for each of the projection images, for each of the corresponding candidate pixels, and the relationship determination unit determines, based on the evaluation value obtained by the evaluation unit, any one of the plurality of corresponding candidate pixels and the first image pixel in the combination obtained by the acquisition unit as a new combination.

[0013] It may also be that the relationship determination unit determines a plurality of the combinations between the plurality of the first capturing pixels included in the first captured image and the plurality of the second capturing pixels included in the second captured image, and the determination unit obtains an evaluation value indicating the distance between a projection pixel of the projection image having the projection coordinates corresponding to the first capturing pixel and a projection pixel of the projection image having the projection coordinates corresponding to the second capturing pixel in the combinations of the first capturing pixels and the second capturing images determined by the relationship determination unit, obtains the evaluation value for each of the plurality of combinations, and determines that a malfunction has occurred in the configuration of any one of the first capturing unit, the second capturing unit, and the projection unit when a statistic of the plurality of the obtained evaluation values ​​exceeds a reference value.

[0014] A three-dimensional shape measuring device according to a second aspect of the present invention comprises: a projection unit that projects a projection image including a pattern for determining projection coordinates onto a measuring object; a first imaging unit that generates a first captured image by imaging the projection image projected onto the measuring object; a second imaging unit that generates a second captured image by imaging the projection image projected onto the measuring object; a first coordinate determining unit that determines the projection coordinates corresponding to a first captured pixel of the first captured image based on the pattern included in the first captured image; a second coordinate determining unit that determines the projection coordinates corresponding to a second captured pixel of the second captured image based on the pattern included in the second captured image; a line determining unit that determines an epipolar line of the projection image corresponding to the first captured pixel based on the configuration of the first imaging unit and the projection unit, and determines an epipolar line of the second captured image corresponding to a projection pixel of the projection image based on the configuration of the projection unit and the second imaging unit; and a relationship determining unit that determines a relationship between the first captured pixel and a combination of projection pixels of the projection image having the projection coordinates corresponding to the first pixel and located on the epipolar line of the projection image corresponding to the first pixel; a determination unit that determines, for each of the combinations of the first pixel and the projection pixel, whether at least one of the first pixel, the main second pixel, or the sub-second pixel is a defective pixel based on a distance between a second pixel corresponding to the projection coordinates identical to the projection coordinates corresponding to the first pixel and a second pixel corresponding to the projection coordinates of the projection pixel and located on the epipolar line of the second image corresponding to the projection coordinates; and a shape measurement unit that measures the shape of the measurement object using at least one of the first pixel, the main second pixel, or the sub-second pixel corresponding to the combination determined as not being a defective pixel.

[0015] The projection unit may project the projection image of the pattern including binary stripes and the projection image of the pattern including stripes having a sinusoidal brightness distribution onto the measurement object.

[0016] The projection unit may sequentially project the plurality of projection images including the pattern of fringes having a sinusoidal luminance distribution with fringes having different periods from each other.

[0017] Alternatively, the projection unit may re-project onto the measurement object a projection image in which all of the projection pixels having the projection coordinates corresponding to any one of the first imaging pixel and the second imaging pixel determined as a defective pixel by the determination unit, among a plurality of pixels included in the projection image, are removed.

[0018] Alternatively, the projection unit may re-project onto the measured object a projection image that includes only the projection pixels having the projection coordinates corresponding to the first shooting pixel or the second shooting pixel determined as a defective pixel by the determination unit, among the plurality of projection pixels included in the projection image.

[0019] The third aspect of the three-dimensional shape measurement method of the present invention comprises the following steps: projecting a projection image including a pattern for determining projection coordinates onto a measurement object; generating a first captured image by capturing the projection image projected onto the measurement object using a first capturing unit; generating a second captured image by capturing the projection image projected onto the measurement object using a second capturing unit; determining the projection coordinates corresponding to a first captured pixel of the first captured image based on the pattern included in the first captured image; determining the projection coordinates corresponding to a second captured pixel of the second captured image based on the pattern included in the second captured image; determining the epipolar line of the second captured image corresponding to the first captured pixel based on the configuration of the first capturing unit and the second capturing unit; determining the epipolar line of the first captured pixel and the epipolar line corresponding to the epipolar line of the second captured image; and determining the epipolar line of the first captured pixel and the epipolar line corresponding to the epipolar line of the second captured image. a combination of the second photographing pixels having the same projection coordinates as the projection coordinates corresponding to the first photographing pixel and being located on the epipolar line of the second photographing image corresponding to the first photographing pixel; for each of the combinations of the first photographing pixel and the second photographing pixel, determining whether at least one of the first photographing pixel or the second photographing pixel is a defective pixel based on a distance between a projection pixel of the projection image having the projection coordinates corresponding to the first photographing pixel and a projection pixel of the projection image having the projection coordinates corresponding to the second photographing pixel; and measuring the shape of the measurement object using at least one of the first photographing pixel or the second photographing pixel corresponding to the combination determined as not being the defective pixel.

[0020] Effects of the Invention

[0021] According to the present invention, it is possible to suppress a decrease in measurement accuracy due to multiple reflected light or the like. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a diagram for explaining the outline of the three-dimensional shape measuring device according to the first embodiment.

[0023] Figure 2 The projection image projected by the projection unit onto the measurement object is shown.

[0024] Figure 3 It is a diagram showing the structure of a three-dimensional shape measuring device.

[0025] Figure 4 It is a diagram showing examples of the types of projection images projected by the projection control unit.

[0026] Figure 5 An example of a grayscale stripe pattern having a sinusoidal luminance distribution is shown.

[0027] Figure 6 Show Figure 4 (c)~ Figure 4 An example of a Gray code corresponding to the binary stripe pattern shown in (f) in FIG.

[0028] Figure 7 It is a diagram showing the relationship between absolute projection coordinates and relative projection coordinates.

[0029] Figure 8 An example is shown in which an image of a measurement target object is captured by the first imaging unit and the second imaging unit.

[0030] Fig. 9 An example of the second epipolar line of the projection image corresponding to the second captured image is shown.

[0031] Fig.10 This is a diagram for explaining multiple reflections.

[0032] Fig.11 This is a diagram showing a method in which the determination unit determines a defective pixel when the projection control unit projects only a plurality of projection images including patterns of stripes extending in the same direction.

[0033] Fig.12 This is a diagram showing a method by which a determination unit determines whether a pixel is a defective pixel.

[0034] Fig.13 This is a diagram showing the relationship between the direction in which fringes of a projected image extend and the multi-reflected light.

[0035] Fig.14 This is a flowchart showing a process of measuring the shape of a measurement object by a three-dimensional shape measuring device.

[0036] Fig.15 It is a diagram showing the configuration of a control unit included in the three-dimensional shape measuring device according to the second embodiment.

[0037] Fig.16This is a diagram showing an evaluation method for corresponding candidate pixels by the evaluation unit.

[0038] Description of Reference Numerals

[0039] 1 First Filming Department

[0040] 2 Second Filming Department

[0041] 3 Projection Department

[0042] 4 Control Unit

[0043] 5 Storage

[0044] 11. Lens

[0045] 12 Camera elements

[0046] 21 Lens

[0047] 22 Photographic elements

[0048] 41 Projection Control Unit

[0049] 42 Shooting control unit

[0050] 43 Coordinate determination unit

[0051] 44 Line determination department

[0052] 45 Relationship Determination Department

[0053] 46 Judgment Department

[0054] 47 Shape measurement unit

[0055] 50 Control Department

[0056] 100 Three-dimensional shape measuring device

[0057] 431 First coordinate determination unit

[0058] 432 Second coordinate determination unit

[0059] 501 Acquisition Department

[0060] 502 Candidate coordinate determination unit

[0061] 503 Evaluation Department DETAILED DESCRIPTION

[0062] <First embodiment>

[0063] [Overview of Three-Dimensional Shape Measuring Device 100]

[0064] Figure 1 (a)~ Figure 1(c) is a diagram for explaining the outline of the three-dimensional shape measuring device 100 according to the first embodiment. Figure 1 (a) in FIG. 1 shows the structure of the three-dimensional shape measuring device 100. The three-dimensional shape measuring device 100 projects a projection image onto a measurement surface of a measurement object. The three-dimensional shape measuring device 100 uses two imaging units, namely, a first imaging unit 1 and a second imaging unit 2, to capture the measurement object in a state where the projection image projected by the projection unit 3 is projected. The three-dimensional shape measuring device 100 optically measures the three-dimensional shape of the measurement object by analyzing the captured image generated by the three-dimensional shape measuring device 100.

[0065] The projection unit 3 is a projection device having a light source such as a light emitting diode or a laser. The projection unit 3 projects a projection image including a pattern for determining projection coordinates onto a measurement surface of the measurement object. The projection coordinates represent the positions of projection pixels constituting the projection image projected by the projection unit 3. The projection coordinates may be one-dimensional coordinates representing the position of either the longitudinal direction or the lateral direction of the projection image, or may be two-dimensional coordinates representing the position of both the longitudinal direction and the lateral direction of the projection image. The pattern is, for example, a striped pattern.

[0066] The first imaging unit 1 includes a lens 11 and an imaging element 12. The first imaging unit 1 captures a projection image projected onto the measuring object while the projection unit 3 projects the projection image onto the measuring object, thereby generating a first captured image. The first imaging unit 1 is configured such that the optical axis of the first imaging unit 1 and the optical axis of the projection unit 3 form a predetermined first angle. The first imaging unit 1 is configured such that the optical axis of the first imaging unit 1 and the optical axis of the second imaging unit 2 form a predetermined second angle.

[0067] The second imaging unit 2 has a lens 21 and an imaging element 22. The second imaging unit 2 captures the projection image projected onto the measuring object while the projection unit 3 projects the projection image onto the measuring object, thereby generating a second captured image. The second imaging unit 2 is configured so that the optical axis of the second imaging unit 2 and the optical axis of the projection unit 3 form a predetermined third angle. Although the optical axis of the second imaging unit 2 may also be in the same plane as the optical axis of the first imaging unit 1 and the optical axis of the projection unit 3, it is not limited thereto. The control unit 4 measures the shape of the measuring object based on the plurality of captured images generated by the first imaging unit 1 and the second imaging unit 2. The control unit 4 is implemented, for example, by a computer.

[0068] Figure 1 (b) and (c) in FIG. 1 show examples of captured images generated by the first imaging unit 1 during the period in which the projection unit 3 projects the projection image onto the measurement object. Figure 1As shown in (b) and (c) of FIG. 1 , the projection unit 3 projects a projection image including a binary stripe pattern onto the measurement object. The binary stripe pattern is composed of a light projection area where light is projected and a non-projection area where light is not projected. Figure 1 (b) in the figure shows the captured image generated by the first imaging unit 1 and the second imaging unit 2 when the projection unit 3 projects the projection image including the binary stripe pattern toward the measurement surface without unevenness. The white area represents the light projection area, and the black area represents the non-projection area. When the measurement surface has no unevenness, the binary stripe pattern of the captured image generated by the first imaging unit 1 and the second imaging unit 2 has a shape that is roughly the same as the binary stripe pattern of the projection image.

[0069] Figure 1 (c) in FIG. 1 is a captured image generated by the first imaging unit 1 or the second imaging unit 2 when the projection unit 3 projects a binary stripe pattern onto the measurement surface including the convex portion. Figure 1 As shown in the captured image (c) in FIG. 1 , a portion of the image of the binary stripe pattern is deformed. In the captured image, the image of the binary stripe pattern is deformed by an amount corresponding to the height of the convex portion. Therefore, the three-dimensional shape measuring device 100 can measure the shape of the object to be measured by determining the height of each position of the convex portion based on the amount of deformation of the image of the binary stripe pattern in the captured image.

[0070] The three-dimensional shape measuring device 100 measures the three-dimensional shape of the measuring object by analyzing the pattern of stripes projected onto the measuring object. However, when the surface of the measuring object is glossy, there is a problem that the measurement accuracy is reduced due to multiple reflections caused by multiple reflections of the projection light from the projection unit 3. Therefore, the three-dimensional shape measuring device 100 uses the first imaging unit 1 and the second imaging unit 2 to photograph the measuring object while projecting the pattern onto the measuring object, and analyzes the two photographed images generated by the first imaging unit 1 and the second imaging unit 2 to determine whether the photographed pixels included in the photographed images are bad pixels affected by the multiple reflection light, etc. The three-dimensional shape measuring device 100 measures the three-dimensional shape of the measuring object by analyzing the photographed pixels other than the bad pixels.

[0071] Although described in detail later, the three-dimensional shape measuring device 100 selects a combination of a first imaging pixel included in a first imaging image generated by the first imaging unit 1 and a second imaging pixel included in a second imaging image generated by the second imaging unit 2. The three-dimensional shape measuring device 100 can determine whether at least one of the first imaging pixel or the second imaging pixel is a defective pixel affected by multiple reflected light or the like by determining whether the first imaging pixel and the second imaging pixel correspond to substantially the same projection pixel of the projection image in the selected combination. Therefore, the three-dimensional shape measuring device 100 can suppress a decrease in the measurement accuracy of the shape of the measurement object caused by multiple reflected light or the like.

[0072] [Direction of extension of stripes]

[0073] Figure 2 (a) and (b) in FIG. 1 show projection images projected by the projection unit 3 onto the measurement object. Figure 2 (a) shows an example of a binary stripe pattern extending along a first direction, Figure 2 (b) in FIG. 1 shows an example of a binary stripe pattern extending along the second direction. Figure 2 As shown in (a) of FIG. 1 , a binary stripe pattern extending in a first direction (hereinafter sometimes referred to as a longitudinal direction) is projected. The first direction is a direction perpendicular to a plane including the optical axis of the projection unit 3 and the optical axis of the first imaging unit 1 .

[0074] Projection unit 3 Figure 2 As shown in (b) in the figure, a binary stripe pattern extending along a second direction (hereinafter sometimes referred to as a transverse direction) is projected. The second direction is a direction parallel to a plane including the optical axis of the projection unit 3 and the optical axis of the first imaging unit 1. In addition, the longitudinal direction may be a direction along any direction of the projected image, and the transverse direction may be a direction orthogonal to the longitudinal direction. The first imaging unit 1, the second imaging unit 2, and the projection unit 3 are configured so that their optical axes pass through any point on the same straight line on a workbench for configuring the measured object, but are not limited to this.

[0075] When a stripe pattern is projected onto an object to be measured, the three-dimensional shape of the object to be measured is as follows: Figure 1As shown in (c) in FIG. 1 , the pattern of the stripes deviates in the width direction. In addition, the width of the pattern of the stripes varies according to the three-dimensional shape of the object to be measured. In the first captured image in which the pattern of the stripes extends along the first direction, the direction of the deviation of the optical axis direction between the projection unit 3 and the first shooting unit 1 and the direction caused by the deviation in the width direction of the pattern of the stripes are made consistent. That is, the direction of the image of the line segment generated by projecting the line segment connecting the projection unit 3 and the first shooting unit 1 onto the plane on which the object to be measured is placed and the direction caused by the deviation in the width direction of the pattern of the stripes are consistent. Therefore, in the captured image in which the pattern of the stripes extends along the first direction, the sensitivity of detecting the deviation in the width direction of the pattern of the stripes is high. Therefore, the resolution when measuring the three-dimensional shape of the object to be measured becomes higher.

[0076] On the other hand, in the captured image in which the stripe pattern extends in the second direction, the direction in which the direction of the optical axis between the projection unit 3 and the first imaging unit 1 deviates is orthogonal to the direction caused by the deviation in the width direction of the stripe pattern. That is, the direction of the image of the line segment generated by projecting the line segment connecting the projection unit 3 and the first imaging unit 1 onto the plane on which the measurement object is placed is orthogonal to the direction caused by the deviation in the width direction of the stripe pattern. Therefore, in the captured image in which the stripe pattern extends in the second direction, the resolution when measuring the three-dimensional shape of the measurement object becomes lower than that in the captured image in which the stripe pattern extends in the first direction.

[0077] [Configuration of three-dimensional shape measuring device]

[0078] Figure 3 1 is a diagram showing the structure of a three-dimensional shape measuring device 100. The three-dimensional shape measuring device 100 includes a first imaging unit 1, a second imaging unit 2, a projection unit 3, a storage unit 5, and a control unit 4. The storage unit 5 includes a storage medium such as a ROM (Read Only Memory), a RAM (Random Access Memory), and a hard disk. The storage unit 5 stores a program executed by the control unit 4. The control unit 4 is, for example, a CPU (Central Processing Unit), and functions as a projection control unit 41, an imaging control unit 42, a coordinate determination unit 43, a line determination unit 44, a relationship determination unit 45, a determination unit 46, and a shape measuring unit 47 by executing the program stored in the storage unit 5.

[0079] The projection control unit 41 generates a control signal for projecting a projection image including a pattern for determining projection coordinates onto the measurement object, and inputs the generated control signal to the projection unit 3. For example, the projection control unit 41 projects a plurality of projection images including a pattern of stripes extending in the same direction onto the measurement object using the projection unit 3. The projection control unit 41 may also project a plurality of projection images including stripes extending in different directions onto the measurement object. Figure 4 as well as Figure 5 , an example of a stripe pattern projected by the projection control unit 41 is described.

[0080] [Types of stripe patterns]

[0081] The projection control unit 41 projects a projection image including a binary fringe pattern onto the measurement object. Figure 4 2 is a diagram showing examples of the types of projection images projected by the projection control unit 41 . Figure 4 The black area in φ represents a non-projection area where the projection unit 3 does not project light, and the white area represents a light projection area where the projection unit 3 projects light.

[0082] Figure 4 (a) is a reference pattern (completely black pattern) in which light is not projected onto the entire object to be measured. Figure 4 (b) is a reference pattern (full white pattern) in which light is projected onto the entire object to be measured. Figure 4 (c) to Figure 4 (f) in FIG. 1 shows a binary stripe pattern consisting of a light projection area and a non-projection area, in which stripes of different widths are arranged in the same direction for each projected image. Although described in detail later, Figure 4 The pattern of the stripes shown corresponds to a Gray code, and is used to identify projection coordinates indicating the position of a projection pixel of a projection image corresponding to a captured pixel of a captured image.

[0083] The projection control unit 41 projects a projection image including a grayscale stripe pattern having a sinusoidal luminance distribution onto the measurement object. Figure 5 (a) to (d) in FIG. 1 show examples of grayscale stripe patterns having a sinusoidal luminance distribution. Figure 4 The binary stripe patterns in (c) to (f) are binary images composed of black and white areas. Figure 5 In the grayscale stripe patterns of (a) to (d), the density from the white area to the black area changes in a sine wave shape along the width direction of the stripe. Figure 5 The intervals between the stripes of the grayscale stripe patterns (a) to (d) are constant, and the spatial frequencies of the stripes of the grayscale stripe patterns are, for example, Figure 4The binary stripe pattern in (f) is 4 times larger.

[0084] Figure 5 The grayscale stripe patterns (a) to (d) in FIG. 1 show the same brightness distribution except that the phases of the sine waves representing the brightness distribution differ by 90 degrees. Figure 4 The two reference patterns shown in (a) and (b) are Figure 4 The four binary stripe patterns shown in (c) to (f) and Figure 5 There are a total of 10 projected images of the four grayscale stripe patterns shown in (a) to (d). Figure 5 The grayscale stripe pattern shown is intended to be consistent with Figure 4 The projection coordinates are determined together with the pattern of the stripes shown. Alternatively, the projection control unit 41 may project a plurality of projection images including patterns in which the widths of the stripes differ depending on the directions in which the stripes extend, onto the measurement object.

[0085] As mentioned above, Figure 4 (c) to Figure 4 The binary stripe pattern shown in (f) corresponds to the Gray code. Figure 6 Show Figure 4 (c)~ Figure 4 An example of a Gray code corresponding to the binary stripe pattern shown in (f) of FIG. 0 in the Gray code corresponds to the non-projection area, and 1 corresponds to the light projection area, thereby generating Figure 4 (c) to Figure 4 The binary stripe pattern shown in (f).

[0086] Figure 4 as well as Figure 6 Each position in the x direction in is represented by a code value that is a combination of 0 or 1 numbers corresponding to the position of each Gray code. Figure 6 Position 0 in corresponds to the code value "0000", position 1 corresponds to the code value "0001", and position 15 corresponds to the code value "1000".

[0087] Figure 3 The imaging control unit 42 generates a first captured image obtained by imaging a projection image projected onto the measurement object using the first imaging unit 1. The imaging control unit 42 generates a second captured image obtained by imaging a projection image projected onto the measurement object using the second imaging unit 2.

[0088] [Determination of projection coordinates]

[0089] The coordinate determination unit 43 includes a first coordinate determination unit 431 and a second coordinate determination unit 432. The first coordinate determination unit 431 determines the projection coordinates corresponding to the first captured pixel of the first captured image based on the pattern included in the first captured image. The first coordinate determination unit 431 determines the projection coordinates indicating the position of the projection pixel corresponding to the captured pixel included in the first captured image by analyzing the change in lightness and darkness in the pattern included in the first captured image.

[0090] The first coordinate determination unit 431 calculates the projection coordinate for each pixel. Figure 4 The brightness value and projection of the all-black pattern shown in (a) Figure 4 The average value of the brightness value in the case of the all-white pattern shown in (b) is taken as the median value. Figure 4 (c)~ Figure 4 The binary stripe pattern in (f) is projected onto the captured image of the state of the measured object, and the brightness value of each captured pixel in the four captured images is compared with the corresponding intermediate value, thereby determining the code value of each captured pixel. The first coordinate determination unit 431 can determine the position to which the binary stripes are projected at the position of each captured pixel by determining the code value. The first coordinate determination unit 431 determines the position of each captured pixel contained in the captured image. Figure 6 Which position of position 0 to position 15 is shown.

[0091] Furthermore, the first coordinate determination unit 431 determines the phase of the sine wave corresponding to the captured pixel in the captured image generated when the pattern of grayscale stripes with a sinusoidal brightness distribution is projected onto the measurement object as the projection coordinate. The pattern of grayscale stripes in the projected image has periodicity, so in the projected image, a plurality of projected pixels have the same projection coordinate. Hereinafter, the projection coordinates having periodicity in the projected image are also referred to as relative projection coordinates. On the other hand, the projection coordinates uniquely determined in the projected image are also particularly referred to as absolute projection coordinates.

[0092] Figure 7 It is a diagram showing the relationship between absolute projection coordinates and relative projection coordinates. Figure 7 The vertical axis represents the projection coordinates. Figure 7 The horizontal axis of represents the position of the projection pixel in the width direction of the stripes contained in the projection image. The width direction is a direction orthogonal to the direction in which the stripes extend. Figure 7 As shown by the solid line, the relative projection coordinates have periodicity. The relative projection coordinates show the same value for each cycle in which the pattern of grayscale stripes with a sinusoidal brightness distribution is repeated. On the other hand, Figure 7 As shown by the dotted line, the absolute projection coordinates are uniquely determined in the projection image.

[0093] The first coordinate determination unit 431 determines the relative projection coordinates corresponding to the captured pixel by analyzing the lightness and darkness of the grayscale stripe pattern. The first coordinate determination unit 431 determines which position of position 0 to position 15 the captured pixel corresponds to based on the Gray code shown by the binary stripe pattern, thereby determining the absolute projection coordinates corresponding to the captured pixel. Let k (=1, 2) be an index representing the longitudinal and lateral directions. At this time, the longitudinal and lateral components (ip, jp) of the projected pixel contained in the projected image are expressed relative to each captured pixel (i, j) as follows.

[0094] [Formula 1]

[0095]

[0096] I AP，k (i, j) {where k=1, 2} is the absolute projection coordinate in the captured image captured in a state where a vertical stripe pattern and a horizontal stripe pattern having a sinusoidal brightness distribution are projected. k It is the number of pixels corresponding to one repetition period of the fringes of the projected image.

[0097] The correspondence between the projection pixel that irradiates light to a certain measurement point of the measurement object and the imaging pixel that receives light from the measurement point is constant even when the pattern of the projection image projected to the measurement object is changed. However, when multiple reflections occur, when the projection image projected by the projection control unit 41 is changed, the first coordinate determination unit 431 may determine different projection coordinates as the projection coordinates corresponding to the same imaging pixel due to changes in the intensity of the multiple reflections.

[0098] The first coordinate determination unit 431 checks whether the projection coordinates corresponding to the captured pixels are changed when the projection image projected by the projection control unit 41 is changed. The first coordinate determination unit 431 determines the projection coordinates corresponding to the first captured pixels for each projection image with different stripe extension directions when the projection control unit 41 projects a plurality of projection images with different stripe extension directions.

[0099] The second coordinate determination unit 432 determines the projection coordinates corresponding to the second captured pixel of the second captured image based on the pattern included in the second captured image, similar to the first coordinate determination unit 431. When the projection control unit 41 projects a plurality of projection images in which the stripes extend in different directions, the second coordinate determination unit 432 determines the projection coordinates corresponding to the second captured pixel for each projection image in which the stripes extend in different directions.

[0100] [Determination of the epipolar line]

[0101] The line determination unit 44 determines the first epipolar line of the second captured image corresponding to the first captured pixel based on the arrangement of the first capturing unit 1 and the second capturing unit 2 . Figure 8 An example is shown in which an image of a measurement target object is captured by the first imaging unit 1 and the second imaging unit 2 .

[0102] The measurement point MP corresponds to Figure 8 The first imaging pixel A in the image plane of the first imaging unit 1 on the left and the second imaging pixel B in the image plane of the second imaging unit 2 on the right corresponding to the same measurement point MP have a certain relationship based on the configuration of the two imaging units. The optical center of the first imaging unit 1 is set as O1, and the straight line extending from the optical center O1 of the first imaging unit through the first imaging pixel A to the measurement point MP is projected onto the image plane of the second imaging unit 2 on the right as the first epipolar line E. BA .

[0103] The second photographing pixel B is located on the first epipolar line E due to geometric constraints. BA The line determination unit 44 reads the configuration information indicating the configuration of the first imaging unit 1 and the second imaging unit 2 which is pre-stored in the storage unit 5, and determines the first epipolar line E of the second captured image corresponding to the first imaging pixel A based on the read configuration information. BA The line determination unit 44 determines the first epipolar line E BA The first image-capturing pixel A is associated with the first image-capturing pixel A and stored in the storage unit 5. For example, the line determination unit 44 stores the first epipolar line E BA The coordinate values ​​of the longitudinal components and the coordinate values ​​of the transverse components of the second imaging pixels on the image sensor 1 are associated with the first imaging pixels A and stored in the storage unit 5 .

[0104] The line determination unit 44 determines the second epipolar line of the projection image corresponding to the second imaging pixel based on the arrangement of the second imaging unit 2 and the projection unit 3 . Fig. 9 The second epipolar line E of the projection image corresponding to the second photographing pixel B CB For example. Fig. 9 In the case of , if the projection unit 3 is regarded as an imaging unit in which the traveling direction of the light is opposite to that of the projection unit 3, the second imaging pixel B of the second imaging unit 2 and the projection pixel C of the projection unit 3 can have the same Figure 8 The line determination unit 44 reads the configuration information indicating the configuration of the second imaging unit 2 and the projection unit 3 which is pre-stored in the storage unit 5, and determines the second epipolar line E of the projection image corresponding to the second imaging pixel B based on the read configuration information. CB .

[0105] When the light emitted from the projection pixel C is reflected only once at the measurement point MP of the measurement object and received by the second imaging pixel B, the projection pixel C is located on the second epipolar line E due to geometric constraints. CB The line determination unit 44 determines the second epipolar line E CB The second image-capturing pixel B is associated with the second image-capturing pixel B and stored in the storage unit 5. For example, the line determination unit 44 sets the second epipolar line E CB The coordinate value of the longitudinal component of each projection pixel on the image sensor 1 is associated with the coordinate value of the lateral component and the second imaging pixel B and is stored in the storage unit 5 .

[0106] The line determination unit 44 determines the third epipolar line (not shown) of the projection image corresponding to the first imaging pixel based on the configuration of the first imaging unit 1 and the projection unit 3. The first imaging pixel A of the first imaging unit and the projection pixel C of the projection unit 3 have the same Figure 8 The line identification unit 44 reads out the arrangement information indicating the arrangement of the first imaging unit 1 and the projection unit 3 which is stored in advance in the storage unit 5, and identifies the third epipolar line of the projection image corresponding to the first imaging pixel A based on the read arrangement information.

[0107] When the light emitted from the light emitting element of the projection pixel C is reflected once by the measurement object and received by the light receiving element of the first imaging pixel A, the projection pixel C is located at any position on the third epipolar line due to geometric constraints. The line determination unit 44 associates the determined third epipolar line with the first imaging pixel A and stores it in the storage unit 5. For example, the line determination unit 44 associates the coordinate values ​​of the longitudinal component and the coordinate values ​​of the transverse component of each projection pixel on the third epipolar line with the first imaging pixel A and stores them in the storage unit 5.

[0108] [Determination of Combination of First Photographing Pixel and Second Photographing Pixel]

[0109] The relationship specifying unit 45 specifies a combination of the first imaging pixel and the second imaging pixel corresponding to the same measurement point MP of the measurement object. The relationship specifying unit 45 obtains the first epipolar line E of the second imaging image associated with the first imaging pixel A from the storage unit 5. BA .

[0110] If the absolute projection coordinate corresponding to the first captured pixel A is set to I 1AP (i A , j A ), then on the first epipolar line E BA The absolute projection coordinate I of the first captured pixel A is the same as that of the first captured pixel A. 1AP (i A , j A). The second pixel corresponds to the first pixel A and the same measurement point MP of the measurement object in the second captured image. Figure 8 As shown, in the second captured image, the first epipolar line E is selected BA and the same absolute projection coordinate I as the first photographed pixel A 1AP (i A , j A ) The relationship specifying unit 45 specifies the combination of the first image-capturing pixel A and the selected second image-capturing pixel B as a combination corresponding to the same measurement point MP.

[0111] Similarly, the relationship determination unit 45 determines a plurality of combinations of first pixels and second pixels corresponding to a common measurement point of the measurement object between the plurality of first pixels included in the first image and the plurality of second pixels included in the second image, and outputs the determined combinations to the determination unit 46.

[0112] [Multiple reflections]

[0113] The determination unit 46 determines whether at least one of the first imaging pixel or the second imaging pixel is a defective pixel caused by multiple reflection or the like. Fig.10 (a) and Fig.10 (b) in FIG. 1 is a diagram for explaining multiple reflections. When the object to be measured is glossy and has a complex shape, the light emitted by the projection unit 3 may be reflected multiple times on the surface of the object to be measured and then enter the first imaging unit 1 or the second imaging unit 2. In this case, Fig.10 As shown in (a) in FIG. 1 , light emitted from the projection unit 3 enters one imaging pixel of the imaging element 12 of the first imaging unit 1 via two or more paths.

[0114] exist Fig.10 In the example (a) in FIG. 1 , the light incident on the imaging element 12 includes direct reflected light emitted from the projection unit 3, which is diffusely reflected on the measurement target surface and directly incident on the first imaging unit 1 or the second imaging unit 2, and multiple reflected light, which is multiple reflected light emitted from the projection unit 3 and then incident on the first imaging unit 1 or the second imaging unit 2. As a result, in the captured image captured by the first imaging unit 1 or the second imaging unit 2, there is a case where a pixel with a brightness value corresponding to black in the absence of the multiple reflected light becomes a brightness value corresponding to white. In particular, multiple reflections are likely to occur because the measurement target includes metal or the like that is prone to diffuse reflection.

[0115] Fig.10 (b) is a diagram showing an example of a captured image affected by multiple reflections. Fig.10(b) and Figure 1 The corresponding part is (c) in FIG. 1 , but the part indicated by the diagonal lines becomes different from the part indicated by the diagonal lines due to the influence of multiple reflected light. Figure 1 The brightness in (c) is different from the brightness in the above. In addition, due to the influence of multiple reflected light, the sine waveform shown in the brightness distribution of the grayscale stripe pattern may be distorted.

[0116] [Determine whether it is a bad pixel]

[0117] The determination unit 46 determines the projection pixel of the projection image having the projection coordinates corresponding to the first imaging pixel and the projection pixel of the projection image having the projection coordinates corresponding to the second imaging pixel, respectively, from the combination of the first imaging pixel and the second imaging pixel determined by the relationship determination unit 45. The determination unit 46 determines whether at least one of the first imaging pixel or the second imaging pixel is a defective pixel based on the distance between the determined two projection pixels.

[0118] As shown below, the determination unit 46 determines whether at least one of the first shooting pixel or the second shooting pixel is a defective pixel by different methods when the projection control unit 41 only projects a plurality of projection images including a pattern of stripes extending in the same direction, and when the projection control unit 41 projects a plurality of projection images including a pattern of stripes having stripes extending in different directions.

[0119] [Single direction fringe measurement]

[0120] Fig.11 FIG. 4 is a diagram showing a method for determining a defective pixel by the determination unit 46 when the projection control unit 41 projects only a plurality of projection images including patterns of stripes extending in the same direction. The determination unit 46 determines a pixel having a projection coordinate (i C , j CA1 )) is a projection pixel of the projection image. The lateral component i of the projection pixel C It is expressed as follows.

[0121] [Formula 2]

[0122]

[0123] Where p is the number of pixels corresponding to one cycle of the repetition of the fringes in the projected image. Fig.11 As shown, the third epipolar line E corresponding to the first shooting pixel A CA There is a horizontal component i of the determined projection pixel C Consistent projected pixel C A The determination unit 46 determines that the projection pixel CA The longitudinal component j CA1 .

[0124] Next, the determination unit 46 determines, among the combinations of the first imaging pixel A and the second imaging pixel B, the projection coordinates (i C , j CB1 ) of the projection pixel. The lateral component ic of the projection pixel is expressed as follows: A same.

[0125] [Formula 3]

[0126]

[0127] like Fig.11 As shown, on the second epipolar line E associated with the second photographing pixel B CB There is a horizontal component i of the determined projection pixel C Consistent projected pixel C B The determination unit 46 determines the projection pixel C B The longitudinal component j CB1 If the first image-capturing pixel A and the second image-capturing pixel B correspond to the same measurement point of the measurement object, then the projection pixel C A And the projected pixel C B Therefore, the determination unit 46 can determine the projection pixel C based on the projection pixel C. A And the projected pixel C B The distance between the first and second photographing pixels A and B is determined to determine whether at least one of the first photographing pixel A and the second photographing pixel B is a defective pixel.

[0128] Represents the projected pixel C A And the projected pixel C B The evaluation value E1 of the distance is expressed as follows.

[0129] [Formula 4]

[0130]

[0131] The determination unit 46 determines whether the evaluation value E1 is larger than a first threshold value. The first threshold value is determined based on the measurement accuracy required of the three-dimensional shape measurement device 100, for example.

[0132] When the evaluation value E1 is greater than the first threshold, the determination unit 46 determines that at least one of the first image pixel A and the second image pixel B is a defective pixel. When the evaluation value E1 is less than the first threshold, the determination unit 46 determines that neither the first image pixel A nor the second image pixel B is a defective pixel. The determination unit 46 obtains the evaluation value E1 for each of the plurality of combinations of the first image pixel and the second image pixel identified by the relationship identification unit 45, and determines whether the obtained evaluation value E1 is greater than the first threshold. The determination unit 46 repeats the determination for the plurality of combinations.

[0133] With this configuration, the determination unit 46 determines whether the first image pixel A and the second image pixel B correspond to substantially the same projection pixel of the projection image in the combination of the first image pixel and the second image pixel determined by the relationship determination unit 45, thereby being able to determine whether at least one of the first image pixel A or the second image pixel B is a defective pixel affected by multiple reflected light or the like. Therefore, the determination unit 46 can suppress a decrease in the measurement accuracy of the shape of the measurement object caused by multiple reflected light or the like.

[0134] in addition, Fig.11 The projected pixel C shown A Need to be located at the second epipolar line E CB With the third nuclear line E CA Therefore, the determination unit 46 may also determine the projection pixel C based on the determined projection pixel C. A The distance between the first photographing pixel A and the projection pixel C is used to determine whether at least one of the first photographing pixel A and the second photographing pixel B is a defective pixel. B It is also necessary to be substantially consistent with the projection pixel C. The determination unit 46 may also determine the projection pixel C based on the determined projection pixel C. B The distance from the projection pixel C is used to determine whether at least one of the first photographing pixel A or the second photographing pixel B is a defective pixel.

[0135] [Measurement of shape]

[0136] By using the principle of triangulation, the shape measuring unit 47 measures the shape of the object to be measured using the projection coordinates corresponding to the combination of the first imaging pixel and the second imaging pixel determined to be not a defective pixel and at least one of the second imaging pixel corresponding to the combination and the projection coordinates corresponding thereto. The shape measuring unit 47 obtains the combination of the first imaging pixel and the second imaging pixel determined by the determining unit 46 as being not a defective pixel.

[0137] The shape measuring unit 47 uses the principle of triangulation in the obtained combination of the first imaging pixel A and the second imaging pixel to obtain the three-dimensional position of the common measurement point corresponding to the first imaging pixel A and the second imaging pixel. The shape measuring unit 47 measures the three-dimensional shape of the measurement object by repeating the same process in the plurality of combinations determined by the relationship determining unit 45. The shape measuring unit 47 measures the three-dimensional shape while excluding defective pixels, so that the measurement accuracy of the three-dimensional shape can be suppressed from being reduced due to the influence of multiple reflections and the like.

[0138] In addition, the shape measuring unit 47 may also use the first image pixel corresponding to the combination of the first image pixel and the second image pixel determined by the determination unit 46 as not being a bad pixel and the projection coordinates corresponding to the first image pixel to obtain the three-dimensional position of the measurement point by using the principle of triangulation. The shape measuring unit 47 measures the three-dimensional shape of the measurement object by repeating the same process for a plurality of first image pixels corresponding to a plurality of combinations of the first image pixel and the second image pixel determined by the determination unit as not being a bad pixel. Similarly, the shape measuring unit 47 may also measure the three-dimensional shape of the measurement object by using the second image pixel corresponding to the combination of the first image pixel and the second image pixel determined by the determination unit 46 as not being a bad pixel and the projection coordinates corresponding to the second image pixel.

[0139] [Modification of Defective Pixel Determination]

[0140] The relationship specifying unit 45 is not limited to specifying a combination of first and second imaging pixels corresponding to the same measurement point MP of the measurement object. For example, the relationship specifying unit 45 may specify a combination of a first imaging pixel and a projection pixel that emits light received by the first imaging pixel.

[0141] The line determination unit 44 and Figure 8 as well as Fig. 9 Similarly, based on the configuration of the second photographing unit 2 and the projection unit 3, the fourth epipolar line E of the second photographed image relative to the projection pixel C is determined. BC The line determination unit 44 reads the configuration information indicating the configuration of the second imaging unit 2 and the projection unit 3 which is pre-stored in the storage unit 5, and determines the fourth epipolar line E of the second captured image corresponding to the projection pixel C based on the read configuration information. BC The line determination unit 44 determines the fourth epipolar line E BC The fourth epipolar line E is associated with the projection pixel C and stored in the storage unit 5. BC The coordinate value of the longitudinal component of each second imaging pixel on the image sensor 10 and the coordinate value of the lateral component of the same second imaging pixel on the image sensor 10 are associated with the projection pixel C and stored in the storage unit 5 .

[0142] The relationship determination unit 45 determines a combination of the first photographing pixel and the projection pixel that emits the light received by the first photographing pixel. The relationship determination unit 45 obtains the third epipolar line of the projection image associated with the first photographing pixel A from the storage unit 5. If the absolute projection coordinate corresponding to the first photographing pixel A is set to I 1AP (i A , j A ), then there exists on the third epipolar line the same absolute projection coordinate I as the first captured pixel A 1AP (i A , j A ). The projection pixel corresponds to the projection pixel that emits the light received by the first photographing pixel A. The relationship determination unit 45 selects a projection pixel having the same absolute projection coordinates as the first photographing pixel from among the projection pixels on the third epipolar line associated with the first photographing pixel A. The relationship determination unit 45 determines a combination of the first photographing pixel A and the selected projection pixel.

[0143] When the relationship specifying unit 45 specifies the combination of the first imaging pixel A and the projection pixel C, the determining unit 46 specifies the same projection coordinates (i C , j CA1 )) corresponds to the second shooting pixel. Fig.12 4 is a diagram showing a method for determining whether a captured pixel is a defective pixel by the determination unit 46. Fig.12 As shown, the first epipolar line E of the second captured image associated with the first captured pixel A BA There is a primary second photographing pixel B corresponding to the same projection coordinates as the first photographing pixel A. A The determination unit 46 determines the main second imaging pixel B A .

[0144] Next, the determination unit 46 determines the second captured pixel of the second captured image having the projection coordinates corresponding to the projection pixel C from the combination of the first captured pixel A and the projection pixel C determined by the relationship determination unit 45. BC There is a secondary second photographing pixel B corresponding to the same projection coordinates as the projection pixel C. C The determination unit 46 determines the auxiliary second imaging pixel B C .

[0145] When the light emitted from the projection pixel C is reflected only once by the measurement object and received by the first imaging pixel A and the second imaging pixel B, the second imaging pixel B A And the secondary second shooting pixel B C Therefore, the determination unit 46 is based on the main second imaging pixel B Aand the secondary second shooting pixel B C The distance between the first shooting pixel A and the second shooting pixel B is determined A Or the secondary second shooting pixel B C The determination unit 46 determines whether at least one of the main second imaging pixel B is a defective pixel. A With the secondary second shooting pixel B C The second threshold value is determined according to the measurement accuracy required by the three-dimensional shape measurement device 100. The determination unit 46 determines whether the distance between the main second imaging pixel B A With the secondary second shooting pixel B C When the distance between the first and second photographing pixels A and B is greater than the second threshold, it is determined that the first photographing pixel A and the second photographing pixel B A Or the secondary second shooting pixel B C At least one of the pixels is a defective pixel. A With the secondary second shooting pixel B C When the distance between the first and second photographing pixels A and B is less than the second threshold, it is determined that the first photographing pixel A and the second photographing pixel B A and the secondary second shooting pixel B C The shape measuring unit 47 uses the determining unit 46 to determine that the first imaging pixel A and the main second imaging pixel B are not defective pixels. A And the secondary second shooting pixel B C The first shooting pixel A and the second shooting pixel B corresponding to the combination that is not a bad pixel A Or the secondary second photographing pixel B C The shape of the object to be measured is measured by at least one of

[0146] [Determination of defective pixels using vertical and horizontal stripes]

[0147] Fig.13 (a) and Fig.13 (b) in FIG. 1 is a diagram showing the relationship between the extension direction of the fringes of the projected image and the multi-reflected light. The multi-reflection is caused by the repeated reflection of the light emitted from the projection unit 3 around the curved part of the measured object. Fig.13 As shown in (a) in FIG. 1 , there is a case where light initially reflected in the light projection area in multiple reflections is then reflected in the non-projection area and then enters the first imaging unit 1 or the second imaging unit 2. In this case, in the captured image generated by the first imaging unit 1 or the second imaging unit 2, the brightness of the non-projection area appears high due to the multiple reflected light, so there is a possibility that the coordinate determination unit 43 erroneously determines the projection coordinates of the captured pixel.

[0148] On the other hand, Fig.13As shown in (b) in FIG. 1 , when the direction in which the bent portion of the object to be measured extends is perpendicular to the direction in which the stripes extend, the probability that the light initially reflected in the light projection area is also reflected in the light projection area in multiple reflections becomes relatively high. Fig.13 In the state shown in (b) in FIG. 1 , the light irradiated from the projection unit 3 toward the light projection area is reflected again in another light projection area and then incident on the first imaging unit 1 or the second imaging unit 2. In this case, the multiple reflected lights have a relatively small effect on the brightness of the light projection area and the non-projection area. Therefore, when the projection control unit 41 projects multiple projection images of patterns including stripes extending in different directions onto the measurement object, the determination unit 46 uses the captured images captured during the projection of images of multiple stripe patterns with different extension directions of the stripes to determine whether it is a bad pixel, thereby further reducing the possibility of overlooking the generation of bad pixels.

[0149] When the projection control unit 41 projects a plurality of projection images including a pattern of stripes extending in the longitudinal direction and in the transverse direction onto the measurement object, the determination unit 46 uses the projection coordinates based on the pattern of stripes extending in the longitudinal direction and the projection coordinates based on the pattern of stripes extending in the transverse direction to determine the projection pixel having the projection coordinates corresponding to the first imaging pixel A. More specifically, the determination unit 46 obtains the combination of the first imaging pixel A and the second imaging pixel B determined by the relationship determination unit 45. As described above, the combination determined by the relationship determination unit 45 is the first imaging pixel A and the first epipolar line E corresponding to the first imaging pixel A in the second imaging image. BA superior( Figure 8 ), and with the first photographing pixel A(i A , j A ) the same absolute projection coordinates I 1AP (i A , j A ) corresponds to the second photographing pixel B(i B , j B ) combination.

[0150] The determination unit 46 determines the absolute projection coordinates (I ) corresponding to the first image-capturing pixel A determined by the first coordinate determination unit, among the combinations of the first image-capturing pixel A and the second image-capturing pixel B determined by the relationship determination unit 45. 1AP , 1 in (i A , j A ), I 1AP , 2 in (i A , j A )) The projected pixel C A I 1AP , 1 in (i A , j A) represents the lateral component of the absolute projection coordinate, I 1AP , 2 in (i A , j A ) represents the longitudinal component of the absolute projection coordinates. The absolute projection coordinates corresponding to the first image-capturing pixel A have two directional components, the lateral component and the longitudinal component. Therefore, the determination unit 46 can obtain the projection pixel (i) having the same absolute projection coordinates as the first image-capturing pixel A as follows: C , j CA2 ).

[0151] [Formula 5]

[0152]

[0153] Wherein, p1 and p2 are the number of pixels corresponding to one period of repetition of stripes extending in the longitudinal direction and the transverse direction, respectively.

[0154] In the combination of the first photographic pixel A and the second photographic pixel B determined by the relationship determination unit 45, the lateral component I of the absolute projection coordinate corresponding to the first photographic pixel A is 1AP , 1 in (i A , j A ) and the lateral component I of the absolute projection coordinate corresponding to the second photographing pixel B 2AP，1 (i A , j A ) are the same. The determination unit 46 determines the second image-capturing pixels B(i B , j B ) and the code value of the horizontal binary stripe pattern, thereby determining the second shooting pixel B(i B , j B ) corresponds to the longitudinal component I of the absolute projection coordinate 2AP，2 (i B , j B The determination unit 46 can determine the projection pixel (i ) having the same absolute projection coordinates as the second imaging pixel B as follows: C , j CB2 ).

[0155] [Formula 6]

[0156]

[0157] When the first imaging pixel A and the second imaging pixel B correspond to the same measurement point of the measurement object, it is necessary to make the projection pixel (i C , j CA2 ) and the projection pixel (i C , j CB2That is, the determination unit 46 can determine whether at least one of the first imaging pixel A or the second imaging pixel B is a defective pixel based on the distance between the two projection pixels.

[0158] Represents the projected pixel (i C , j CA2 ) and the projection pixel (i C , j CB2 ) is expressed as follows.

[0159] [Formula 7]

[0160]

[0161] In essence, the determination unit 46 can use I 1AP , 2 in (i A , j A ) and I 2AP，2 (i B , j B ) calculates the evaluation value E2. The determination unit 46 determines whether the evaluation value E2 is greater than a third threshold value. The third threshold value is determined according to the measurement accuracy required by the three-dimensional shape measurement device 100. When the evaluation value E2 is greater than the third threshold value, the determination unit 46 determines that at least one of the first imaging pixel A and the second imaging pixel B is a defective pixel. When the evaluation value E2 is less than the third threshold value, the determination unit 46 determines that neither the first imaging pixel A nor the second imaging pixel B is a defective pixel.

[0162] [Determination of bad pixels using relative projection coordinates in the horizontal direction]

[0163] The determination unit 46 may also use the absolute projection coordinates in the horizontal direction and the relative projection coordinates in the vertical direction to determine whether the image pixel is a defective pixel. In the combination of the first image pixel A and the second image pixel B determined by the relationship determination unit 45, the projection coordinates of the first image pixel A are set to (I 1AP , 1 in (i A , j A ), I 1RP，2 (i A , j A )). 1AP , 1 in (i A , j A ) is the absolute horizontal projection coordinate, I 1RP，2 (i A , j A ) is the relative projection coordinate in the longitudinal direction.

[0164] In addition, in the combination of the first image-capturing pixel A and the second image-capturing pixel B determined by the relationship determination unit 45, the projection coordinates of the second image-capturing pixel B are (I2AP，1 (i B , j B ), I 2RP，2 (i B , j B )). 2AP，1 (i B , j B ) is the absolute horizontal projection coordinate, I 2RP，2 (i B , j B ) is the relative projection coordinate in the longitudinal direction. The projection pixel C having the same projection coordinate as the first photographing pixel A A The lateral component i C and the longitudinal component j CA3 It is expressed as follows.

[0165] [Formula 8]

[0166]

[0167] The projection pixel C having the same projection coordinates as the second photographing pixel B B The lateral component i C and the longitudinal component j CB3 In this case, the projection pixel C B The lateral component ic is also related to the projected pixel C A The lateral component ic is the same.

[0168] [Formula 9]

[0169]

[0170] Wherein, q2 is an unknown number, but q2 is considered to be common in the first photographing pixel A and the second photographing pixel B. C , j CA3 ) and the projection pixel (i C , j CB3 ) is expressed as follows.

[0171] [Formula 10]

[0172]

[0173] The determination unit 46 determines whether the evaluation value E3 is greater than a fourth threshold value. The fourth threshold value is determined according to the measurement accuracy required by the three-dimensional shape measurement device 100. When the evaluation value E3 is greater than the fourth threshold value, the determination unit 46 determines that at least one of the first imaging pixel A and the second imaging pixel B is a defective pixel. When the evaluation value E3 is less than the fourth threshold value, the determination unit 46 determines that neither the first imaging pixel A nor the second imaging pixel B is a defective pixel.

[0174] In relative projection coordinates, such as Figure 7 In this way, the range of values ​​is limited to 2π and has periodicity. At this time, at a position where the value changes sharply by 2π, there is a possibility that the error of the evaluation value E3 becomes large due to a slight difference in the corresponding pixels. Therefore, the determination unit 46 may also calculate the following evaluation value E3 in addition to the evaluation value E3. 3a and the evaluation value E 3B The minimum value of these three evaluation values ​​can be used as a new evaluation value for determination. 3A And the evaluation value E 3B It is expressed as follows.

[0175] [Formula 11]

[0176]

[0177] [Formula 12]

[0178] E 3B =√(j CA3 -j CB3 -p2) 2

[0179] [Determination of defective pixels using stripes in three or more directions]

[0180] When the projection control unit 41 projects multiple projection images of a pattern including stripes extending in more than three different directions onto the measured object, the determination unit 46 determines, for each projection image of the pattern including stripes extending in different directions, a projection pixel having the projection coordinates corresponding to the first shooting pixel A using the projection coordinates determined by the first coordinate determination unit 431.

[0181] The determination unit 46 determines the projection coordinates (I ) corresponding to the first imaging pixel A determined by the first coordinate determination unit 431 in the combination of the first imaging pixel A and the second imaging pixel B. 1AP , 1 in (i A , j A ), I 1RP，2 (i A , j A ), I 1RP，3 (iA , j A ),···、I 1RP，M (i A , j A )), determine a projection pixel having the same projection coordinates as the first photographing pixel A. The component I in the first direction of the projection coordinates corresponding to the first photographing pixel A 1AP , 1 in (i A , j A ) is the absolute projection coordinate, and the components I in the second direction to the Mth direction of the projection coordinate corresponding to the first captured pixel A 1RP，2 (i A , j A ), I 1RP，3 (i A , j A ),···、I 1RP，M (i A , j A ) are all relative projection coordinates.

[0182] The determination unit 46 uses the projection coordinates (I ) corresponding to the second image-capturing pixel B determined by the second coordinate determination unit 432 in the combination of the first image-capturing pixel A and the second image-capturing pixel B. 2AP，1 (i B , j B ), I 2RP，2 (i B , j B ), I 2RP，3 (i B , j B ),···、I 2RP，M (i B , j B )), determine a projection pixel having the same projection coordinates as the second photographing pixel B. The component I in the first direction of the projection coordinates corresponding to the second photographing pixel B 2AP，1 (i B , j B ) is the absolute projection coordinate, and the component I in the second direction to the Mth direction of the projection coordinate corresponding to the second captured pixel B 2RP，2 (i B , j B ), I 2RP，3 (i B , j B ),···、I 2RP，M (i B , j B ) are all relative projection coordinates.

[0183] The determination unit 46 obtains an evaluation value E4 for evaluating the deviation between the projection pixel having the projection coordinates corresponding to the first imaging pixel A and the projection pixel having the projection coordinates corresponding to the second imaging pixel B.

[0184] The evaluation value E4 is expressed as follows.

[0185] [Formula 13]

[0186]

[0187] The determination unit 46 determines whether the evaluation value E4 is greater than a fifth threshold value. The fifth threshold value is determined according to the measurement accuracy required by the three-dimensional shape measurement device 100. When the evaluation value E4 is greater than the fifth threshold value, the determination unit 46 determines that at least one of the first imaging pixel A and the second imaging pixel B is a defective pixel. When the evaluation value E4 is less than the fifth threshold value, the determination unit 46 determines that neither the first imaging pixel A nor the second imaging pixel B is a defective pixel.

[0188] [Determination of Defective Pixel Based on Maximum Comparison Value]

[0189] In addition, the determination unit 46 may obtain a comparison value between the projection coordinates corresponding to the first pixel A and the projection coordinates corresponding to the second pixel B for each of a plurality of projection images having different extending directions of fringes in the combination of the first pixel A and the second pixel B determined by the relationship determination unit 45. The comparison value may be, for example, a difference value between the projection coordinates corresponding to the first pixel A and the projection coordinates corresponding to the second pixel B, or may be a value of a ratio between the projection coordinates corresponding to the first pixel A and the projection coordinates corresponding to the second pixel B.

[0190] The determination unit 46 obtains the projection coordinates (I 1AP , 1 in (i A , j A ), I 1RP，2 (i A , j A ), I 1RP，3 (i A , j A ),···、I 1RP，M (i A , j A )) and the projection coordinates corresponding to the second captured pixel (I 2AP，1 (i B , j B ), I 2RP，2 (i B , j B ), I 2RP，3 (iB , j B ),···、I 2RP，M (i B , j B ))Comparison values ​​of components in respective directions.

[0191] The determination unit 46 determines whether at least one of the first imaging pixel or the second imaging pixel is a defective pixel by using an evaluation value based on the obtained comparison value. 1AP , 1 in (i A , j A )-I 2AP，1 (i B , j B ))、(I 1RP，2 (i A , j A )-I 2RP，2 (i B , j B )、(I 1RP，3 (i A , j A )-I 2RP，3 (i B , j B ))、···、(I 1RP，M (i A , j A )-I 2RP，M (i B , j B )) determines the component in the direction of the maximum value.

[0192] The determination unit 46 uses the determined maximum value as the evaluation value E5, and determines whether the evaluation value E5 is greater than a sixth threshold value. The sixth threshold value is determined according to the measurement accuracy required by the three-dimensional shape measurement device 100. When the evaluation value E5 is greater than the sixth threshold value, the determination unit 46 determines that at least one of the first imaging pixel A and the second imaging pixel B is a defective pixel.

[0193] When the determined evaluation value E5 is less than or equal to the sixth threshold value, the determination unit 46 determines that neither the first imaging pixel A nor the second imaging pixel B is a defective pixel. The determination unit 46 determines whether at least one of the first imaging pixel and the second imaging pixel is a defective pixel using the pattern of the stripes that have the greatest influence on the defective pixel. Therefore, the determination unit 46 can suppress the omission of defective pixels with high accuracy.

[0194] In addition to this, the determination unit 46 may use the standard deviation of the comparison value as a comparison method to determine the first imaging pixel whose evaluation value determined with respect to other first imaging pixels is intentionally large as a defective pixel. In addition, the determination unit 46 may use a coefficient of weight that specifies the importance for each stripe direction to calculate the above-mentioned average value, maximum value, standard deviation, etc. For example, the number of projection images projected by the projection control unit 41 and the stripe cycle may be changed for each stripe direction, and the determination unit 46 may calculate the evaluation value using a coefficient of different weight.

[0195] [Measurement Processing of Three-Dimensional Shape Measuring Device]

[0196] Fig.14 1 is a flowchart showing a process of measuring the shape of the object to be measured by the three-dimensional shape measuring device 100. This process sequence starts when an operation accepting unit (not shown) accepts an operation by a user instructing to start measuring the shape of the object to be measured.

[0197] First, the projection control unit 41 projects a projection image including a pattern for determining projection coordinates onto the measurement object using the projection unit 3 (S101). Next, the imaging control unit 42 generates a first captured image obtained by imaging the projection image projected onto the measurement object using the first imaging unit 1. In addition, the imaging control unit 42 generates a second captured image obtained by imaging the projection image projected onto the measurement object using the second imaging unit 2 (S102).

[0198] The first coordinate determination unit 431 determines the projection coordinates corresponding to the first pixel of the first captured image based on the pattern included in the first captured image. The second coordinate determination unit 432 determines the projection coordinates corresponding to the second pixel of the second captured image based on the pattern included in the second captured image (S103). The relationship determination unit 45 obtains the first epipolar line of the second captured image associated with the first pixel from the storage unit 5 (S104). The relationship determination unit 45 selects the second pixel corresponding to the same projection coordinates as the first pixel from the second pixels located on the obtained first epipolar line.

[0199] The relationship determination unit 45 determines the combination of the first image pixel and the selected second image pixel as a combination corresponding to the same measurement point (S105). The relationship determination unit 45 also determines a plurality of combinations corresponding to the common measurement point of the measurement object between the plurality of first image pixels included in the first image and the plurality of second image pixels included in the second image.

[0200] The determination unit 46 obtains the distance between the projection pixel of the projection image having the projection coordinates corresponding to the first image pixel and the projection pixel of the projection image having the projection coordinates corresponding to the second image pixel in the combination of the first image pixel and the second image pixel determined by the relationship determination unit 45. The determination unit 46 determines whether the evaluation value E1 indicating the obtained distance is greater than the first threshold value (S106).

[0201] When the evaluation value E1 is greater than the first threshold value (Yes in S106), the determination unit 46 determines that at least one of the first imaging pixel and the second imaging pixel is a defective pixel in the combination determined by the relationship determination unit 45 (S107). The shape measurement unit 47 determines whether there is another combination determined by the relationship determination unit 45 (S108).

[0202] When the shape measurement unit 47 determines that there is no other combination determined by the relationship determination unit 45 (No in S108), it uses the pixels corresponding to at least one of the combinations of the first shooting pixel and the second shooting pixel determined by the determination unit 46 as not being defective pixels to measure the three-dimensional shape of the measurement object (S109), and ends the processing.

[0203] When the evaluation value E1 is less than the first threshold value in the determination of S106 (No in S106), the determination unit 46 determines that neither the first image-capturing pixel nor the second image-capturing pixel is a defective pixel (S110), and proceeds to the process of S108. When the shape measurement unit 47 determines that there is another combination determined by the relationship determination unit 45 in the determination of S108 (Yes in S108), the process of S106 is performed again.

[0204] [Effects of the present invention]

[0205] The determination unit 46 determines whether the first image pixel A and the second image pixel B correspond to substantially the same projection pixel of the projection image in the combination of the first image pixel and the second image pixel determined by the relationship determination unit 45, thereby being able to determine whether at least one of the first image pixel A or the second image pixel B is a defective pixel affected by multiple reflected light or the like. Therefore, the determination unit 46 can suppress a decrease in the measurement accuracy of the shape of the measurement object by the shape measurement unit 47 due to multiple reflected light or the like.

[0206] <Second embodiment>

[0207] [Readjustment based on a combination of stripes in multiple directions]

[0208] In the first embodiment, an example is described in which the relationship determination unit 45 determines the combination of the first image pixel and the second image pixel corresponding to the same measurement point of the measurement object using the projection coordinates based on the stripe pattern extending in a single direction. In this case, the first image pixel in the first image and the second image pixel in the second image do not correspond to the same position of the measurement object. Therefore, the relationship determination unit 45 in the second embodiment determines the combination of the first image pixel and the second image pixel using the projection coordinates based on the stripe pattern extending in multiple different directions, thereby improving the accuracy of determining the combination corresponding to the same measurement point of the measurement object. Therefore, the shape measurement unit 47 can improve the measurement accuracy of the three-dimensional shape of the measurement object.

[0209] Specifically, in the second embodiment, the relationship specifying unit 45 readjusts the combination of the first imaging pixels and the second imaging pixels corresponding to the same measurement point of the measurement object using projection coordinates based on fringe patterns extending in a plurality of different directions.

[0210] Fig.15 FIG. 2 is a diagram showing the configuration of a control unit 50 included in the three-dimensional shape measuring apparatus according to the second embodiment. Figure 1 In addition to the modules of the control unit 4, the device further includes an acquisition unit 501, a candidate coordinate determination unit 502, and an evaluation unit 503. Figure 1 The same blocks of the control unit 4 are denoted by the same reference numerals and their description is omitted.

[0211] The acquisition unit 501 first acquires the combination of the first imaging pixel and the second imaging pixel determined by the relationship determination unit 45 using the projection coordinates based on the stripe pattern extending in a single direction. The acquisition unit 501 acquires a plurality of second imaging pixels within a predetermined range from the second imaging pixels of the acquired combination as corresponding candidate pixels. The predetermined range is, for example, a range adjacent to the second imaging pixels in the combination acquired by the acquisition unit 501. The acquisition unit 501 may also acquire corresponding candidate pixels in units smaller than pixels. For example, when the pixel is a 1-pixel unit, the acquisition unit 501 acquires a plurality of corresponding candidate pixels in a sub-pixel unit. A sub-pixel indicates a length smaller than a pixel in a 1-pixel unit.

[0212] When the projection control unit 41 projects a plurality of projection images with different extension directions of the stripes, the candidate coordinate determination unit 502 determines the projection coordinates corresponding to the corresponding candidate pixels for each projection image with different extension directions of the stripes. The candidate coordinate determination unit 502 determines the projection coordinates of the plurality of corresponding candidate coordinates. As an example, the candidate coordinate determination unit 502 determines the projection coordinates of the plurality of corresponding candidate coordinates in sub-pixel units.

[0213] The evaluation unit 503 obtains a comparison value between the projection coordinates corresponding to the first captured pixel and the projection coordinates corresponding to the corresponding candidate pixel for each projection image in which the extending directions of the fringes are different from each other, and obtains an evaluation value based on the obtained comparison value for each corresponding candidate pixel. The comparison value is, for example, a difference value between the projection coordinates corresponding to the first captured pixel and the projection coordinates corresponding to the corresponding candidate pixel, but may also be a value of the ratio of the projection coordinates corresponding to the first captured pixel to the projection coordinates corresponding to the corresponding candidate pixel. Fig.16 3 is a diagram showing an evaluation method of the corresponding candidate pixel by the evaluation unit 503 .

[0214] First, with Fig.11 Similarly, the evaluation unit 503 determines the projection coordinates (I ) that are the same as the first image-capturing pixel A in the combination of the first image-capturing pixel A and the second image-capturing pixel B determined by the relationship determination unit 45. 1AP , 1 in (i A , j A ), I 1RP，2 (i A , j A ), I 1RP，3 (i A , j A ),···、I 1RP，M (i A , j A ))'s projected pixel C A The evaluation unit 503 determines the pixel having the same projection coordinates (I 2AP，1 In ( iB1 , j iB1 ), I 2RP，2 In ( iB1 , j iB1 ), I 2RP，3 In ( iB1 , j iB1 )···、I 2RP，M ( iB1 , j iB1 ))'s projected pixel C B1 The evaluation unit 503 calculates the projection pixel C A The projection coordinates and projection pixel C B1 The following evaluation value E6 is the comparison value in each direction component of the projection coordinates.

[0215] [Formula 14]

[0216]

[0217] The evaluation unit 503 similarly determines the projection pixel C having the same projection coordinates as the corresponding candidate pixel B2. B2 , calculated based on the projection pixel C AThe projection coordinates and projection pixels C B1 The evaluation unit 503 calculates the evaluation value E6 of the comparison value in each direction component of the projection coordinates. In addition, the evaluation unit 503 calculates the evaluation value E6 of the comparison value in each direction component of the projection coordinates of the projection image C having the same projection coordinates as the second pixel B in the combination of the first pixel A and the second pixel B determined by the relationship determination unit 45. B The evaluation value E6 is calculated in the same manner.

[0218] The relationship determination unit 45 determines, as a new combination, a combination of any one of the plurality of corresponding candidate pixels and the first imaging pixel A acquired by the acquisition unit 501, based on the evaluation value E6 calculated by the evaluation unit 503. For example, the relationship determination unit 45 selects a corresponding candidate pixel for which the evaluation value E6 calculated by the evaluation unit 503 is the smallest, and determines the combination of the first imaging pixel A and the corresponding candidate pixel as a new combination.

[0219] According to this embodiment, the relationship determination unit 45 uses the projection coordinates based on the pattern of stripes extending in multiple directions to more accurately determine the combination of the first imaging pixel and the second imaging pixel corresponding to the same measurement point of the measurement object. Therefore, the accuracy of the determination unit 46 in determining whether the imaging pixel is a defective pixel is also improved, so the shape measurement unit 47 can measure the shape of the measurement object with improved accuracy.

[0220] [Remeasurement processing using the projected image excluding the defective pixel]

[0221] Since bad pixels are often generated by multiple reflected light, the influence of multiple reflected light can be suppressed by projecting a projection image other than the projection pixels of the projection coordinates corresponding to the imaging pixels determined as bad pixels. Therefore, when the determination unit 46 determines that at least one of the first imaging pixel or the second imaging pixel is a bad pixel, the projection control unit 41 may re-project a projection image (hereinafter referred to as a first selected projection image) to the measurement object, removing all the projection pixels having the projection coordinates corresponding to any one of the first imaging pixel and the second imaging pixel corresponding to the combination determined by the determination unit 46 as not being bad pixels, among the plurality of projection pixels included in the projection image.

[0222] The bad pixel may be affected by multiple reflected light, so the projection control unit 41 can suppress the influence of multiple reflected light by projecting the first selected projection image excluding the bad pixel. Therefore, the first coordinate determination unit 431 can determine the projection coordinates corresponding to the first captured pixel with high accuracy. Similarly, the second coordinate determination unit 432 can determine the projection coordinates corresponding to the second captured pixel with high accuracy.

[0223] [Remeasurement processing using a projected image containing only defective pixels]

[0224] It is also considered that bad pixels caused by multiple reflections are generated by simultaneously projecting a projection image containing multiple pixels onto the measurement object. Therefore, when the determination unit 46 determines that at least one of the first imaging pixel or the second imaging pixel is a bad pixel, the projection control unit 41 may re-project a projection image (hereinafter referred to as a second selected projection image) containing only projection pixels having projection coordinates corresponding to the first imaging pixel or the second imaging pixel corresponding to the combination determined as bad pixels by the determination unit 46 to the measurement object, and perform measurement again, for the purpose of reducing the number of simultaneously projected projection pixels.

[0225] The projection control unit 41 reduces the number of pixels projected simultaneously by projecting the second selected projection image, compared with the case where all projection images are projected. The first coordinate determination unit 431 and the second coordinate determination unit 432 can increase the number of imaging pixels used for measuring the three-dimensional shape of the measurement object by re-determining the projection coordinates corresponding to the first imaging pixel or the second imaging pixel determined as a defective pixel by the determination unit 46.

[0226] [Projection of multiple projection images including fringe patterns with different periods]

[0227] In addition, the projection control unit 41 may sequentially project a plurality of projection images including a pattern of stripes having different sinusoidal brightness distributions in the period of the stripes. For example, the projection control unit 41 may additionally project a projection image including a pattern of stripes extending in the first direction and having different periods onto the measurement object after projecting a projection image including a pattern of stripes extending in the first direction onto the measurement object.

[0228] In addition, the projection control unit 41 may additionally project a projection image including a pattern of stripes extending in the second direction and having different periods onto the measurement object after projecting the projection image including a pattern of stripes extending in the second direction onto the measurement object. If light that is reflected only once on the measurement surface of the measurement object and received in the imaging pixel is set as direct reflected light, then when the projection image including a pattern of stripes with different periods is projected, the phase of the multiple reflected light overlapped with the direct reflected light changes. Therefore, the determination unit 46 can detect the pixel affected by the multiple reflected light with higher accuracy.

[0229] In addition, in the first and second embodiments, an example is described in which the projection control unit 41 projects a plurality of projection images including binary stripe patterns with different periods and a projection image including a pattern of stripes with a sinusoidal brightness distribution. However, the present invention is not limited thereto. For example, the projection control unit 41 may project a plurality of projection images including a pattern of stripes with a sinusoidal brightness distribution with different periods onto the measurement object instead of the plurality of projection images including binary stripe patterns.

[0230] [Detection of system failure]

[0231] The determination unit 46 is not limited to the example of determining whether the imaging pixel is a defective pixel, and may determine whether a defect has occurred in the configuration of any one of the first imaging unit 1, the second imaging unit 2, and the projection unit 3. The determination unit 46 determines whether a defect has occurred in the configuration of any one of the first imaging unit 1, the second imaging unit 2, and the projection unit 3 using the statistics of the plurality of evaluation values ​​E1 obtained for each of the plurality of combinations of the first imaging pixel and the second imaging pixel determined by the relationship determination unit 45.

[0232] When the statistic of the obtained plurality of evaluation values ​​E1 exceeds the reference value, the determination unit 46 determines that a fault has occurred in the configuration of any one of the first imaging unit 1, the second imaging unit 2, and the projection unit 3. The fault in the configuration refers to, for example, that the configuration of any one of the first imaging unit 1, the second imaging unit 2, and the projection unit 3 deviates from the configuration pre-stored in the storage unit 5.

[0233] The statistic is, for example, an average value of a plurality of evaluation values ​​E1 obtained by the determination unit 46 for each of a plurality of combinations of the first imaging pixel and the second imaging image. The reference value is a value determined as a value indicating that the accuracy of the combination of the first imaging pixel and the second imaging pixel determined by the relationship determination unit 45 is reduced due to an important factor other than the multiple reflected light. With this configuration, the determination unit 46 can determine whether a malfunction has occurred in the configuration of any one of the first imaging unit 1, the second imaging unit 2, and the projection unit 3.

[0234] In the first embodiment and the second embodiment, an example is described in which the three-dimensional shape measuring device 100 includes the first imaging unit 1, the second imaging unit 2, and the projection unit 3. However, the present invention is not limited to this. The three-dimensional shape measuring device 100 may include three or more imaging units and two or more projection units, and the series of processes described in the first embodiment and the second embodiment may be repeated for any combination of these imaging units and projection units.

[0235] The present invention has been described above using the embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments, and various deformations and changes can be made within the scope of its main purpose. For example, all or part of the device can be functionally or physically dispersed and uniformly constructed in arbitrary units. In addition, new embodiments generated by any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments generated by the combination also have the effects of the embodiments.

Claims

1. A three-dimensional shape measuring device, characterized in that: have: a projection unit that projects a projection image of a pattern for determining projection coordinates and a plurality of projection images of the pattern including stripes extending in the same direction onto a measurement object; a first imaging unit that generates a first captured image by imaging the projection image projected onto the measurement object; a second imaging unit that generates a second captured image by imaging the projection image projected onto the measurement object; a first coordinate determining unit, which determines the projection coordinate corresponding to the first captured pixel of the first captured image based on the pattern included in the first captured image; a second coordinate determining unit, which determines the projection coordinate corresponding to the second captured pixel of the second captured image based on the pattern included in the second captured image; a line determining unit, which determines a first epipolar line of the second captured image corresponding to the first captured pixel based on the configuration of the first captured unit and the second captured unit, determines a second epipolar line of the projection image corresponding to the second captured pixel based on the configuration of the second captured unit and the projection unit, and determines a third epipolar line of the projection image corresponding to the first captured pixel based on the configuration of the first captured unit and the projection unit; a relationship determining unit, which determines a combination of the first photographing pixel and the second photographing pixel corresponding to the same projection coordinates as the projection coordinates corresponding to the first photographing pixel and located on the first epipolar line of the second photographing image corresponding to the first photographing pixel; a determination unit that determines, for each combination of the first photographing pixel and the second photographing pixel, whether at least one of the first photographing pixel or the second photographing pixel is a defective pixel based on a distance between a projection pixel of the projection image having the projection coordinates corresponding to the first photographing pixel and a projection pixel of the projection image having the projection coordinates corresponding to the second photographing pixel; as well as a shape measuring unit configured to measure the shape of the measurement object using at least one of the first imaging pixel and the second imaging pixel corresponding to the combination determined as not being the defective pixel, The determination unit determines, for each combination of the first imaging pixel and the second imaging pixel, whether at least one of the first imaging pixel or the second imaging pixel is a defective pixel based on a distance between the projection pixel having the projection coordinates corresponding to the first imaging pixel and located on the third epipolar line of the projection image corresponding to the first imaging pixel and the projection pixel located at an intersection of the second epipolar line and the third epipolar line.

2. A three-dimensional shape measuring device, characterized in that: have: a projection unit that projects a plurality of projection images of a pattern including stripes for determining projection coordinates, wherein the stripes included in the pattern extend in different directions to the object to be measured; a first imaging unit that generates a first captured image by imaging the projection image projected onto the measurement object; a second imaging unit that generates a second captured image by imaging the projection image projected onto the measurement object; a first coordinate determination unit for determining, based on the pattern included in the first captured image, the projection coordinate corresponding to the first captured pixel of the first captured image for each of the plurality of projection images in which the stripes extend in different directions; a second coordinate determination unit for determining, based on the pattern included in the second captured image, the projection coordinate corresponding to the second captured pixel of the second captured image for each of the plurality of projection images in which the stripes extend in different directions; a line determining unit, which determines a first epipolar line of the second captured image corresponding to the first captured pixel based on the configuration of the first captured unit and the second captured unit; a relationship determining unit, which determines a combination of the first photographing pixel and the second photographing pixel corresponding to the same projection coordinates as the projection coordinates corresponding to the first photographing pixel and located on the first epipolar line of the second photographing image corresponding to the first photographing pixel; a determination unit that determines, for each combination of the first photographing pixel and the second photographing pixel, whether at least one of the first photographing pixel or the second photographing pixel is a defective pixel based on a distance between a projection pixel of the projection image having the projection coordinates corresponding to the first photographing pixel and a projection pixel of the projection image having the projection coordinates corresponding to the second photographing pixel; as well as a shape measuring unit configured to measure the shape of the measurement object using at least one of the first imaging pixel and the second imaging pixel corresponding to the combination determined as not being the defective pixel, The determination unit obtains a comparison value between the projection coordinates corresponding to the first imaging pixel and the projection coordinates corresponding to the second imaging pixel in the combination of the first imaging pixel and the second imaging pixel determined by the relationship determination unit for each of the plurality of projection images in which the stripes extend in different directions, and determines whether at least one of the first imaging pixel or the second imaging pixel is a defective pixel by using an evaluation value based on the obtained comparison value.

3. A three-dimensional shape measuring device, characterized in that: have: a projection unit that projects a plurality of projection images of a pattern including stripes for determining projection coordinates, wherein the stripes included in the pattern extend in different directions to the object to be measured; a first imaging unit that generates a first captured image by imaging the projection image projected onto the measurement object; a second imaging unit that generates a second captured image by imaging the projection image projected onto the measurement object; a first coordinate determination unit for determining, based on the pattern included in the first captured image, the projection coordinate corresponding to the first captured pixel of the first captured image for each of the plurality of projection images in which the stripes extend in different directions; a second coordinate determining unit, which determines the projection coordinate corresponding to the second captured pixel of the second captured image based on the pattern included in the second captured image; a line determining unit, which determines a first epipolar line of the second captured image corresponding to the first captured pixel based on the configuration of the first captured unit and the second captured unit; a relationship determining unit, which determines a combination of the first photographing pixel and the second photographing pixel corresponding to the same projection coordinates as the projection coordinates corresponding to the first photographing pixel and located on the first epipolar line of the second photographing image corresponding to the first photographing pixel; a determination unit that determines, for each combination of the first photographing pixel and the second photographing pixel, whether at least one of the first photographing pixel or the second photographing pixel is a defective pixel based on a distance between a projection pixel of the projection image having the projection coordinates corresponding to the first photographing pixel and a projection pixel of the projection image having the projection coordinates corresponding to the second photographing pixel; a shape measuring unit configured to measure the shape of the measurement object using at least one of the first imaging pixel and the second imaging pixel corresponding to the combination determined as not being the defective pixel; an acquisition unit configured to acquire the combination of the first photographing pixel and the second photographing pixel determined by the relationship determination unit, and acquire, from the acquired combination, a plurality of the second photographing pixels within a predetermined range from the second photographing pixel as corresponding candidate pixels; a candidate coordinate determination unit for determining the projection coordinates corresponding to the plurality of corresponding candidate pixels for each of the projection images having stripes extending in different directions; as well as an evaluation unit that obtains, for each of the projection images having different extending directions of the fringes, a comparison value between the projection coordinates corresponding to the first captured pixel and the projection coordinates corresponding to the corresponding candidate pixel, and obtains, for each of the corresponding candidate pixels, an evaluation value based on a plurality of the comparison values ​​obtained for each of the projection images, The relationship specifying unit specifies, based on the evaluation value obtained by the evaluating unit, a combination of any one of the plurality of corresponding candidate pixels and the first imaging pixel acquired by the acquiring unit as a new combination.

4. A three-dimensional shape measuring device, characterized in that: have: a projection unit that projects a projection image including a pattern for determining projection coordinates onto a measurement object; a first imaging unit that generates a first captured image by imaging the projection image projected onto the measurement object; a second imaging unit that generates a second captured image by imaging the projection image projected onto the measurement object; a first coordinate determining unit, which determines the projection coordinate corresponding to the first captured pixel of the first captured image based on the pattern included in the first captured image; a second coordinate determining unit, which determines the projection coordinate corresponding to the second captured pixel of the second captured image based on the pattern included in the second captured image; a line determining unit, which determines a first epipolar line of the second captured image corresponding to the first captured pixel based on the configuration of the first captured unit and the second captured unit; a relationship determining unit, which determines a combination of the first photographing pixel and the second photographing pixel corresponding to the same projection coordinates as the projection coordinates corresponding to the first photographing pixel and located on the first epipolar line of the second photographing image corresponding to the first photographing pixel; a determination unit that determines, for each combination of the first photographing pixel and the second photographing pixel, whether at least one of the first photographing pixel or the second photographing pixel is a defective pixel based on a distance between a projection pixel of the projection image having the projection coordinates corresponding to the first photographing pixel and a projection pixel of the projection image having the projection coordinates corresponding to the second photographing pixel; as well as a shape measuring unit that measures the shape of the measurement object using at least one of the first imaging pixel and the second imaging pixel corresponding to the combination of the first imaging pixel and the second imaging pixel determined to be not the defective pixel, wherein the relationship specifying unit specifies a plurality of the combinations between the plurality of the first imaging pixels included in the first imaging image and the plurality of the second imaging pixels included in the second imaging image, The determination unit obtains an evaluation value indicating a distance between a projection pixel of the projection image having the projection coordinates corresponding to the first imaging pixel and a projection pixel of the projection image having the projection coordinates corresponding to the second imaging pixel in the combination of the first imaging pixel and the second imaging pixel determined by the relationship determination unit, obtains the evaluation value for each of the plurality of combinations, and determines that a malfunction has occurred in the configuration of any one of the first imaging unit, the second imaging unit, and the projection unit when a statistic of the obtained plurality of evaluation values ​​exceeds a reference value.

5. A three-dimensional shape measuring device, characterized in that: have: a projection unit that projects a projection image including a pattern for determining projection coordinates onto a measurement object; a first imaging unit that generates a first captured image by imaging the projection image projected onto the measurement object; a second imaging unit that generates a second captured image by imaging the projection image projected onto the measurement object; a first coordinate determining unit, which determines the projection coordinate corresponding to the first captured pixel of the first captured image based on the pattern included in the first captured image; a second coordinate determining unit, which determines the projection coordinate corresponding to the second captured pixel of the second captured image based on the pattern included in the second captured image; a line determination unit, which determines an epipolar line of the projection image corresponding to the first imaging pixel based on the configuration of the first imaging unit and the projection unit, and determines an epipolar line of the second imaging image corresponding to the projection pixel of the projection image based on the configuration of the projection unit and the second imaging unit; a relationship determining unit, which determines a combination of the first photographing pixel and a projection pixel of the projection image having the projection coordinates corresponding to the first photographing pixel and located on the epipolar line of the projection image corresponding to the first photographing pixel; a determination unit for determining, for each combination of the first photographic pixel and the projection pixel, whether at least one of the first photographic pixel, the main second photographic pixel, or the sub-second photographic pixel is a defective pixel based on a distance between a second photographic pixel (i.e., a main second photographic pixel) corresponding to the projection coordinates identical to the projection coordinates corresponding to the first photographic pixel and a second photographic pixel (i.e., a sub-second photographic pixel) corresponding to the projection coordinates of the projection pixel and located on an epipolar line of the second photographic image corresponding to the projection coordinates; as well as A shape measuring unit measures the shape of the measurement object using at least any one of the first imaging pixel, the main second imaging pixel, or the sub-second imaging pixel corresponding to the combination determined as not being a defective pixel.

6. The three-dimensional shape measuring device according to any one of claims 1, 2 or 5, characterized in that: The projection unit projects the projection image of the pattern including binary stripes and the projection image of the pattern including stripes having a sinusoidal brightness distribution onto the measurement object.

7. The three-dimensional shape measuring device according to claim 1 or 2, characterized in that: The projection unit sequentially projects the plurality of projection images including the pattern of fringes having a sinusoidal luminance distribution with fringes having different periods from each other.

8. The three-dimensional shape measuring device according to any one of claims 1, 2 or 5, characterized in that: The projection unit re-projects, onto the measurement object, a projection image in which all of the projection pixels having the projection coordinates corresponding to any one of the first imaging pixel and the second imaging pixel determined as a defective pixel by the determination unit are removed from among a plurality of pixels included in the projection image.

9. The three-dimensional shape measuring device according to any one of claims 1, 2 or 5, characterized in that: The projection unit re-projects, onto the measurement object, a projection image including only the projection pixels having the projection coordinates corresponding to the first imaging pixel or the second imaging pixel determined as a defective pixel by the determination unit, among the plurality of projection pixels included in the projection image.

10. A three-dimensional shape measurement method, characterized in that: The following steps are required: projecting, by a projection unit, a projection image of a pattern for determining projection coordinates and a plurality of projection images of the pattern including stripes extending in the same direction onto a measurement object; Generate a first captured image by capturing the projection image projected onto the measurement object using a first capturing unit; generating a second captured image by capturing the projection image projected onto the measurement object using a second capturing unit; Determining the projection coordinates corresponding to the first captured pixel of the first captured image based on the pattern contained in the first captured image; Determining the projection coordinates corresponding to the second captured pixel of the second captured image based on the pattern contained in the second captured image; Determine a first epipolar line of the second captured image corresponding to the first captured pixel based on the configuration of the first captured unit and the second captured unit, determine a second epipolar line of the projection image corresponding to the second captured pixel based on the configuration of the second captured unit and the projection unit, and determine a third epipolar line of the projection image corresponding to the first captured pixel based on the configuration of the first captured unit and the projection unit; determining a combination of the first photographing pixel and the second photographing pixel corresponding to the same projection coordinates as the projection coordinates corresponding to the first photographing pixel and located on the first epipolar line of the second photographing image corresponding to the first photographing pixel; for each combination of the first photographing pixel and the second photographing pixel, determining whether at least one of the first photographing pixel or the second photographing pixel is a defective pixel based on a distance between a projection pixel of the projection image having the projection coordinates corresponding to the first photographing pixel and a projection pixel of the projection image having the projection coordinates corresponding to the second photographing pixel; as well as measuring the shape of the measurement object using at least one of the first imaging pixel or the second imaging pixel corresponding to the combination of the first imaging pixel and the second imaging pixel determined not to be the defective pixel; In the step of determining whether it is the bad pixel, for each combination of the first photographing pixel and the second photographing pixel, based on a distance between the projection pixel having the projection coordinates corresponding to the first photographing pixel and located on the third epipolar line of the projection image corresponding to the first photographing pixel and the projection pixel located at an intersection of the second epipolar line and the third epipolar line, it is determined whether at least one of the first photographing pixel or the second photographing pixel is a bad pixel.

11. A three-dimensional shape measuring method, characterized in that: The following steps are required: projecting a plurality of projection images of a pattern including fringes for determining projection coordinates, wherein the fringes included in the pattern extend in different directions from each other, onto the object to be measured; Generate a first captured image by capturing the projection image projected onto the measurement object using a first capturing unit; generating a second captured image by capturing the projection image projected onto the measurement object using a second capturing unit; Based on the pattern included in the first captured image, determining the projection coordinates corresponding to the first captured pixel of the first captured image for each of the plurality of projection images in which the stripes extend in different directions; Based on the pattern included in the second captured image, determining the projection coordinates corresponding to the second captured pixel of the second captured image for each of the plurality of projection images in which the stripes extend in different directions; determining an epipolar line of the second captured image corresponding to the first captured pixel based on the configuration of the first captured unit and the second captured unit; determining a combination of the first photographing pixel and the second photographing pixel corresponding to the same projection coordinates as the projection coordinates corresponding to the first photographing pixel and located on the epipolar line of the second photographing image corresponding to the first photographing pixel; for each combination of the first photographing pixel and the second photographing pixel, determining whether at least one of the first photographing pixel or the second photographing pixel is a defective pixel based on a distance between a projection pixel of the projection image having the projection coordinates corresponding to the first photographing pixel and a projection pixel of the projection image having the projection coordinates corresponding to the second photographing pixel; as well as measuring the shape of the measurement object using at least one of the first imaging pixel or the second imaging pixel corresponding to the combination of the first imaging pixel and the second imaging pixel determined not to be the defective pixel; In the step of determining whether it is the bad pixel, in the determined combination of the first shooting pixel and the second shooting pixel, for each of the plurality of projection images in which the stripes extend in different directions, a comparison value between the projection coordinates corresponding to the first shooting pixel and the projection coordinates corresponding to the second shooting pixel is obtained, and whether at least one of the first shooting pixel or the second shooting pixel is a bad pixel is determined by an evaluation value based on the obtained comparison value.

12. A three-dimensional shape measuring method, characterized in that: The following steps are required: projecting a plurality of projection images of a pattern including fringes for determining projection coordinates, wherein the fringes included in the pattern extend in different directions from each other, onto the object to be measured; Generate a first captured image by capturing the projection image projected onto the measurement object using a first capturing unit; generating a second captured image by capturing the projection image projected onto the measurement object using a second capturing unit; Based on the pattern included in the first captured image, determining the projection coordinates corresponding to the first captured pixel of the first captured image for each of the plurality of projection images in which the stripes extend in different directions; Determining the projection coordinates corresponding to the second captured pixel of the second captured image based on the pattern contained in the second captured image; determining an epipolar line of the second captured image corresponding to the first captured pixel based on the configuration of the first captured unit and the second captured unit; determining a combination of the first photographing pixel and the second photographing pixel corresponding to the same projection coordinates as the projection coordinates corresponding to the first photographing pixel and located on the epipolar line of the second photographing image corresponding to the first photographing pixel; for each combination of the first photographing pixel and the second photographing pixel, determining whether at least one of the first photographing pixel or the second photographing pixel is a defective pixel based on a distance between a projection pixel of the projection image having the projection coordinates corresponding to the first photographing pixel and a projection pixel of the projection image having the projection coordinates corresponding to the second photographing pixel; measuring the shape of the measurement object using at least one of the first imaging pixel or the second imaging pixel corresponding to the combination of the first imaging pixel and the second imaging pixel determined not to be the defective pixel; acquiring the determined combination of the first photographing pixel and the second photographing pixel, and acquiring, in the acquired combination, a plurality of the second photographing pixels within a predetermined range from the second photographing pixel as corresponding candidate pixels; For each of the projection images in which the stripes extend in different directions, respectively determining the projection coordinates corresponding to the plurality of corresponding candidate pixels; as well as for each of the projection images having stripes extending in different directions, a comparison value between the projection coordinates corresponding to the first captured pixel and the projection coordinates corresponding to the corresponding candidate pixel is obtained, and for each of the corresponding candidate pixels, an evaluation value based on a plurality of the comparison values ​​obtained for each of the projection images is obtained, In the step of determining the combination of the first imaging pixel and the second imaging pixel, based on the obtained evaluation value, any one of the plurality of corresponding candidate pixels and the first imaging pixel in the obtained combination are determined as a new combination.

13. A three-dimensional shape measurement method, characterized in that: The following steps are required: projecting a projection image including a pattern for determining projection coordinates onto the measurement object by a projection unit; Generate a first captured image by capturing the projection image projected onto the measurement object using a first capturing unit; generating a second captured image by capturing the projection image projected onto the measurement object using a second capturing unit; Determining the projection coordinates corresponding to the first captured pixel of the first captured image based on the pattern contained in the first captured image; Determining the projection coordinates corresponding to the second captured pixel of the second captured image based on the pattern contained in the second captured image; determining an epipolar line of the second captured image corresponding to the first captured pixel based on the configuration of the first captured unit and the second captured unit; determining a combination of the first photographing pixel and the second photographing pixel corresponding to the same projection coordinates as the projection coordinates corresponding to the first photographing pixel and located on the epipolar line of the second photographing image corresponding to the first photographing pixel; for each combination of the first photographing pixel and the second photographing pixel, determining whether at least one of the first photographing pixel or the second photographing pixel is a defective pixel based on a distance between a projection pixel of the projection image having the projection coordinates corresponding to the first photographing pixel and a projection pixel of the projection image having the projection coordinates corresponding to the second photographing pixel; as well as measuring the shape of the measurement object using at least one of the first imaging pixel or the second imaging pixel corresponding to the combination of the first imaging pixel and the second imaging pixel determined not to be the defective pixel; In the step of determining a combination of the first photographing pixels and the second photographing pixels, a plurality of the combinations are determined between a plurality of the first photographing pixels included in the first photographing image and a plurality of the second photographing pixels included in the second photographing image. In the step of determining whether it is the bad pixel, in the determined combination of the first photographing pixel and the second photographing image, an evaluation value representing the distance between a projection pixel of the projection image having the projection coordinates corresponding to the first photographing pixel and a projection pixel of the projection image having the projection coordinates corresponding to the second photographing pixel is obtained, and the evaluation value is obtained for each of the plurality of combinations, and when a statistic of the obtained plurality of evaluation values ​​exceeds a reference value, it is determined that a bad situation has occurred in the configuration of any one of the first photographing unit, the second photographing unit, and the projection unit.

14. A three-dimensional shape measurement method, characterized in that: The following steps are required: projecting a projection image including a pattern for determining projection coordinates onto the measurement object by a projection unit; Generate a first captured image by capturing the projection image projected onto the measurement object using a first capturing unit; generating a second captured image by capturing the projection image projected onto the measurement object using a second capturing unit; Determining the projection coordinates corresponding to the first captured pixel of the first captured image based on the pattern contained in the first captured image; Determining the projection coordinates corresponding to the second captured pixel of the second captured image based on the pattern contained in the second captured image; Determine, based on the configuration of the first imaging unit and the projection unit, an epipolar line of the projection image corresponding to the first imaging pixel, and determine, based on the configuration of the projection unit and the second imaging unit, an epipolar line of the second imaging image corresponding to the projection pixel of the projection image; determining a combination of the first photographing pixel and a projection pixel of the projection image having the projection coordinates corresponding to the first photographing pixel and located on the epipolar line of the projection image corresponding to the first photographing pixel; for each combination of the first photographing pixel and the projection pixel, determining whether at least one of the first photographing pixel, the main second photographing pixel, or the sub-second photographing pixel is a defective pixel based on a distance between a second photographing pixel corresponding to the projection coordinates identical to the projection coordinates corresponding to the first photographing pixel, namely a main second photographing pixel, and a second photographing pixel corresponding to the projection coordinates of the projection pixel and located on an epipolar line of the second photographing image corresponding to the projection coordinates, namely a sub-second photographing pixel; as well as The shape of the object to be measured is measured using at least one of the first pixel, the main second pixel, or the sub-second pixel corresponding to the combination of the first pixel, the main second pixel, and the sub-second pixel determined not to be a defective pixel.

Images