Three-dimensional shape measurement device, measurement method, and measurement program

Abstract

To improve measurement accuracy of a three-dimensional shape in a three-dimensional shape measurement device using a phase shift method.SOLUTION: A three-dimensional shape measuring device 100 comprises: a projector 1 that includes a projector lens 131 and projects a stripe pattern; a camera 2 that includes a camera lens 221 and picks up the stripe pattern projected by the projector 1 to acquire a picked-up image of the stripe pattern; a measurement part that measures a three-dimensional shape of a measurement target 300 on the basis of the picked-up image; and a determination part that determines a period of the stripe pattern projected by the projector 1 from among a plurality of period candidates. The plurality of period candidates are set on the basis of composite MTF characteristics generated by synthesizing a first MTF characteristic of the projector lens 131 and a second MTF characteristic of the camera lens 221.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to a three-dimensional shape measurement device, a measurement method, and a measurement program.

Background Art

[0002] Conventionally, a device for measuring a three-dimensional shape using the principle of the phase shift method (hereinafter referred to as a "three-dimensional shape measurement device") has been developed. For example, three-dimensional shape measurement devices are disclosed in Japanese Patent Application Laid-Open No. 2019-191310 (Patent Document 1), Japanese Patent Application Laid-Open No. 2018-146521 (Patent Document 2), Japanese Patent Application Laid-Open No. 2013-88261 (Patent Document 3), and Japanese Patent Application Laid-Open No. 2016-31284 (Patent Document 4). A three-dimensional shape measurement device using the phase shift method includes a projector that projects a stripe pattern onto a measurement object and a camera that images the measurement object onto which the stripe pattern is projected.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a three-dimensional shape measurement device using the phase shift method, the contrast of a captured image obtained by imaging a stripe pattern with a camera changes depending on the period of the stripe pattern projected onto the measurement object. When the contrast of the captured image decreases, the variation in the height position of the output three-dimensional shape increases, and the measurement accuracy of the three-dimensional shape decreases.

[0005] One object of the present disclosure is to improve the measurement accuracy of a three-dimensional shape in a three-dimensional shape measurement apparatus using a phase shift method.

Means for Solving the Problems

[0006] A three-dimensional shape measurement apparatus according to an aspect of the present disclosure includes a projector lens and includes a projector that projects a stripe pattern. The three-dimensional shape measurement apparatus includes a camera lens and includes a camera that captures the stripe pattern projected by the projector to obtain a captured image of the stripe pattern. The three-dimensional shape measurement apparatus includes a measurement unit that measures the three-dimensional shape of a measurement object based on the captured image. The three-dimensional shape measurement apparatus includes a determination unit that determines the period of the stripe pattern from a plurality of period candidates. The plurality of period candidates are set based on a combined MTF (Modulation Transfer Function) characteristic generated by combining a first MTF characteristic of the projector lens and a second MTF characteristic of the camera lens.

[0007] Preferably, the three-dimensional shape measurement apparatus further includes a storage unit that stores a table associating each of the plurality of period candidates with the distance between the projector and the measurement object. The combined MTF characteristic is generated for each of a plurality of distances between the projector and the measurement object. As each of the plurality of period candidates, a period at which the contrast in the combined MTF characteristic for the distance corresponding to the period candidate is equal to or greater than a predetermined value is set. The determination unit determines the period of the stripe pattern based on the period candidate corresponding to the measured distance between the projector and the measurement object among the plurality of period candidates.

[0008] Preferably, the projector further includes an image display element. The camera further includes a camera sensor. The first MTF characteristic shows the correspondence between the spatial frequency on the image display element and the first contrast representing the contrast performance of the projector lens. The second MTF characteristic shows the correspondence between the spatial frequency on the camera sensor and the second contrast representing the contrast performance of the camera lens. The combined MTF characteristic shows the correspondence between the spatial frequency on the image display element and the third contrast. The third contrast corresponding to the first spatial frequency on the image display element is calculated based on the first contrast corresponding to the first spatial frequency on the image display element and the second contrast corresponding to the second spatial frequency on the camera sensor. The first spatial frequency on the image display element and the second spatial frequency on the camera sensor have the same spatial frequency on the measurement surface of the three-dimensional shape measurement device.

[0009] Preferably, the three-dimensional shape measurement device further includes a storage unit that stores a table associating each of a plurality of period candidates with a combination of the distance between the projector and the measurement target, the F-number of the projector lens, and the F-number of the camera lens. The combined MTF characteristic is generated for each of a plurality of combinations of the distance between the projector and the measurement target, the F-number of the projector lens, and the F-number of the camera lens. As each of the plurality of period candidates, a period is set in which the contrast becomes a predetermined value or more in the combined MTF characteristic for the combination corresponding to the period candidate. The determination unit determines the period of the stripe pattern based on the period candidate corresponding to the combination among the plurality of period candidates.

[0010] Preferably, the first MTF characteristic corresponds to the first position farthest from the first optical axis of the projector lens within the measurement region of the three-dimensional shape measurement device. The second MTF characteristic corresponds to the second position farthest from the second optical axis of the camera lens within the measurement region.

[0011] Preferably, the first MTF characteristic corresponds to the measurement upper surface in the measurement region in the first state where the focus of the projector lens is adjusted to the measurement bottom surface in the measurement region of the three-dimensional shape measuring device. The second MTF characteristic corresponds to the measurement upper surface in the second state where the focus of the camera lens is adjusted to the measurement bottom surface.

[0012] Preferably, the three-dimensional shape measuring device further includes a distance measuring device that measures the measurement distance between the projector and the measurement object. The determination unit determines the period of the stripe pattern from a plurality of period candidates according to the measurement distance measured by the distance measuring device.

[0013] A method for measuring the three-dimensional shape of a measurement object according to another aspect of the present disclosure includes determining the period of a stripe pattern projected by a projector including a projector lens from a plurality of period candidates. The measurement method includes projecting a stripe pattern by the projector. The measurement method includes imaging the stripe pattern projected by the projector with a camera including a camera lens to obtain a captured image of the stripe pattern. The measurement method includes measuring the three-dimensional shape of the measurement object based on the captured image. The plurality of period candidates are set based on a combined MTF characteristic generated by combining a first MTF (Modulation Transfer Function) characteristic of the projector lens and a second MTF characteristic of the camera lens.

[0014] A measurement program according to another aspect of the present disclosure causes one or more computers to execute the above measurement method.

Advantages of the Invention

[0015] According to the present disclosure, in a three-dimensional shape measuring device using a phase shift method, the measurement accuracy of the three-dimensional shape can be improved.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Embodiments for Carrying Out the Invention

[0017] Hereinafter, embodiments and modifications according to the present disclosure will be described with reference to the drawings. In the following description, the same parts and components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated. Note that the embodiments and modifications described below may be selectively combined as appropriate.

[0018] <Application Examples of Three-Dimensional Shape Measurement Devices> FIG. 1 is a diagram schematically showing an example of a system to which the three-dimensional shape measurement apparatus according to the embodiment is applied. The system 1000 includes a three-dimensional shape measurement apparatus 100 and a robot 200.

[0019] The three-dimensional shape measurement apparatus 100 measures the three-dimensional shape of the measurement object 300 using the principle of the phase shift method. The measurement object 300 includes a tray 302 and a plurality of workpieces 304 randomly stacked on the tray 302. The three-dimensional shape measurement apparatus 100 outputs data indicating the measured three-dimensional shape (hereinafter referred to as "three-dimensional point cloud data") to the robot 200.

[0020] Based on the three-dimensional point cloud data received from the three-dimensional shape measurement apparatus 100, the robot 200 determines the workpiece 304 to be gripped and performs a pick-and-place operation on the workpiece 304 to be gripped. Specifically, the robot 200 grips the workpiece 304 on the tray 302 and places the workpiece 304 on the conveyor belt 500.

[0021] Note that the system to which the three-dimensional shape measurement apparatus 100 is applied is not limited to the system 1000 shown in FIG. 1. The three-dimensional shape measurement apparatus 100 is used for various applications.

[0022] <Configuration of the three-dimensional shape measurement apparatus 100> FIG. 2 is a diagram showing an example of the hardware configuration of the three-dimensional shape measurement apparatus 100. As shown in FIG. 2, the three-dimensional shape measurement apparatus 100 includes a projector 1, a camera 2, a computer 3, and a distance measuring device 4.

[0023] The projector 1 includes a light source 11, a light modulation device 12, and a projection optical system 13. The light beam emitted from the light source 11 is modulated by the light modulation device 12 and projected onto the measurement object 300 through the projection optical system 13.

[0024] The light source 11 includes, for example, an LED (Light Emitting Diode), an ultra-high pressure mercury lamp, or a halogen lamp. For example, the light source 11 emits white light. Note that the light source 11 may emit red, blue, or green monochromatic light.

[0025] The light modulation device 12 modulates the light from the light source 11 to generate a projection image. The light modulation device 12 includes an image display element 121. The image display element 121 is, for example, a Digital Micromirror Device (DMD) or a liquid crystal panel. In the present embodiment, the light modulation device 12 can generate a stripe pattern as a projection image. Here, with reference to FIG. 3, the stripe pattern will be described.

[0026] FIG. 3 is a diagram for explaining a stripe pattern and the period of the stripe pattern. At the upper part of FIG. 3, a stripe pattern PA is shown as an example of the stripe pattern. At the lower part of FIG. 3, a graph is shown in which the horizontal axis is the position and the vertical axis is the luminance value. As shown in FIG. 3, the stripe pattern is a pattern in which black lines and white lines are alternately repeated. More specifically, in the stripe pattern, the luminance value changes in a sine wave shape according to the position. The light modulation device 12 changes the period L of the stripe pattern according to the control from the computer 3. That is, the period L of the stripe pattern generated by the light modulation device 12 is variable.

[0027] Referring again to FIG. 2, the projection optical system 13 projects the projection image (that is, the stripe pattern) generated by the light modulation device 12 onto the measurement target 300. The projection optical system 13 has one or more lenses. In the example shown in FIG. 2, the projection optical system 13 has a projector lens 131. The stripe pattern generated by the light modulation device 12 is enlarged by the projector lens 131 and projected onto the measurement target 300. As a result, an enlarged stripe pattern is formed on the measurement surface 305 of the three-dimensional shape measurement device 100. That is, the period L is different between the stripe pattern on the image display element 121 and the stripe pattern on the measurement surface 305.

[0028] The camera 2 images a subject and generates image data (hereinafter referred to as "captured image"). As an example, the camera 2 images the stripe pattern projected onto the measurement target 300 by the projector 1 to obtain a captured image of the stripe pattern. The camera 2 includes a camera sensor 21 and a condensing optical system 22.

[0029] The camera sensor 21 converts the intensity of the light obtained through the condensing optical system 22 into an electrical signal. The camera sensor 21 includes, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor).

[0030] The condensing optical system 22 is an optical system for condensing the light incident from the outside and typically has one or more lenses. In the example shown in FIG. 2, the condensing optical system 22 has a camera lens 221.

[0031] The period L of the stripe pattern in the captured image, that is, the stripe pattern on the camera sensor 21 and the stripe pattern on the measurement surface 305, is different.

[0032] The distance measuring device 4 is arranged at the same height as the projector 1. The distance measuring device 4 measures the distance d1 between the projector 1 and the measurement target 300. Hereinafter, the distance d1 measured by the distance measuring device 4 is referred to as the "measured distance". The distance measuring device 4 is constituted by, for example, a millimeter wave sensor that can measure the distance in a short time. However, the distance measuring device 4 is not limited thereto and may be constituted by a known distance measuring sensor.

[0033] The distance measuring device 4 may measure, for example, the distance to a partial range 308 of the measurement target 300. The partial range 308 has, for example, a length corresponding to one period of the stripe pattern on the measurement surface 305.

[0034] The user adjusts the focus of the projector lens 131 and the focus of the camera lens 221 according to the distance d1 between the projector 1 and the measurement target 300. Note that the projector 1 may be provided with an autofocus mechanism that automatically focuses the projector lens 131 on the measurement target 300. Also, the camera 2 may be provided with an autofocus mechanism that automatically focuses the camera lens 221 on the measurement target 300.

[0035] The computer 3 controls the projector 1, the camera 2, and the distance measuring device 4. The computer 3 includes a processor 31, a memory 32, and a storage 33.

[0036] The processor 31 is composed of, for example, a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit). The memory 32 is composed of a volatile storage device such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory). The storage 33 is composed of a non-volatile storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory.

[0037] The storage 33 is an example of the "storage unit" in the present disclosure. The storage 33 stores a program 34. The program 34 includes one or more computer-readable instructions for controlling the projector 1, the camera 2, and the distance measuring device 4. The program 34 includes the "measurement program" in the present disclosure. The processor 31 controls the projector 1, the camera 2, and the distance measuring device 4 by executing the program 34, and realizes various processes according to the present embodiment.

[0038] Program 34 may be provided not as a single program but as incorporated into a part of any program. In this case, processing according to the present embodiment is realized in cooperation with any program. Even a program that does not include such a part of the module does not deviate from the gist of the three-dimensional shape measurement device 100 according to the present embodiment. Further, part or all of the functions provided by program 34 may be realized by dedicated hardware.

[0039] Storage 33 further stores table 35. Table 35 is used to control the period L of the stripe pattern projected by projector 1.

[0040] <Phase shift method> Processor 31 measures the three-dimensional shape of measurement object 300 using the principle of the phase shift method. Specifically, processor 31 controls projector 1 to project a stripe pattern while shifting the phase. Processor 31 acquires a plurality of captured images corresponding to a plurality of phase shift amounts of the stripe pattern from camera 2. For example, processor 31 acquires four captured images when the phase shift amounts are 0, π / 2, π, and 3π / 2.

[0041] The luminance value of each pixel in the plurality of captured images is represented by the following formula (1). Ii = acos(φ + δi) + b Formula (1)

[0042] Ii represents the luminance value in the i-th captured image. a represents the contrast component. b represents the offset component. δi represents the phase shift amount corresponding to the i-th captured image. φ represents the phase.

[0043] In formula (1), the unknowns are a, b, and φ. Therefore, processor 31 can calculate a, b, and φ for each pixel by acquiring at least three captured images.

[0044] FIG. 4 is a diagram showing the luminance value and the phase shift amount of a certain pixel in four captured images corresponding to phase shift amounts of 0, π / 2, π, and 3π / 2. The processor 31 calculates the phase φ by identifying a cosine curve 50 that approximates the luminance values and the phase shift amounts of the four captured images. Specifically, if the luminance value in the captured image corresponding to the phase shift amount of 0 is I0, the luminance value in the captured image corresponding to the phase shift amount of π / 2 is I1, the luminance value in the captured image corresponding to the phase shift amount of π is I2, and the luminance value in the captured image corresponding to the phase shift amount of 3π / 2 is I3, the phase φ is calculated according to the following formula (2).

[0045] tanφ = -(I3 - I1) / (I2 - I0) Formula (2)

[0046] The phase φ and the height h of the measurement object 300 imaged on the pixel satisfy the following formula (3). h = (p / sinα)×(φ + 2nπ) / 2π Formula (3)

[0047] p represents the pitch of the stripe pattern. α represents the projection angle of the stripe pattern. n represents the order. The processor 31 converts the phase φ into the height h according to Formula (3) and generates three-dimensional point cloud data indicating the height h for each pixel.

[0048] The luminance value of each pixel shown in the captured image may include errors due to the influence of noise of the camera 2 and linearity errors of the projector 1, etc. Hereinafter, the error included in the luminance value is referred to as "luminance error".

[0049] FIG. 5 is a diagram for explaining the influence of luminance error on three-dimensional shape measurement. In FIG. 5, the curve 52 is a cosine curve that approximates the luminance values and the phase shift amounts in a plurality of captured images. The curve 52 is identified using luminance values including luminance error. In FIG. 5, the curve 54 is a cosine curve that approximates the ideal luminance values and the phase shift amounts without luminance error. As shown in FIG. 5, due to the influence of the luminance error, a phase error Δφ occurs between the curve 52 and the curve 54. That is, the phase φ calculated from the curve 52 includes the phase error Δφ.

[0050] Due to the influence of the phase error Δφ, an error also occurs in the height h converted from the phase φ. In the example shown in FIG. 5, since a phase error Δφ of 350° is included, the height h converted from the phase φ includes an error of 10 mm. Thus, due to the influence of the luminance error, an error also occurs in the height h.

[0051] When the amplitude of the luminance value when changing the phase shift amount becomes small, the influence of the luminance error becomes relatively large. As a result, the phase error Δφ becomes large, and the error included in the height h converted from the phase φ also becomes large. Hereinafter, the error of the height h is also referred to as "measurement error".

[0052] The amplitude of the luminance value when changing the phase shift amount correlates with the contrast of the stripe pattern in the captured image. When the contrast of the stripe pattern in the captured image decreases, the amplitude of the luminance value when changing the phase shift amount becomes small. Therefore, when the contrast of the stripe pattern in the captured image decreases, the error included in the height h converted from the phase φ becomes large.

[0053] FIG. 6 is a diagram for explaining the relationship between the MTF characteristics of the lens and the conversion accuracy from the phase φ to the height h. The MTF (Modulation Transfer Function) characteristics are an index for evaluating the performance of the lens. The MTF characteristics vary depending on the lens. The MTF characteristics show the correspondence relationship between the spatial frequency and the contrast performance of the lens. The contrast performance of the lens is represented by the contrast of the image of the stripe pattern formed by the lens. The spatial frequency represents the number of line pairs (one black line and one white line in the stripe pattern) per millimeter. The unit of the spatial frequency is lp / mm. The higher the spatial frequency, the shorter the period of the stripe pattern.

[0054] Line 61 represents the MTF characteristic of lens a. The MTF characteristic of lens a shows the correspondence between the spatial frequency and the contrast of the image of the stripe pattern formed by lens a. Line 62 represents the MTF characteristic of lens b, which is different from lens a. The MTF characteristic of lens b shows the correspondence between the spatial frequency and the contrast of the image of the stripe pattern formed by lens b. Line 63 shows the correspondence between the spatial frequency and the conversion accuracy from phase φ to height h.

[0055] As shown in FIG. 6, generally, the higher the spatial frequency, the lower the contrast. In other words, the shorter the period of the stripe pattern, the lower the contrast. Also, the higher the spatial frequency, the higher the conversion accuracy from phase φ to height h. In other words, the shorter the period of the stripe pattern, the higher the conversion accuracy from phase φ to height h. This is because there is a coefficient of the pitch p of the stripe pattern on the right side of the above formula (3) used for the conversion from phase φ to height h.

[0056] From the characteristics shown in FIG. 6, the decrease in contrast is suppressed by increasing the period of the stripe pattern. However, if the period of the stripe pattern is increased too much, the conversion accuracy from phase φ to height h decreases, and as a result, the measurement accuracy of the three-dimensional shape decreases.

[0057] Therefore, in the three-dimensional shape measurement device 100 according to the present embodiment, candidates for the period of the stripe pattern are set in consideration of the contrast and the conversion accuracy from phase φ to height h.

[0058] <Processing in the three-dimensional shape measurement device 100> (Outline of processing in the three-dimensional shape measurement device 100) FIG. 7 is a diagram for explaining the outline of the processing in the three-dimensional shape measurement device 100. The three-dimensional shape measurement device 100 includes a projector 1, a camera 2, a computer 3, and a distance measuring device 4. The computer 3 includes a determination unit 351, a measurement unit 352, and a storage 33. The determination unit 351 and the measurement unit 352 are realized by a processor 31 that executes a program 34.

[0059] The distance measuring device 4 measures the distance d1 (i.e., the measurement distance) between the projector 1 and the measurement target 300. The determination unit 351 determines the period L of the stripe pattern projected by the projector 1 onto the measurement target 300 from a plurality of period candidates according to the measurement distance measured by the distance measuring device 4. The plurality of period candidates are included in the table 35 stored in the storage 33.

[0060] The light modulation device 12 generates a stripe pattern having the period L determined by the determination unit 351. The projection optical system 13 magnifies and projects the stripe pattern generated by the light modulation device 12 onto the measurement target 300.

[0061] The camera 2 images the stripe pattern projected by the projector 1 and obtains a captured image of the stripe pattern. The measurement unit 352 measures the three-dimensional shape of the measurement target 300 based on the captured image using the principle of the phase shift method.

[0062] (Table) FIG. 8 is a diagram showing an example of the table 35. As shown in FIG. 8, the table 35 associates each of the plurality of period candidates with the distance d1 between the projector 1 and the measurement target 300. The period candidate represents the period L of the stripe pattern (for example, the stripe pattern on the image display element 121) generated in the light modulation device 12. The period candidate is preset so that the measurement error is minimized when measuring the three-dimensional shape of the measurement target 300 at a position separated from the projector 1 by the distance corresponding to the period candidate.

[0063] (Method for setting period candidate) With reference to FIG. 9, the method for setting the period candidate will be described. FIG. 9 is a diagram for explaining the method for setting the period candidate.

[0064] In the first step of the cycle candidate setting method, the MTF characteristics of the projector lens 131 and the MTF characteristics of the camera lens 221 corresponding to the distance d1 between the projector 1 and the measurement target 300 are obtained by experiments. Line 91a represents the MTF characteristics of the projector lens 131 when the distance d1 between the projector 1 and the measurement target 300 is 200 mm. Line 91b represents the MTF characteristics of the projector lens 131 when the distance d1 between the projector 1 and the measurement target 300 is 300 mm. Line 91c represents the MTF characteristics of the projector lens 131 when the distance d1 between the projector 1 and the measurement target 300 is 500 mm. Line 92a represents the MTF characteristics of the camera lens 221 when the distance d1 between the projector 1 and the measurement target 300 is 200 mm. Line 92b represents the MTF characteristics of the camera lens 221 when the distance d1 between the projector 1 and the measurement target 300 is 300 mm. Line 92c represents the MTF characteristics of the camera lens 221 when the distance d1 between the projector 1 and the measurement target 300 is 500 mm.

[0065] The MTF characteristics of the projector lens 131 are an example of the "first MTF characteristics" in the present disclosure. The MTF characteristics of the projector lens 131 show the correspondence between the spatial frequency on the image display element 121 (the period L of the stripe pattern on the image display element 121) and the first contrast representing the contrast performance of the projector lens 131.

[0066] The MTF characteristics of the camera lens 221 are an example of the "second MTF characteristics" in the present disclosure. The MTF characteristics of the camera lens 221 show the correspondence between the spatial frequency on the camera sensor 21 (the period L of the stripe pattern on the camera sensor 21) and the second contrast representing the contrast performance of the camera lens 221.

[0067] Among the multiple distances between the projector 1 and the measurement target 300, the MTF characteristics of the projector lens 131 for the target distance and the MTF characteristics of the camera lens 221 for the target distance are obtained by the following MTF characteristic acquisition method.

[0068] In the first step of the method for obtaining the MTF characteristic, a screen is installed at a position separated from the projector 1 by the target distance, and the focus of the projector lens 131 and the focus of the camera lens 221 are aligned on the surface of the screen.

[0069] In the second step of the method for obtaining the MTF characteristic, a plurality of stripe patterns with different periods L are sequentially projected onto the screen by the projector 1.

[0070] In the third step of the method for obtaining the MTF characteristic, for each stripe pattern projected onto the screen, the contrast in a predetermined region including the position on the optical axis k1 (see FIG. 2) of the projector lens 131 in the stripe pattern is measured. The predetermined region including the position on the optical axis k1 of the projector lens 131 in the stripe pattern projected onto the screen is, for example, the region G1 shown in FIG. 2. The optical axis k1 is an example of the "first optical axis" in the present disclosure.

[0071] In the fourth step of the method for obtaining the MTF characteristic, the MTF characteristic of the projector lens 131 is generated by associating the spatial frequency on the image display element 121 of the stripe pattern projected onto the screen with the contrast measured in the third step of the method for obtaining the MTF characteristic for the stripe pattern.

[0072] In the fifth step of the method for obtaining the MTF characteristic, a plurality of test charts are sequentially arranged on the screen and imaged by the camera 2. Each test chart has a stripe pattern drawn thereon. The plurality of test charts have different periods L of the stripe pattern from each other.

[0073] In the sixth step of the method for obtaining MTF characteristics, for each test chart, for a predetermined region including the position on the optical axis k2 (see FIG. 2) of the camera lens 221 in the test chart, the contrast on the camera sensor 21 is measured. The predetermined region including the position on the optical axis k2 of the camera lens 221 in the test chart is the region G1 shown in FIG. 2. The optical axis k2 is an example of the "second optical axis" in the present disclosure.

[0074] In the seventh step of the method for obtaining MTF characteristics, by associating the spatial frequency on the camera sensor 21 of the stripe pattern depicted on the test chart with the contrast measured in the sixth step of the method for obtaining MTF characteristics for the test chart, the MTF characteristics of the camera lens 221 are generated.

[0075] In this way, for all of the plurality of distances between the projector 1 and the measurement target 300, the MTF characteristics of the projector lens 131 and the MTF characteristics of the camera lens 221 are obtained.

[0076] In the second step of the method for setting cycle candidates, a combined MTF characteristic corresponding to the distance d1 between the projector 1 and the measurement target 300 is generated. Line 93a represents the combined MTF characteristic when the distance d1 between the projector 1 and the measurement target 300 is 200 mm. Line 93b represents the combined MTF characteristic when the distance d1 between the projector 1 and the measurement target 300 is 300 mm. Line 93c represents the combined MTF characteristic when the distance d1 between the projector 1 and the measurement target 300 is 500 mm.

[0077] The combined MTF characteristic is generated by combining the MTF characteristic of the projector lens 131 and the MTF characteristic of the camera lens 221. The combined MTF characteristic shows the correspondence between the spatial frequency on the image display element 121 (the period L of the stripe pattern on the image display element 121) and the third contrast. The third contrast is represented by the following formula (4).

[0078] R = Q1 / 100 × Q2 / 100 × 100 Equation (4)

[0079] R represents the third contrast (%) corresponding to the first spatial frequency P1 on the image display element 121, as indicated by the combined MTF characteristic. Q1 represents the first contrast (%) corresponding to the first spatial frequency P1 on the image display element 121, as indicated by the MTF characteristic of the projector lens 131. Q2 represents the second contrast (%) corresponding to the second spatial frequency P2 on the camera sensor 21, as indicated by the MTF characteristic of the camera lens 221. The first spatial frequency P1 on the image display element 121 and the second spatial frequency P2 on the camera sensor 21 have the same spatial frequency on the measurement surface 305 of the three-dimensional shape measuring device 100.

[0080] The first spatial frequency on the image display element 121 and the second spatial frequency on the camera sensor 21, which have the same spatial frequency on the measurement surface 305, are specified by the first conversion process and the second conversion process.

[0081] The first conversion process is a process of converting the first spatial frequency on the image display element 121 in the MTF characteristic of the projector lens 131 into the spatial frequency on the measurement surface 305 of the three-dimensional shape measuring device 100 based on the following three elements e1 to e3.

[0082] · Element e1: The distance between the projector 1 and the measurement target 300 · Element e2: The focal length of the projector lens 131 · Element e3: The inclination of the optical axis k1 of the projector lens 131

[0083] The second conversion process is a process of converting the second spatial frequency on the camera sensor 21 in the MTF characteristic of the camera lens 221 into the spatial frequency on the measurement surface 305 of the three-dimensional shape measuring device 100 based on the following three elements e4 to e6.

[0084] · Element e4: The distance between the camera 2 and the measurement target 300 · Element e5: The focal length of the camera lens 221 ·Element e6: Inclination of the optical axis k2 of the camera lens 221

[0085] Ra shown in FIG. 9 represents the third contrast (%) corresponding to the first spatial frequency P1a on the image display element 121, which is indicated by the combined MTF characteristics when the distance d1 between the projector 1 and the measurement object 300 is 200 mm. Q1a shown in FIG. 9 represents the first contrast (%) corresponding to the first spatial frequency P1a on the image display element 121, which is indicated by the MTF characteristics of the projector lens 131 when the distance d1 between the projector 1 and the measurement object 300 is 200 mm. Q2a shown in FIG. 9 represents the second contrast (%) corresponding to the second spatial frequency P2a on the camera sensor 21, which is indicated by the MTF characteristics of the camera lens 221 when the distance d1 between the projector 1 and the measurement object 300 is 200 mm. The first spatial frequency P1a on the image display element 121 and the second spatial frequency P2a on the camera sensor 21 have the same spatial frequency on the measurement surface 305 of the three-dimensional shape measurement device 100.

[0086] In this way, the combined MTF characteristics are generated for all of the plurality of distances between the projector 1 and the measurement object 300.

[0087] In the third step of the method for setting the period candidate, a period candidate is set for each of the plurality of distances between the projector 1 and the measurement object 300. More specifically, when a period candidate corresponding to the target distance among the plurality of distances between the projector 1 and the measurement object 300 is set, the shortest period among the period ranges of the stripe pattern in which the third contrast becomes a predetermined value or more in the combined MTF characteristics for the target distance is set.

[0088] (Processing procedure of the processor 31) FIG. 10 is a flowchart showing the procedure of the measurement process by the processor 31. When the processor 31 receives an instruction to start measurement, the measurement process is started.

[0089] In step S1, the processor 31 obtains the measured distance from the distance measuring device 4. In the next step S2, the processor 31 determines the period L of the stripe pattern projected by the projector 1 according to the measured distance.

[0090] Step S2 includes steps S21 and S22. In step S21, the processor 31 reads out a period candidate corresponding to the measured distance from the table 35. In the next step S22, the processor 31 determines the period L of the stripe pattern based on the read period candidate.

[0091] For example, when there is a distance in the table 35 whose difference from the measured distance is within a specified range, the processor 31 reads out the period candidate corresponding to the distance and determines the read period candidate as the period L of the stripe pattern. When there is no distance in the table 35 whose difference from the measured distance is within the specified range, the processor 31 reads out two period candidates corresponding to two distances with a small difference from the measured distance, and determines the period L of the stripe pattern by interpolation using the two read period candidates.

[0092] In the next step S3, the processor 31 instructs the projector 1 to project the stripe pattern having the period L determined in step S2 onto the measurement object 300. Thereby, the projector 1 projects the stripe pattern having the period L determined in step S2 onto the measurement object 300 after magnification.

[0093] In the next step S4, the processor 31 causes the camera 2 to image the stripe pattern projected onto the measurement object 300, and obtains a photographed image of the stripe pattern from the camera 2. In the next step S5, the processor 31 specifies the three-dimensional shape of the measurement object 300 based on the photographed image using the principle of the phase shift method. After step S5, the measurement process ends.

[0094] In this way, the three-dimensional shape measurement device 100 according to the present embodiment determines the period L of the stripe pattern projected by the projector 1 from a plurality of period candidates. The plurality of period candidates are set based on the combined MTF characteristic generated by combining the MTF characteristic of the projector lens 131 and the MTF characteristic of the camera lens 221. Therefore, the three-dimensional shape measurement device 100 according to the present embodiment can improve the measurement accuracy of the three-dimensional shape compared to the case where the period L of the stripe pattern is set using the MTF characteristic of the projector lens without using the MTF characteristic of the camera lens.

[0095] The third contrast corresponding to the first spatial frequency on the image display element 121 indicated by the combined MTF characteristic is obtained by multiplying the first contrast corresponding to the first spatial frequency on the image display element 121 indicated by the MTF characteristic of the projector lens 131 and the second contrast corresponding to the second spatial frequency on the camera sensor 21 indicated by the MTF characteristic of the camera lens 221. The first spatial frequency on the image display element 121 and the second spatial frequency on the camera sensor 21 have the same spatial frequency on the measurement surface 305 of the three-dimensional shape measurement device 100. Therefore, the error between the third contrast indicated by the combined MTF characteristic and the contrast of the stripe pattern on the camera sensor 21 in the actual measurement process is reduced. Therefore, the three-dimensional shape measurement device 100 according to the present embodiment can improve the measurement accuracy of the three-dimensional shape.

[0096] In addition, for the three-dimensional shape measurement device 100 according to the present embodiment, period candidates are set for all of the plurality of distances between the projector 1 and the measurement object 300. As described with reference to FIG. 6, by increasing the period of the stripe pattern, a decrease in contrast is suppressed. However, if the period is increased too much, the conversion accuracy from the phase φ to the height h decreases. Therefore, in the three-dimensional shape measurement device 100 according to the present embodiment, the shortest period among the period ranges of the stripe pattern in which the third contrast is equal to or greater than a predetermined value is set as the period candidate. Therefore, the three-dimensional shape measurement device 100 according to the present embodiment can improve the measurement accuracy of the three-dimensional shape.

[0097] <Modification Example 1> In Modification Example 1, the table 35 associates each of the plurality of period candidates with a combination of the distance between the projector 1 and the measurement object 300, the F value of the projector lens 131, and the F value of the camera lens 221.

[0098] In Modification Example 1, in the first step of the method for setting period candidates, the MTF characteristics of the projector lens 131 and the MTF characteristics of the camera lens 221 corresponding to the combination of the distance between the projector 1 and the measurement object 300, the F value of the projector lens 131, and the F value of the camera lens 221 are obtained by experiments.

[0099] In Modification Example 1, in the second step of the method for setting period candidates, a combined MTF characteristic corresponding to the combination of the distance between the projector 1 and the measurement object 300, the F value of the projector lens 131, and the F value of the camera lens 221 is generated.

[0100] In Modification 1, in the third step of the periodic candidate setting method, for each of a plurality of combinations of the distance between the projector 1 and the measurement target 300, the F value of the projector lens 131, and the F value of the camera lens 221, a periodic candidate is set. More specifically, when a periodic candidate corresponding to the target combination among the plurality of combinations of the distance between the projector 1 and the measurement target 300, the F value of the projector lens 131, and the F value of the camera lens 221 is set, the shortest period among the period ranges of the stripe patterns in which the third contrast becomes a predetermined value or more in the combined MTF characteristic for the target combination is set.

[0101] The determination unit 351 determines the period L of the stripe pattern projected onto the measurement target 300 by the projector 1 from a plurality of periodic candidates included in the table 35 according to the combination of the distance between the projector 1 and the measurement target 300, the F value of the projector lens 131, and the F value of the camera lens 221. In other respects, Modification 1 is the same as the above-described embodiment.

[0102] As described above, according to Modification 1, as the period L of the stripe pattern projected onto the measurement target 300 by the projector 1, a period corresponding to the combination of the distance between the projector 1 and the measurement target 300, the F value of the projector lens 131, and the F value of the camera lens 221 is determined. Therefore, according to Modification 1, the measurement accuracy of the three-dimensional shape is improved.

[0103] <Modification 2> FIG. 11 is a diagram for explaining the MTF characteristic of the projector lens 131 and the MTF characteristic of the camera lens 221 in Modification 2.

[0104] Generally, the contrast of the image of the stripe pattern formed by a lens decreases as the distance from the optical axis of the lens increases. Therefore, in Modification 2, the MTF characteristics of the projector lens 131 correspond to the first position that is farthest from the optical axis k1 of the projector lens 131 within the measurement region 306 of the three-dimensional shape measuring device 100. Also, in Modification 2, the MTF characteristics of the camera lens 221 correspond to the second position that is farthest from the optical axis k2 of the camera lens 221 within the measurement region 306. The measurement region 306 is a region where the projection range of the projector 1 and the imaging range of the camera 2 are overlapped. In FIG. 11, the measurement region 306 is a region defined by eight vertices v1 to v8.

[0105] In Modification 2, the MTF characteristics of the projector lens 131 for the target distance among the plurality of distances between the projector 1 and the measurement target 300, and the MTF characteristics of the camera lens 221 for the target distance are obtained by the MTF characteristic acquisition method shown below.

[0106] In the method for acquiring MTF characteristics in Modification 2, the steps different from the method for acquiring MTF characteristics in the above embodiment are the third step and the sixth step. In steps other than the third step and the sixth step, the method for acquiring MTF characteristics in Modification 2 is the same as the method for acquiring MTF characteristics in the above embodiment.

[0107] In Modification 2, in the third step of the method for acquiring MTF characteristics, for each stripe pattern projected on the screen, the contrast in a predetermined region including the first position that is farthest from the optical axis k1 of the projector lens 131 within the measurement region 306 is measured. The first position that is farthest from the optical axis k1 of the projector lens 131 within the measurement region 306 is, for example, the vertex v4 shown in FIG. 11. The predetermined region including the first position is, for example, the region G2 shown in FIG. 11. In Modification 2, the contrast measured in this way is used as the first contrast.

[0108] In Modification 2, in the sixth step of the method for obtaining the MTF characteristics, for each test chart, the contrast on the camera sensor 21 for a predetermined area including the second position farthest from the optical axis k2 of the camera lens 221 within the measurement area 306 is measured. The second position farthest from the optical axis k2 of the camera lens 221 within the measurement area 306 is, for example, the vertex v2 shown in FIG. 11. The predetermined area including the second position is, for example, the area G3 shown in FIG. 11. In Modification 2, the contrast measured in this way is used as the second contrast.

[0109] Regarding the projector lens 131, the contrast is likely to decrease at the first position within the measurement area 306. Regarding the camera lens 221, the contrast is likely to decrease at the second position within the measurement area 306.

[0110] Thus, in Modification 2, the first contrast and the second contrast are different from those in the above-described embodiment. However, in other respects, Modification 2 is the same as the above-described embodiment.

[0111] Therefore, according to Modification 2, the combined MTF characteristics are generated by combining the MTF characteristics of the projector lens 131 and the MTF characteristics of the camera lens 221 at the position where the contrast is low within the measurement area 306. Therefore, according to Modification 2, the three-dimensional shape measurement device 100 can make the measurement accuracy of the three-dimensional shape equal to or higher than the specified value at any position within the measurement area 306.

[0112] <Modification 3> FIG. 12 is a diagram for explaining the MTF characteristics of the projector lens 131 and the MTF characteristics of the camera lens 221 in Modification 3.

[0113] Generally, when the distance d1 between the projector 1 and the measurement target 300 changes while the focus of the projector lens 131 is adjusted to the measurement target 300, the contrast of the fringe pattern image formed by the projector lens 131 decreases. That is, in a state where the focus of the projector lens 131 is adjusted to the measurement bottom surface 309a in the measurement region 306 of the three-dimensional shape measurement device 100, the contrast of the fringe pattern projected onto the measurement upper surface 309b in the measurement region 306 of the three-dimensional shape measurement device 100 is lower than the contrast of the fringe pattern projected onto the measurement bottom surface 309a.

[0114] Also, when the distance between the camera 2 and the measurement target 300 changes while the focus of the camera lens 221 is adjusted to the measurement target 300, the contrast of the fringe pattern image formed by the camera lens 221 decreases.

[0115] Therefore, in the third modification, the MTF characteristics of the projector lens 131 correspond to the measurement upper surface 309b in the measurement region 306 in a first state where the focus of the projector lens 131 is adjusted to the measurement bottom surface 309a in the measurement region 306 of the three-dimensional shape measurement device 100. Also, in the third modification, the MTF characteristics of the camera lens 221 correspond to the measurement upper surface 309b in the measurement region 306 in a second state where the focus of the camera lens 221 is adjusted to the measurement bottom surface 309a in the measurement region 306 of the three-dimensional shape measurement device 100.

[0116] Among the plurality of distances between the projector 1 and the measurement target 300 in the third modification, the MTF characteristics of the projector lens 131 for the target distance and the MTF characteristics of the camera lens 221 for the target distance are obtained by the following MTF characteristic acquisition method.

[0117] In the MTF characteristic acquisition method in the third modification, step 1A and step 1B shown below are added to the MTF characteristic acquisition method in the above embodiment. Step 1A is performed after the first step in the MTF characteristic acquisition method in the above embodiment.

[0118] In step 1A, the height d2 of the measurement region 306 of the three-dimensional shape measurement device 100 is calculated. The height d2 is the distance between the measurement bottom surface 309a and the measurement upper surface 309b. After step 1A, step 1B is performed.

[0119] In step 1B, the distance between the projector 1 and the screen is narrowed by the height d2 calculated in step 1A. After step 1B, the second to seventh steps in the method for obtaining the MTF characteristics in the above embodiment are performed.

[0120] Thus, in Modification 3, the first contrast and the second contrast are different from those in the above embodiment. However, in other respects, Modification 3 is the same as the above embodiment.

[0121] Therefore, according to Modification 3, in a state where the focus of the projector lens 131 and the focus of the camera lens 221 are aligned with the measurement bottom surface 309a, the MTF characteristics of the projector lens 131 and the MTF characteristics of the camera lens 221 at the measurement upper surface 309b, which is the position farthest from the measurement bottom surface 309a within the measurement region 306, are combined to generate a combined MTF characteristic. Therefore, according to Modification 3, the three-dimensional shape measurement device 100 can make the measurement accuracy of the three-dimensional shape equal to or higher than the specified value at any position within the measurement region 306.

[0122] <Modification 4> In the above embodiment, the three-dimensional shape measurement device 100 includes the distance measurer 4, but it may not include the distance measurer 4. When the three-dimensional shape measurement device 100 does not include the distance measurer 4, the user may input the measurement distance to the three-dimensional shape measurement device 100. When the user inputs the measurement distance to the three-dimensional shape measurement device 100, in step S1 in FIG. 10, the processor 31 receives the measurement distance from the user.

[0123] The embodiments disclosed this time should be considered as illustrative in all respects and not restrictive. The scope of the present invention is shown not by the above description but by the claims, and it is intended that all modifications within the meaning and scope equivalent to the claims are included.

Explanation of Signs

[0124] 1 Projector, 2 Camera, 3 Computer, 4 Distance Measuring Device, 11 Light Source, 12 Light Modulation Device, 13 Projection Optical System, 21 Camera Sensor, 22 Condensing Optical System, 31 Processor, 32 Memory, 33 Storage, 34 Program, 35 Table, 50 Cosine Curve, 52, 54 Curves, 61, 62, 63, 91a, 91b, 91c, 92a, 92b, 92c, 93a, 93b, 93c Lines, 100 Three-Dimensional Shape Measuring Device, 121 Image Display Element, 131 Projector Lens, 200 Robot, 221 Camera Lens, 300 Measurement Target, 302 Tray, 304 Workpiece, 305 Measurement Surface, 306 Measurement Region, 308 Range, 309a Measurement Bottom Surface, 309b Measurement Top Surface, 351 Determination Unit, 352 Measurement Unit, 500 Conveyor Belt, 1000 System, L Period, PA Strip Pattern, h, d2 Heights, k1, k2 Optical Axes, v1 to v8 Vertices.

Claims

1. A three-dimensional shape measurement device, comprising a projector lens and a projector that projects a stripe pattern, comprising a camera lens and a camera that captures the stripe pattern projected by the projector to obtain a captured image of the stripe pattern, a measurement unit that measures the three-dimensional shape of a measurement object based on the captured image, and a determination unit that determines the period of the stripe pattern from a plurality of period candidates, wherein the plurality of period candidates are set based on a combined MTF (Modulation Transfer Function) characteristic generated by combining a first MTF characteristic of the projector lens and a second MTF characteristic of the camera lens. A three-dimensional shape measurement device.

2. The three-dimensional shape measurement device further comprises a storage unit that stores a table associating each of the plurality of period candidates with the distance between the projector and the measurement object, the combined MTF characteristic is generated for each of a plurality of distances between the projector and the measurement object, as each of the plurality of period candidates, a period at which the contrast in the combined MTF characteristic for the distance corresponding to the period candidate is equal to or greater than a predetermined value is set, and the determination unit determines the period of the stripe pattern based on the period candidate corresponding to the measured distance between the projector and the measurement object among the plurality of period candidates. The three-dimensional shape measurement device according to claim 1.

3. The projector further comprises an image display element, the camera further comprises a camera sensor, the first MTF characteristic shows the correspondence between the spatial frequency on the image display element and a first contrast representing the contrast performance of the projector lens, the second MTF characteristic shows the correspondence between the spatial frequency on the camera sensor and a second contrast representing the contrast performance of the camera lens, the combined MTF characteristic shows the correspondence between the spatial frequency on the image display element and a third contrast, and the third contrast corresponding to the first spatial frequency on the image display element is calculated based on the first contrast corresponding to the first spatial frequency on the image display element and the second contrast corresponding to the second spatial frequency on the camera sensor. The three-dimensional shape measurement device according to claim 1, wherein the first spatial frequency on the image display element and the second spatial frequency on the camera sensor have the same spatial frequency on the measurement surface of the three-dimensional shape measurement device.

4. The three-dimensional shape measurement device further includes a storage unit that stores a table associating each of the plurality of period candidates with a combination of the distance between the projector and the measurement object, the F value of the projector lens, and the F value of the camera lens. The combined MTF characteristics are generated for each of a plurality of combinations of the distance between the projector and the measurement object, the F value of the projector lens, and the F value of the camera lens. As each of the plurality of period candidates, a period at which the contrast becomes equal to or greater than a predetermined value in the combined MTF characteristics for the combination corresponding to the period candidate is set. The determination unit determines the period of the stripe pattern based on the period candidate corresponding to the combination among the plurality of period candidates, in the three-dimensional shape measurement device according to claim 1.

5. The first MTF characteristic corresponds to a first position that is farthest from the first optical axis of the projector lens within the measurement region of the three-dimensional shape measurement device. The three-dimensional shape measurement device according to any one of claims 1 to 4, wherein the second MTF characteristic corresponds to a second position that is farthest from the second optical axis of the camera lens within the measurement region.

6. The first MTF characteristic corresponds to the upper measurement surface in the measurement region in a first state where the focus of the projector lens is adjusted to the measurement bottom surface in the measurement region of the three-dimensional shape measurement device. The three-dimensional shape measurement device according to any one of claims 1 to 4, wherein the second MTF characteristic corresponds to the upper measurement surface in a second state where the focus of the camera lens is adjusted to the measurement bottom surface.

7. The three-dimensional shape measurement device further includes a distance measuring device that measures the measurement distance between the projector and the measurement object. The determination unit determines the period of the stripe pattern from the plurality of period candidates according to the measurement distance measured by the distance measuring device, in the three-dimensional shape measurement device according to claim 1.

8. A method for measuring the three-dimensional shape of a measurement object, comprising: determining a period of a stripe pattern projected by a projector including a projector lens from a plurality of period candidates; projecting the stripe pattern by the projector; Imaging the stripe pattern projected by the projector with a camera including a camera lens to obtain a captured image of the stripe pattern; measuring the three-dimensional shape of the measurement target based on the captured image; The plurality of period candidates are set based on a combined MTF characteristic generated by combining a first MTF (Modulation Transfer Function) characteristic of the projector lens and a second MTF characteristic of the camera lens. A measurement method.

9. A measurement program that causes one or more computers to execute the measurement method according to claim 8.

Images