Three-dimensional measurement device, three-dimensional measurement method, three-dimensional measurement program, and recording medium

Abstract

In the present invention, three-dimensional data Ds on a workpiece 10 acquired by a phase shift method (step S102) and three-dimensional data Dl on the workpiece 10 acquired by an optical cutting method (step S104) are combined, and composite three-dimensional data Dt can be generated (step S105). Therefore, in the workpiece 10, the three-dimensional data Ds is acquired by the phase shift method for a range Rm1 in which the phase shift method effectively functions, and the three-dimensional data Dl is acquired by the optical cutting method for a range Rm2 in which the phase shift method does not function effectively, whereby the composite three-dimensional data Dt can be generated. As a result, it is possible to handle a workpiece 10 that includes a range in which the phase shift method does not function effectively while taking advantage of the phase shift method that makes it possible to efficiently acquire three-dimensional data of the workpiece 10.

Description

Three-dimensional measurement device, three-dimensional measurement method, three-dimensional measurement program, and recording medium

[0001] The present invention relates to a technique for acquiring three-dimensional data indicating the three-dimensional shape of a workpiece, and more particularly to a technique for suitably acquiring three-dimensional data of a workpiece including a range in which the phase shift method does not function effectively.

[0002] Patent Document 1 describes a phase shift method and a light section method as techniques for acquiring the three-dimensional shape of a workpiece, which is an object to be inspected. In the phase shift method, a periodic pattern, which is a periodically changing light pattern, is irradiated onto the workpiece, and the three-dimensional shape of the workpiece is acquired based on the result of capturing an image of the periodic pattern irradiated onto the workpiece. In the light section method, a line pattern, which is a linearly extending light pattern, is irradiated onto the workpiece, and the three-dimensional shape of the workpiece is acquired based on the result of capturing an image of the line pattern irradiated onto the workpiece.

[0003] Japanese Patent Application Laid-Open No. 2021-177186

[0004] In the field of component mounting, three-dimensional data showing the three-dimensional shape of a circuit board or component is acquired to inspect the condition of components mounted on the circuit board. From the perspective of efficiently acquiring three-dimensional data for large workpieces such as circuit boards, the phase shift method has been considered advantageous. This is because the phase shift method can acquire three-dimensional data for each large area illuminated with a periodic pattern, whereas the light section method can only acquire three-dimensional data for each small area illuminated with a line pattern.

[0005] However, when attempting to simultaneously inspect the condition of the solder that joins the component to the board using a device for inspecting the condition of the component, the phase shift method sometimes did not work effectively. Because solder has a metallic luster, periodic patterns incident on the board or other areas other than the solder are mirror-reflected by the solder and captured in the image. As a result, periodic patterns unrelated to the three-dimensional shape of the solder are captured within the area where the solder is present, making it impossible to properly obtain three-dimensional data of the solder.

[0006] This invention has been made in consideration of the above-mentioned problems, and aims to take advantage of the advantages of the phase shift method, such as the ability to efficiently obtain three-dimensional data of a workpiece, while also being able to handle workpieces that include areas in which the phase shift method does not function effectively.

[0007] The three-dimensional measuring device according to the present invention includes a workpiece holding unit that holds a workpiece, a light irradiation unit that irradiates the workpiece with light, an imaging unit that images the light irradiated onto the workpiece by the light irradiation unit, a phase shift measurement control unit that causes the light irradiation unit to irradiate a periodic pattern, which is a periodically changing light pattern, and causes the imaging unit to image the periodic pattern irradiated onto the workpiece to obtain a periodic pattern image, and acquires three-dimensional data that indicates the three-dimensional shape of the workpiece by a phase shift method based on the periodic pattern image, a light section measurement control unit that causes the light irradiation unit to irradiate a line pattern, which is a linearly extending light pattern, and causes the imaging unit to image the line pattern irradiated onto the workpiece to obtain a line pattern image, and acquires three-dimensional data that indicates the three-dimensional shape of the workpiece by the light section method based on the line pattern image, and a data synthesis unit that synthesizes the three-dimensional data acquired by the phase shift measurement control unit and the three-dimensional data acquired by the light section measurement control unit to generate synthesized three-dimensional data.

[0008] The three-dimensional measurement method according to the present invention includes the steps of irradiating a workpiece with a periodic pattern, which is a periodically changing light pattern, capturing an image of the periodic pattern irradiated onto the workpiece to obtain a periodic pattern image, and obtaining three-dimensional data indicating the three-dimensional shape of the workpiece by a phase shift method based on the periodic pattern image; irradiating a workpiece with a line pattern, which is a light pattern extending in a straight line, capturing an image of the line pattern irradiated onto the workpiece to obtain a line pattern image, and obtaining three-dimensional data indicating the three-dimensional shape of the workpiece by a light section method based on the line pattern image; and combining the three-dimensional data obtained by the phase shift method and the three-dimensional data obtained by the light section method to generate composite three-dimensional data.

[0009] A three-dimensional measurement program according to the present invention causes a computer to execute the above three-dimensional measurement method.

[0010] A recording medium according to the present invention records the above three-dimensional measurement program in a computer-readable manner.

[0011] The present invention (three-dimensional measurement device, three-dimensional measurement method, three-dimensional measurement program, and recording medium) configured in this manner can generate composite three-dimensional data by combining three-dimensional data of a workpiece acquired by the phase shift method and three-dimensional data of a workpiece acquired by the light-section method. Therefore, it is possible to acquire three-dimensional data by the phase shift method for areas of the workpiece where the phase shift method effectively functions, and acquire three-dimensional data by the light-section method for areas where the phase shift method does not effectively function, and generate composite three-dimensional data from these. As a result, it is possible to take advantage of the phase shift method, such as its ability to efficiently acquire three-dimensional data of a workpiece, while also being able to handle workpieces that include areas where the phase shift method does not effectively function.

[0012] The three-dimensional measuring device may also be configured so that the phase-shift measurement control unit causes the light irradiation unit to irradiate a periodic pattern onto a first measurement range of the workpiece to acquire three-dimensional data indicating the three-dimensional shape of the workpiece in the first measurement range, the light-section measurement control unit causes the light irradiation unit to irradiate a line pattern onto a second measurement range of the workpiece that is different from the first measurement range to acquire three-dimensional data indicating the three-dimensional shape of the workpiece in the second measurement range, and the data synthesis unit synthesizes the three-dimensional data from the first measurement range acquired by the phase-shift measurement control unit and the three-dimensional data from the second measurement range acquired by the light-section measurement control unit to generate synthesized three-dimensional data. With this configuration, three-dimensional data from the first measurement range, where the phase-shift method effectively functions, can be measured using the phase-shift method, and three-dimensional data from the second measurement range, where the phase-shift method does not effectively function, can be measured using the light-section method. As a result, while taking advantage of the phase-shift method, such as its ability to efficiently acquire three-dimensional data of the workpiece, it is also possible to handle workpieces that include areas where the phase-shift method does not effectively function.

[0013] The three-dimensional measuring device may also be configured so that the second measurement range is narrower than the first measurement range. In this configuration, acquisition of three-dimensional data by the light-section method is limited to a second measurement range that is narrower than the first measurement range for acquiring three-dimensional data by the phase-shift method. As a result, it is possible to take advantage of the advantage of the phase-shift method, such as the ability to efficiently acquire three-dimensional data of the workpiece.

[0014] The three-dimensional measuring device may be configured to further include a measurement range storage unit that stores the second measurement range, and the light-section measurement control unit acquires three-dimensional data in the second measurement range stored in the measurement range storage unit. In this configuration, by storing the range in which the phase-shift method does not function effectively in the measurement range storage unit as the second measurement range, it is possible to appropriately deal with workpieces that include a range in which the phase-shift method does not function effectively.

[0015] The three-dimensional measuring device may also be configured such that the light-section measurement control unit sets the range in which the phase-shift measurement control unit failed to acquire three-dimensional data as the second measurement range. In this configuration, the second measurement range in which the phase-shift method does not function effectively is actually measured based on the failure of the phase-shift measurement control unit to acquire three-dimensional data. As a result, the second measurement range in which the phase-shift method does not function effectively can be accurately determined, and appropriate measures can be taken for the workpiece including the second measurement range.

[0016] The three-dimensional measuring device may also be configured so that the phase-shift measurement control unit causes the light irradiation unit to irradiate periodic patterns onto both the first measurement range and the second measurement range, and the light-section measurement control unit sets the second measurement range based on the success or failure of the results of acquiring three-dimensional data indicating the three-dimensional shape of the workpiece in the first measurement range and the second measurement range. In this configuration, the second measurement range in which the phase-shift method does not effectively function is actually measured based on the failure of the phase-shift measurement control unit to acquire three-dimensional data. As a result, the second measurement range in which the phase-shift method does not effectively function can be accurately determined, and appropriate measures can be taken for the workpiece including the second measurement range.

[0017] The three-dimensional measuring device may also be configured so that the workpiece holder holds a reference member having a reference plane so that the reference plane is horizontal, the phase-shift measurement control unit causes the light irradiation unit to irradiate a periodic pattern and the imaging unit to capture the periodic pattern irradiated on the reference plane to obtain a periodic pattern reference image, and the height of the reference plane is obtained by the phase-shift method based on the periodic pattern reference image, the light-section measurement control unit causes the light irradiation unit to irradiate a line pattern and the imaging unit to capture the line pattern irradiated on the reference plane to obtain a line pattern reference image, and the height of the reference plane is obtained by the light-section method based on the line pattern reference image, and the data synthesis unit generates composite three-dimensional data while correcting a difference in height between the height of the reference plane acquired by the phase-shift measurement control unit and the height of the reference plane acquired by the light-section measurement control unit. With this configuration, the difference in height between the three-dimensional data acquired by the phase-shift method and the three-dimensional data acquired by the light-section method can be corrected to appropriately generate composite three-dimensional data.

[0018] The three-dimensional measuring device may further include a height information storage unit that stores the maximum height and minimum height of the workpiece in the height direction, and the light section measurement control unit simultaneously irradiates a first line pattern and a second line pattern, which are line patterns each extending in a predetermined extension direction, at an interval in an orthogonal direction orthogonal to the extension direction, onto the light irradiation unit, acquires a first line pattern image by having the imaging unit capture the first line pattern irradiated onto the workpiece, acquires three-dimensional data indicating the three-dimensional shape of the workpiece by the light section method based on the first line pattern image, acquires a second line pattern image by having the imaging unit capture the second line pattern irradiated onto the workpiece, acquires three-dimensional data indicating the three-dimensional shape of the workpiece by the light section method based on the second line pattern image, and controls the light irradiation unit so that the spacing in the orthogonal direction between the first line pattern and the second line pattern is wider than the spacing in the orthogonal direction between the positions at the maximum height and the positions at the minimum height of the line pattern. With this configuration, the first and second line patterns are simultaneously irradiated onto the second measurement range, and three-dimensional data of the second measurement range can be efficiently acquired by the light-section method. Moreover, the first line pattern and the second line pattern are sufficiently separated to accurately distinguish them, and the three-dimensional data of the second measurement range can be appropriately acquired.

[0019] The three-dimensional measuring device may also be configured so that the light irradiation unit has a projector capable of irradiating both the periodic pattern and the line pattern, the phase-shift measurement control unit controls the projector to irradiate the periodic pattern onto the first measurement range, and the light-section measurement control unit controls the projector to irradiate the line pattern onto the second measurement range. With this configuration, irradiation of the periodic pattern and irradiation of the line pattern can be efficiently performed by the same projector.

[0020] The three-dimensional measuring device may be configured so that, while the phase-shift measurement control unit causes the light irradiation unit to irradiate the first measurement range with a periodic pattern, the light-section measurement control unit causes the light irradiation unit to irradiate the second measurement range with a line pattern. With this configuration, it is possible to efficiently acquire three-dimensional data for each of the first and second measurement ranges while simultaneously irradiating the first measurement range with the periodic pattern and the line pattern.

[0021] The three-dimensional measurement device may also be configured so that the second measurement range is located inside the first measurement range and is surrounded by the first measurement range, the light irradiation unit irradiates the first measurement range and the second measurement range with a periodic pattern generated by light of a first wavelength so as to overlap, and irradiates the second measurement range with a periodic pattern generated by light of a second wavelength different from the first wavelength so as to overlap, the phase-shift measurement control unit acquires a periodic pattern image by causing the imaging unit to capture the light of the first wavelength irradiated onto the first measurement range, and the light-section measurement control unit acquires a line pattern image by causing the imaging unit to capture the light of the second wavelength irradiated onto the second measurement range. With this configuration, the periodic pattern and the line pattern, which are simultaneously irradiated and captured, can be distinguished based on the difference in wavelength of light, and three-dimensional data of the first measurement range can be appropriately acquired from the periodic pattern irradiated onto the first measurement range, and three-dimensional data of the second measurement range can be appropriately acquired from the line pattern irradiated onto the second measurement range.

[0022] The three-dimensional measurement device may also be configured so that the light irradiation unit has a first projector and a second projector, the phase-shift measurement control unit causes the first projector to irradiate a periodic pattern onto a first measurement range, and the light-section measurement control unit causes the second projector to irradiate a line pattern onto a second measurement range outside the range into which the first projector is irradiating the periodic pattern. With this configuration, the periodic pattern and the line pattern that are irradiated simultaneously can be spaced apart on the workpiece, and three-dimensional data for the first measurement range can be appropriately acquired from the periodic pattern irradiated onto the first measurement range, and three-dimensional data for the second measurement range can be appropriately acquired from the line pattern irradiated onto the second measurement range.

[0023] The three-dimensional measuring device may be configured so that the first measurement range is a range that diffusely reflects light irradiated from the light irradiating unit, and the second measurement range is a range that specularly reflects light irradiated from the light irradiating unit. With this configuration, three-dimensional data of the first measurement range of the workpiece that diffusely reflects light can be efficiently obtained by the phase shift method, and three-dimensional data of the second measurement range of the workpiece that specularly reflects light can be accurately obtained by the light section method.

[0024] The three-dimensional measuring device may be configured so that the workpiece includes a substrate, components placed on the substrate, and solder joining the components to the substrate, the first measurement range being a range where no solder is present, and the second measurement range being a range where solder is present. With this configuration, three-dimensional data of the first measurement range of the workpiece where no solder is present can be efficiently obtained by the phase shift method, and three-dimensional data of the second measurement range of the workpiece where solder is present can be accurately obtained by the light-section method.

[0025] According to the present invention, it is possible to utilize the advantage of the phase shift method, such as the ability to efficiently obtain three-dimensional data of a workpiece, while also being able to handle workpieces that include areas in which the phase shift method does not function effectively.

[0026] 10 is a block diagram showing a mounted substrate inspection apparatus that is an example of a three-dimensional measuring apparatus according to the present invention. FIG. 11 is a diagram schematically showing a sine wave pattern used in the phase shift method. FIG. 12 is a diagram schematically showing a line pattern used in the light section method. FIG. 13 is a diagram schematically explaining a first example of a method for acquiring three-dimensional data using the light section method. FIG. 14 is a diagram schematically explaining a first example of a method for acquiring three-dimensional data using the light section method. FIG. 15 is a diagram schematically explaining a first example of a method for acquiring three-dimensional data using the light section method. FIG. 16 is a diagram schematically explaining a second example of a method for acquiring three-dimensional data using the light section method. FIG. 17 is a diagram schematically showing a situation that occurs when inspecting solder in a mounted substrate inspection apparatus that inspects components. A flowchart showing a first example of three-dimensional measurement. FIG. 18 is a diagram schematically showing operations executed according to the flowchart of FIG. 7. FIG. 19 is a diagram schematically showing operations executed according to the flowchart of FIG. 7. A flowchart showing an example of height calibration. FIG. 19 is a diagram schematically showing operations executed in the height calibration of FIG. 1 is a diagram showing a relationship between the distance between two line patterns simultaneously irradiated by the light-section method and the shape of the workpiece; FIG. 2 is a diagram showing a first example of parallel measurement in which the phase shift method and the light-section method are performed in parallel; FIG. 3 is a diagram showing a second example of parallel measurement in which the phase shift method and the light-section method are performed in parallel; and FIG. 4 is a flowchart showing a second example of three-dimensional measurement.

[0027] FIG. 1 is a block diagram showing a mounted substrate inspection apparatus, an example of a three-dimensional measuring apparatus according to the present invention. In this specification, the horizontal X-direction, the horizontal Y-direction perpendicular to the X-direction, and the vertical Z-direction are appropriately indicated. The mounted substrate inspection apparatus 1 includes a transfer stage 2, an inspection head 3, a drive mechanism 4, and a controller 5. The controller 5 controls the transfer stage 2, the inspection head 3, and the drive mechanism 4 to inspect a workpiece 10, which is a mounted substrate. The workpiece 10 includes a flat substrate 11 (printed circuit board), components 13 mounted on a surface 11A of the substrate 11, and solder 15 that joins the components 13 to the surface 11A of the substrate 11. The components 13 are electronic components such as capacitors, transistors, or integrated circuits. The mounted substrate inspection apparatus 1 inspects the surface condition of the workpiece 10, specifically, the presence or absence of misalignment of the components 13 and the shape of the solder 15.

[0028] The transfer stage 2 holds the workpiece 10, and the drive mechanism 4 drives the transfer stage 2 in the X and Y directions. The surface 11A of the substrate 11 of the workpiece 10 is held horizontally by the transfer stage 2. In other words, the normal to the surface 11A of the substrate 11 held by the transfer stage 2 is parallel to the Z direction.

[0029] The inspection head 3 has a camera 31 that captures an image of an imaging field R3 (camera field of view) from above, and two projectors 33 that irradiate the imaging field R3 with light L. Each projector 33 irradiates the light L onto the workpiece 10 held on the transport stage 2 from a direction tilted with respect to the Z direction. In particular, the two projectors 33 are tilted on opposite sides to each other with respect to the Z direction, and irradiate the light L onto the workpiece 10 in the imaging field R3 from opposite sides in the X direction when viewed in a plan view from the Z direction.

[0030] The camera 31 faces the workpiece 10 held on the transfer stage 2 from the Z direction. The camera 31 has an objective lens 311 and a solid-state image sensor 32. The objective lens 311 faces the workpiece 10 from the Z direction and collects light L reflected by the workpiece 10 in an imaging field of view R3 onto the solid-state image sensor 32. The solid-state image sensor 32 detects the light L collected by the objective lens 311, thereby acquiring an image I of the light L irradiated onto the workpiece 10 in the imaging field of view R3.

[0031] The controller 5 has a main control unit 51 composed of a CPU (Central Processing Unit) and memory, and the main control unit 51 controls all components of the device to inspect the workpiece 10. The controller 5 has an imaging control unit 52 that controls the camera 31 and each projector 33, and a drive control unit 53 that controls the drive mechanism 4. The main control unit 51 controls the drive mechanism 4 via the drive control unit 53 to move the transport stage 2 relative to the inspection head 3, thereby fitting the inspection target area of ​​the workpiece 10 within the imaging field of view R3 of the inspection head 3. Next, the main control unit 51 controls the projector 33 via the imaging control unit 52, causing the projector 33 to irradiate light L onto the workpiece 10 in the imaging field of view R3. The projector 33 is, for example, a liquid crystal projector or a DLP (Digital Lighting Processing) projector, and can irradiate light L of any pattern onto the imaging field of view R3 in response to commands from the imaging control unit 52. Furthermore, the main control unit 51 controls the camera 31 via the imaging control unit 52 to cause the camera 31 to capture the light L irradiated onto the workpiece 10 in the imaging field of view R3, thereby obtaining an image I of the light L.

[0032] Furthermore, the controller 5 has a storage unit 54 and a communication unit 55. The storage unit 54 is configured with non-volatile memory. The communication unit 55 communicates with a server computer 9 provided external to the mounted board inspection apparatus 1. This server computer 9 stores a board inspection program 91 for causing the mounted board inspection apparatus 1 to perform board inspection, and the communication unit 55 downloads the board inspection program 91 from the server computer 9. The main control unit 51 executes the board inspection program 91 downloaded by the communication unit 55, thereby performing various controls such as three-dimensional measurement, which will be described later.

[0033] The imaging control unit 52 can acquire three-dimensional data indicating the three-dimensional shape of the workpiece 10 using both the phase shift method and the light section method. FIG. 2 is a diagram schematically illustrating a sine wave pattern used in the phase shift method. When performing the phase shift method, the projector 33, upon receiving a command from the imaging control unit 52, irradiates the workpiece 10 in the imaging field of view R3 with a sine wave pattern Ls, which is a pattern of light L in which the change in light intensity depending on the position indicates a sine wave. In particular, the projector 33 sequentially irradiates the workpiece 10 in the imaging field of view R3 with four sine wave patterns Ls(N) (N = 1, 2, 3, 4) having different phases. In addition, each time the projector 33 irradiates the workpiece 10 with a sine wave pattern Ls(N), the imaging control unit 52 causes the camera 31 to capture the sine wave pattern Ls(N) in the imaging field of view R3 and acquires an image I of the sine wave pattern Ls(N) as a sine wave pattern image Is(N). In this way, four sine wave pattern images Is(1), Is(2), Is(3), and Is(4) are obtained by capturing images of the sine wave patterns Ls(1), Ls(2), Ls(3), and Ls(4) that are out of phase with one another. Then, the main control unit 51 acquires three-dimensional data Ds that indicates the three-dimensional shape of the workpiece 10 in the imaging field of view R3 based on the sine wave pattern images Is(1), Is(2), Is(3), and Is(4), and stores the data in the memory unit 54.

[0034] 3 is a diagram schematically illustrating a line pattern used in the light-section method. When the light-section method is performed, the projector 33 receives a command from the imaging control unit 52 and irradiates the workpiece 10 in the imaging field of view R3 with a line pattern Ll, which is a pattern of light L extending parallel to the Y direction for cleaning. The imaging control unit 52 also causes the camera 31 to capture an image of the line pattern Ll in the imaging field of view R3, and acquires the image of the line pattern Ll as a line pattern image Il.

[0035] The image acquired by the imaging control unit 52 in one imaging session with the camera 31 is a linear line pattern image Il extending in the Y direction. Therefore, the imaging control unit 52 repeatedly captures the line pattern Ll with the camera 31 while changing the position at which the projector 33 projects the line pattern Ll in the X direction (scanning direction) by the width of the line pattern Ll. In this way, multiple line pattern images Il are acquired by capturing images of the line pattern Ll projected at different positions. Based on the multiple line pattern images Il, the main control unit 51 acquires three-dimensional data Dl that indicates the three-dimensional shape of the workpiece 10 in the X direction within a range wider than the width of the line pattern image Il.

[0036] 4A, 4B, and 4C are diagrams illustrating a first example of a method for acquiring three-dimensional data using the light-section method. The object W shown in FIGS. 4A, 4B, and 4C has a length Wx in the X direction, a width Wy in the Y direction, and a height Wz in the Z direction. A line pattern Ll is irradiated onto the object W from the cutting direction Q.

[0037] Therefore, the X-direction position Xh of the line pattern Ll on the top surface of the object W and the X-direction position Xl of the line pattern Ll on the mounting plane W0 on which the object W is placed differ depending on the difference in height between the top surface of the object W and the mounting plane W0 on which the object W is placed. In this case, the height Wz of the object W is given by the following equation, Wz = |Xh - Xl| × tan θ, using the angle θ between the line pattern Ll and the mounting plane W0. Therefore, the main control unit 51 can capture multiple line images Il and obtain information indicating the height of the object W based on the position at which the line pattern Ll is irradiated in each line image Il, thereby acquiring three-dimensional data Dl indicating the three-dimensional shape of the object W.

[0038] Fig. 5 is a diagram illustrating a second example of a method for acquiring three-dimensional data using the light-section method. Fig. 5 illustrates multiple pixel rows 321 to 325, each consisting of multiple pixels aligned in the Y direction. In particular, Fig. 5 illustrates, in chronological order (at times T1 to T5), how the pixel rows 321 to 325 sequentially capture images of the target position TG each time the projector 33 moves the line pattern Ll in the X direction by one pixel of the solid-state imaging element 32.

[0039] 4A, 4B, and 4C, three-dimensional data Dl of the object W is acquired based on differences in the irradiation position of the line pattern Ll in the X direction (scanning direction). In contrast, in the example of Fig. 5, three-dimensional data Dl of the object W is acquired based on differences in the time at which the line pattern Ll is imaged.

[0040] At time T1, the pixel array 321 captures an image of the target position TG. At time T1, the line pattern Ll is not irradiated onto the target position TG, and the image captured by the pixel array 321 does not include the line pattern Ll.

[0041] At time T2, which follows time T1, pixel row 322 captures an image of target position TG. At time T2, the line pattern Ll projected onto the upper surface of object W is located at target position TG. Therefore, among the multiple pixels belonging to pixel row 322, pixels corresponding to the upper surface of object W capture the image of line pattern Ll, while pixels corresponding to mounting plane W0 do not capture the image of line pattern Ll.

[0042] At time T3 after time T2, the pixel array 323 captures an image of the target position TG. At time T3, the line pattern Ll is not irradiated onto the target position TG, and the image captured by the pixel array 323 does not include the line pattern Ll.

[0043] At time T4, which follows time T3, the pixel array 324 captures an image of the target position TG. At time T4, the line pattern Ll projected onto the mounting plane W0 is located at the target position TG. Therefore, among the multiple pixels belonging to the pixel array 324, the pixels corresponding to the mounting plane W0 capture the image of the line pattern Ll, while the pixels corresponding to the top surface of the object W do not capture the image of the line pattern Ll.

[0044] At time T5 after time T4, the pixel array 325 captures an image of the target position TG. At time T5, the line pattern Ll is not irradiated onto the target position TG, and the image captured by the pixel array 325 does not include the line pattern Ll.

[0045] In this way, the times T2 and T4 at which the line pattern Ll projected onto the same target position TG is imaged differ depending on the height at which the line pattern Ll is projected (the upper surface of the object W, the mounting plane W0). The difference between times T2 and T4 corresponds to the distance in the X direction between pixel 322 at which the line pattern Ll was detected at time T2 and pixel 324 at which the line pattern Ll was detected at time T4. Therefore, three-dimensional data Dl of the object W can be acquired using the same principle as described using Figures 4A to 4C.

[0046] Of the phase shift method and the light section method, the phase shift method is considered to be advantageous from the viewpoint of efficiently acquiring three-dimensional data. However, in an apparatus that inspects components 13 mounted on a substrate 11, such as the mounted substrate inspection apparatus 1, when attempting to also inspect the condition of the solder 15 that joins the components 13 to the substrate 11, the phase shift method may not function effectively.

[0047] FIG. 6 is a diagram schematically illustrating a situation that occurs when inspecting solder using a mounting board inspection device that inspects components. In the example of FIG. 6 , of the sine wave patterns Ls projected from the projector 33, sine wave pattern Ls1 is projected directly onto the solder 15 and accurately reflects the shape of the solder 15. Therefore, the main control unit 51 can acquire three-dimensional data Ds of the solder 15 based on the sine wave pattern image Is acquired by capturing the sine wave pattern Ls1 with the camera 31. Meanwhile, of the sine wave patterns Ls projected from the projector 33, sine wave pattern Ls2 is projected onto an area adjacent to the solder 15 on the surface 11A of the substrate 11. Furthermore, a portion of the sine wave pattern Ls2 is reflected from the adjacent area before entering the solder 15 and being specularly reflected by the solder 15. Thus, a portion of the sine wave pattern Ls2 specularly reflected by the solder 15 enters the camera 31 and is captured in the line pattern image Il. As a result, the main control unit 51 fails in the calculation to calculate the three-dimensional data Ds indicated by the sine wave pattern image Is. To address this, the main control unit 51 executes three-dimensional measurement as follows.

[0048] Fig. 7 is a flowchart showing a first example of three-dimensional measurement, and Figs. 8A, 8B and 9 are diagrams schematically showing the operations executed in accordance with the flowchart of Fig. 7. The flowchart of Fig. 7 is executed under the control of the main control unit 51 in accordance with the substrate inspection program 91.

[0049] In step S101, the drive control unit 53 controls the drive mechanism 4 to adjust the position of the workpiece 10 held by the transfer stage 2, thereby moving the inspection target range Rt of the workpiece 10 into the imaging field of view R3. As a result, the camera 31 faces the inspection target range Rt from the Z direction, and the projector 33 faces the inspection target range Rt from a direction inclined toward the Z direction. In the examples of FIGS. 8A and 8B , the inspection target range Rt is set to overlap with the board 11, the component 13, and the solder 15.

[0050] In step S102, as shown in the "Phase Shift Method" column of FIG. 8B, the projector 33 sequentially projects sine wave patterns Ls(N) onto the inspection target area Rt (N = 1, 2, 3, 4). Each time a sine wave pattern Ls(N) is projected onto the inspection target area Rt, the camera 31 captures the sine wave pattern Ls(N). In this manner, four sine wave pattern images Is(1), Is(2), Is(3), and Is(4) are acquired, each capturing the sine wave patterns Ls(1), Ls(2), Ls(3), and Ls(4) that are out of phase with one another. The main control unit 51 then performs a calculation to obtain three-dimensional data Ds representing the three-dimensional shape of the workpiece 10 in the inspection target area Rt based on the sine wave pattern images Is(1), Is(2), Is(3), and Is(4).

[0051] In step S103, the main control unit 51 determines whether or not there is a failure area Rf in the calculation for obtaining the three-dimensional data Ds for the inspection target area Rt. If it determines that there is no failure area Rf ("NO" in step S103), the main control unit 51 proceeds to step S106 to determine whether the inspection is complete. However, in this example, the inspection target area Rt overlaps with the solder 15. Therefore, the calculation for obtaining the three-dimensional data Ds for the inspection target area Rt is successful in the area where the solder 15 does not exist, but fails in the area where the solder 15 does exist. In other words, based on the sine wave pattern image Is(N) in the area Rm1 where the solder 15 (i.e., the specularly reflective component) does not exist, three-dimensional data Ds representing the three-dimensional shape of the board surface 11A and the component 13 in the area Rm1 is obtained. On the other hand, the acquisition of the three-dimensional data Ds in the area Rm2 where the solder 15 (i.e., the specularly reflective component) exists fails. Therefore, the main control unit 51 determines that a failure range Rf exists ("YES" in step S103), and proceeds to step S104.

[0052] In step S104, as shown in the "Light-Section Method" section of FIG. 8B , the projector 33 projects a line pattern Ll onto the failure area Rf (i.e., area Rm2) within the inspection target area Rt. Note that the line pattern Ll only needs to be projected so as to overlap the entire failure area Rf in the Y direction; both ends of the line pattern Ll may extend beyond the failure area Rf. As described above, the projector 33 changes the position at which the line pattern Ll is projected in the X direction by the width of the line pattern Ll ( FIG. 9 ). Specifically, as shown in FIG. 9 , the projector 33 has a plurality of gratings 330 arranged two-dimensionally in a matrix. These gratings 330 are provided with grating rows 331, 332, 333, etc., each consisting of a row of gratings 330 aligned in a direction corresponding to the extension direction (Y direction) of the line pattern Ll. In other words, these grating rows 331, 332, 333 are aligned in order in the X direction. The projector 33 then emits light L sequentially onto these lattice columns 331, 332, 333, ..., thereby moving the position of the line pattern Ll in the X direction (times T1 to T6). In step S104, the main control unit 51 acquires multiple line pattern images Il by having the camera 31 capture images of the multiple line patterns Ll that have been projected at different positions in the X direction. The main control unit 51 then acquires three-dimensional data Dl that indicates the failure range Rf, i.e., the three-dimensional shape of the solder 15, based on the multiple line pattern images Il.

[0053] As a result, three-dimensional data Ds indicating the three-dimensional shape of the workpiece 10 in range Rm1 (i.e., the board surface 11A and the components 13) and three-dimensional data Dl indicating the three-dimensional shape of the workpiece 10 in range Rm2 (i.e., the solder 15) are acquired. In step S105, the main control unit 51 combines the three-dimensional data Ds and the three-dimensional data Dl to generate combined three-dimensional data Dt indicating the three-dimensional shape of the workpiece 10 (i.e., the board surface 11A, the components 13, and the solder 15) in the inspection target range Rt.

[0054] 8A and 8B, the relationship between the inspection target range Rt, range Rm1, and range Rm2 will be explained. The inspection target range Rt includes both range Rm1 and range Rm2, and range Rm1 and range Rm2 do not overlap. In other words, range Rm1 is the range of the inspection target range Rt other than range Rm2, and range Rm2 is the range of the inspection target range Rt other than range Rm1. In this example, range Rm2 is narrower than range Rm1.

[0055] In the following step S106, the main control unit 51 determines whether or not to end the inspection. That is, if the acquisition of three-dimensional data for all of the inspection target ranges Rt set for the workpiece 10 has been completed, the main control unit 51 determines "YES" in step S106 and ends the three-dimensional measurement of Fig. 7. On the other hand, if three-dimensional data has not been acquired for some of the inspection target ranges Rt set for the workpiece 10, the main control unit 51 determines "NO" in step S106 and returns to step S101. Then, steps S101 to S105 are executed for that part of the inspection target range Rt.

[0056] In the embodiment described above, the three-dimensional data Ds of the workpiece 10 acquired by the phase shift method (step S102) and the three-dimensional data Dl of the workpiece 10 acquired by the light-section method (step S104) can be combined to generate composite three-dimensional data Dt (step S105). Therefore, for the range Rm1 of the workpiece 10 where the phase shift method effectively functions, the three-dimensional data Ds can be acquired by the phase shift method, and for the range Rm2 where the phase shift method does not effectively function, the three-dimensional data Dl can be acquired by the light-section method, and the composite three-dimensional data Dt can be generated. As a result, while taking advantage of the phase shift method's ability to efficiently acquire three-dimensional data of the workpiece 10, it is possible to handle workpieces 10 that include ranges where the phase shift method does not effectively function.

[0057] The main control unit 51 (phase shift measurement control unit) also causes the projector 33 (light irradiation unit) to irradiate a sine wave pattern Ls (periodic pattern) onto a range Rm1 (first measurement range) of the workpiece 10, thereby acquiring three-dimensional data Ds representing the three-dimensional shape of the workpiece 10 within the range Rm1 (step S102). The main control unit 51 (light section measurement control unit) also causes the projector 33 (light irradiation unit) to irradiate a line pattern Ll onto a range Rm2 (second measurement range) of the workpiece 10, which is different from the range Rm1, thereby acquiring three-dimensional data Dl representing the three-dimensional shape of the workpiece 10 within the range Rm2 (step S104). The main control unit 51 (data synthesis unit) then synthesizes the three-dimensional data Ds for the range Rm1 and the three-dimensional data Dl for the range Rm2 to generate synthesized three-dimensional data Dt (step S105). In this configuration, the three-dimensional data Ds in the range Rm1 where the phase shift method effectively functions can be measured by the phase shift method, and the three-dimensional data Dl in the range Rm2 where the phase shift method does not effectively function can be measured by the light-section method. As a result, while taking advantage of the phase shift method, such as the ability to efficiently obtain three-dimensional data of the workpiece 10, it is also possible to handle workpieces 10 that include a range where the phase shift method does not effectively function.

[0058] Furthermore, range Rm2 is narrower than range Rm1. In this configuration, acquisition of three-dimensional data Dl by the light-section method is limited to range Rm2, which is narrower than range Rm1 for acquiring three-dimensional data Ds by the phase shift method. As a result, it is possible to take advantage of the advantage of the phase shift method, such as the ability to efficiently acquire three-dimensional data of the workpiece 10.

[0059] Furthermore, the main control unit 51 sets the failure range Rf, where the measurement using the phase shift method failed to acquire three-dimensional data, as the range Rm2 to be measured using the light-section method (steps S103 and S104). In this configuration, the range Rm2 in which the phase shift method does not function effectively (i.e., the failure range Rf) is actually measured based on the failure to acquire three-dimensional data using the phase shift method. As a result, the range Rm2 in which the phase shift method does not function effectively can be accurately determined, and appropriate measures can be taken for the workpiece 10 that includes the range Rm2.

[0060] Furthermore, the main control unit 51 causes the projector 33 to project the sine wave pattern Ls onto both the range Rm1 and the range Rm2 (i.e., the inspection target range Rt), and executes (in other words, attempts) acquisition of three-dimensional data indicating the three-dimensional shape of the workpiece 10 within the range Rm1 and the range Rm2. Based on the success or failure of the results (steps S102 and S103), the main control unit 51 sets the range Rm2 (step S104). In this configuration, based on the failure of acquisition of three-dimensional data using the phase shift method, the range Rm2 in which the phase shift method does not function effectively is actually measured. As a result, the range Rm2 in which the phase shift method does not function effectively can be accurately determined, and appropriate measures can be taken for the workpiece 10 including the range Rm2.

[0061] Furthermore, range Rm1 is a range (the range where the board surface 11A and the component 13 are present) where the light L irradiated from the projector 33 is diffusely reflected, and range Rm2 is a range (the range where the solder 15 is present) where the light L irradiated from the projector 33 is specularly reflected. With this configuration, the three-dimensional data Ds of the range Rm1 of the workpiece 10 where the light is diffusely reflected can be efficiently obtained by the phase shift method, and the three-dimensional data Dl of the range Rm2 of the workpiece 10 where the light L is specularly reflected can be accurately obtained by the light section method.

[0062] The workpiece 10 also has a substrate 11, a component 13 placed on the substrate 11, and solder 15 that joins the component 13 to the substrate 11. Range Rm1 is a range where the solder 15 is not present, and range Rm2 is a range where the solder 15 is present. With this configuration, three-dimensional data Ds of the range Rm1 of the workpiece 10 where the solder 15 is not present can be efficiently obtained by the phase shift method, and three-dimensional data Dl of the range Rm2 of the workpiece 10 where the solder 15 is present can be accurately obtained by the optical section method.

[0063] Furthermore, the projectors 33 are capable of projecting both the sine wave pattern Ls and the line pattern Ll. In response to this, the main control unit 51 controls one projector 33 to project the sine wave pattern Ls onto the range Rm1 (step S102), and controls the one projector 33 to project the line pattern Ll onto the range Rm2 (step S104). With this configuration, the projection of the sine wave pattern Ls and the projection of the line pattern Ll can be efficiently performed by the same projector 33.

[0064] In the above embodiment, the three-dimensional data Ds acquired by the phase shift method and the three-dimensional data Dl acquired by the light-section method are combined. In this case, if there is a discrepancy in the height references used when acquiring the three-dimensional data Ds and the three-dimensional data Dl, it is difficult to properly combine them. Therefore, height calibration as shown in FIG. 10 may be performed.

[0065] Fig. 10 is a flowchart showing an example of height calibration, and Fig. 11 is a diagram showing a schematic diagram of the operations executed in the height calibration of Fig. 10. The flowchart of Fig. 10 is executed under the control of the main control unit 51 in accordance with the board inspection program 91.

[0066] This height calibration is performed using a reference plate 19 held horizontally by the transfer stage 2. The upper surface of this reference plate 19 is a horizontal reference plane 191. In step S201, the drive control unit 53 controls the drive mechanism 4 to make the reference plane 191 of the reference plate 19 held by the transfer stage 2 overlap with the imaging field of view R3.

[0067] In step S202, as shown in the "Phase Shift Method" section of FIG. 11 , the projector 33 sequentially projects sine wave patterns Ls(N) onto the reference plane 191 (N = 1, 2, 3, 4). Then, each time a sine wave pattern Ls(N) is projected onto the reference plane 191, the camera 31 captures the sine wave pattern Ls(N). In this manner, four sine wave pattern images Is(1), Is(2), Is(3), and Is(4) are acquired by capturing the sine wave patterns Ls(1), Ls(2), Ls(3), and Ls(4) that are out of phase with one another. Then, the main control unit 51 calculates the height Hs (position in the Z direction) of the reference plane 191 based on the sine wave pattern images Is(1), Is(2), Is(3), and Is(4).

[0068] In step S203, as shown in the "Light-section method" section of FIG. 11 , the projector 33 projects a line pattern Ll onto the reference plane 191. As described above, the projector 33 changes the position at which the line pattern Ll is projected in the X direction by the width of the line pattern Ll. The main control unit 51 also acquires a plurality of line pattern images Il by having the camera 31 capture images of the plurality of line patterns Ll projected at different positions in the X direction. The main control unit 51 then calculates the height Hl (position in the Z direction) of the reference plane 191 based on the plurality of line pattern images Il.

[0069] In step S204, the main control unit 51 stores the difference between the height Hs of the reference plane 191 acquired by the phase shift method and the height Hl of the reference plane 191 acquired by the light-section method as an offset in the storage unit 54. Then, in the data synthesis of the three-dimensional measurement executed thereafter (step S105), the difference in height between the three-dimensional data Ds and the three-dimensional data Dl is corrected by the offset, thereby adjusting the heights of the three-dimensional data Ds and the three-dimensional data Dl, and then the three-dimensional data Ds and the three-dimensional data Dl are synthesized.

[0070] In the height calibration shown in FIG. 10 , the transfer stage 2 (workpiece holder) holds a reference plate 19 having a reference plane 191 so that the reference plane 191 is horizontal. The main control unit 51 then causes the projector 33 to project a sine wave pattern Ls(N) and the camera 31 to capture the sine wave pattern Ls(N) projected onto the reference plane 191, thereby acquiring sine wave pattern images Is(1), Is(2), Is(3), and Is(4) (periodic pattern reference images). The main control unit 51 also acquires the height of the reference plane 191 using the phase shift method based on the sine wave pattern images Is(1), Is(2), Is(3), and Is(4) (step S202). The main control unit 51 then causes the projector 33 to project a line pattern Ll and causes the camera 31 to capture the line pattern Ll projected onto the reference plane 191, thereby acquiring a line pattern image Il (line pattern reference image). The main control unit 51 also acquires the height of the reference plane 191 by the light-section method based on the line pattern image Il (step S203). The main control unit 51 then generates composite three-dimensional data Dt while correcting the difference between the height Hs of the reference plane 191 acquired by the phase shift method and the height Hs of the reference plane 191 acquired by the light-section method (step S105). With this configuration, the difference in the Z direction (height direction) between the three-dimensional data Ds acquired by the phase shift method and the three-dimensional data Dl acquired by the light-section method can be corrected, thereby appropriately generating the composite three-dimensional data Dt.

[0071] In the light-section method in the above example, the three-dimensional data Dl is obtained based on a single line pattern Ll projected by the projector 33. However, the light-section method may also be performed to obtain the three-dimensional data Dl by simultaneously projecting multiple line patterns Ll from the projector 33. In this case, the intervals between the multiple line patterns Ll may be set as follows:

[0072] 12 is a diagram showing a schematic diagram of the relationship between the spacing between two line patterns simultaneously irradiated by the light-section method and the shape of the workpiece. In FIG. 12, the tallest (in other words, thickest) component 13 of all components 13 mounted on the board surface 11A is shown. In other words, the position of the top surface 13A of the component 13 is the maximum height Zh of the workpiece 10. Furthermore, the position of the board surface 11A is the minimum height Zl of the workpiece 10.

[0073] 12, the line pattern Ll1 is projected onto the upper surface 13A of the component 13 and the board surface 11A so as to overlap, and the line pattern Ll1 projected onto the upper surface 13A of the component 13 and the line pattern Ll1 projected onto the board surface 11A are shifted in the X direction by a shift amount ΔLl. This shift amount ΔLl and the angle θ that the line pattern Ll1 forms with respect to the horizontal direction satisfy the following relational expressions: (Zh-Zl) / ΔLl=tan θ ΔLl=(Zh-Zl) / tan θ The same applies to the line pattern Ll2.

[0074] 12, the line pattern Ll2 irradiated onto the upper surface 13A of the component 13 is located between the line pattern Ll1 irradiated onto the upper surface 13A of the component 13 and the line pattern Ll1 irradiated onto the board surface 11A in the X direction. Therefore, in the line pattern image Il obtained by capturing the two line patterns Ll1 and Ll2 irradiated simultaneously, it is difficult to distinguish the portions that make up the same line pattern Ll. This occurs because the distance Pl in the X direction between the two line patterns Ll1 and Ll2 irradiated onto the workpiece 10 is narrow.

[0075] 12 , ignoring the width of the line pattern Ll, the following inequality 1 Pl>ΔLl=(Zh-Zl) / tan θ (Inequality 1) needs to be satisfied. Furthermore, taking into account the width Lw of the line pattern Ll in the X direction, the following inequality 2 Pl>ΔLl=(Zh-Zl) / tan θ+Lw (Inequality 2) needs to be satisfied. Therefore, the imaging control unit 52 controls the interval Pl between the two line patterns Ll1 and Ll2 projected by the projector 33 so as to satisfy inequality 2.

[0076] Various methods for acquiring the maximum height Zh and the minimum height Zl are possible. For example, the main control unit 51 determines the maximum height Zh and the minimum height Zl based on the three-dimensional data Ds acquired by the phase shift method (step S102), which is executed before the light-section method (step S104), and stores the determined maximum height Zh and minimum height Zl in the storage unit 54. The light-section method in step S104 uses the maximum height Zh and minimum height Zl stored in the storage unit 54. Alternatively, the main control unit 51 may determine the maximum height Zh and minimum height Zl in advance based on board data or the like indicating the configuration of the workpiece 10, and store the determined maximum height Zh and minimum height Zl in the storage unit 54.

[0077] In this way, the maximum height Zh and minimum height Zl of the workpiece 10 in the Z direction (height direction) are stored in the memory unit 54 (height information memory unit). The main control unit 51 also causes the projector 33 to simultaneously project line patterns Ll1 and Ll2, each extending in the Y direction (extension direction), at an interval Pl in the X direction (orthogonal direction) (step S104). The main control unit 51 then causes the camera 31 to capture the line pattern Ll1 (first line pattern) projected onto the workpiece 10, thereby acquiring a line pattern image Il1 (first line pattern image). The main control unit 51 then acquires three-dimensional data Dl indicating the three-dimensional shape of the workpiece 10 by the light-section method based on the line pattern image Il1 (step S104). Furthermore, the main control unit 51 acquires a line pattern image Il2 (second line pattern image) by having the camera 31 capture the line pattern Ll2 (second line pattern) projected onto the workpiece 10, and acquires three-dimensional data Dl representing the three-dimensional shape of the workpiece 10 by the light-section method based on the line pattern image Il2 (step S104). At this time, the main control unit 51 controls the projector 33 so that the distance Pl between the line patterns Ll1 and Ll2 in the X direction is wider than the distance ΔLl in the X direction between the position of one line pattern Ll at the maximum height Zh and the position of the line pattern Ll at the minimum height Zl. In other words, this distance Pl corresponds to the distance in the X direction between the positions of the line patterns Ll1 and Ll2 at the same height. With this configuration, the line patterns Ll1 and Ll2 are simultaneously projected onto the range Rm2, and the three-dimensional data Dl of the range Rm2 can be efficiently acquired by the light-section method. Moreover, the line pattern Ll1 and the line pattern Ll2 are sufficiently spaced apart in the X direction to accurately distinguish them, and the three-dimensional data Dl of the range Rm2 can be appropriately acquired.

[0078] As described above, the light-section method is performed while moving the position of the line pattern Ll in the X direction. When multiple line patterns Ll are irradiated simultaneously, it is preferable to move the positions of the multiple line patterns Ll in the X direction while keeping the intervals Pl between the multiple line patterns Ll constant.

[0079] In the three-dimensional measurement shown in Fig. 7, the phase shifting method is performed first, followed by the light sectioning method. However, the phase shifting method and the light sectioning method can also be performed in parallel (Figs. 13A and 13B).

[0080] 13A is a diagram schematically illustrating a first example of parallel measurement in which the phase shift method and the light section method are performed in parallel. In the example of FIG. 13A, it is assumed that ranges Rm1 and Rm2 are calculated in advance and stored in the storage unit 54.

[0081] In the example of FIG. 13A , the projector 33 projects a sine wave pattern Ls(N) onto the range Rm1, while the camera 31 captures the sine wave pattern Ls(N) to acquire a sine wave pattern image Is(N). The main control unit 51 then acquires three-dimensional data Ds representing the three-dimensional shape of the workpiece 10 (board surface 11A and components 13) within the range Rm1 based on the sine wave pattern image Is(N) using a phase shift method. The projector 33 also projects a line pattern Ll onto the range Rm2, while the camera 31 captures the line pattern Ll to acquire a line pattern image Il. The main control unit 51 then acquires three-dimensional data Dl representing the three-dimensional shape of the workpiece 10 (solder 15) within the range Rm2 using a light-section method based on the line pattern image Il. Here, the period during which the projector 33 projects the sine wave pattern Ls overlaps with the period during which the projector 33 projects the line pattern Ll.

[0082] 13A, similarly to the example of FIG. 8A, the area Rm2 in which the solder 15 exists is located inside the area Rm1 surrounding the solder 15 and is surrounded by this area Rm1. Therefore, the sine wave pattern Ls(N) projected from the projector 33 onto the area Rm1 is projected not only onto the area Rm1 but also onto the area Rm2. Furthermore, the line pattern Ll projected from the projector 33 onto the area Rm2 is projected not only onto the area Rm2 but also onto the area Rm1.

[0083] To distinguish between the sine wave pattern Ls and the line pattern Ll, the projector 33, in response to a command from the imaging control unit 52, illuminates the sine wave pattern Ls with light of a first wavelength and illuminates the line pattern Ll with light of a second wavelength different from the first wavelength. For example, one of the sine wave pattern Ls and the line pattern Ll is illuminated with blue light and the other with red light. The camera 31 also captures the sine wave pattern image Is(N) and the line pattern image Il as color images. The main control unit 51 then distinguishes the sine wave pattern Ls indicated by the sine wave pattern image Is based on color, and distinguishes the line pattern Ll indicated by the line pattern image Il based on color.

[0084] 13A , the main control unit 51 causes the projector 33 to project a line pattern Ll onto the range Rm2 during a period in which the projector 33 is projecting a sine wave pattern Ls onto the range Rm1. With this configuration, the 3D data Ds and Dl for the range Rm1 and the range Rm2 can be efficiently acquired while simultaneously projecting the sine wave pattern Ls and the line pattern Ll.

[0085] Furthermore, range Rm2 is located inside range Rm1 and is surrounded by range Rm1. In response to this, the projector 33 irradiates ranges Rm1 and Rm2 with a sine wave pattern Ls generated using light of a first wavelength in an overlapping manner, and irradiates range Rm2 with a line pattern Ll generated using light of a second wavelength different from the first wavelength in an overlapping manner. The main control unit 51 then acquires a sine wave pattern image Is(N) by having the camera 31 capture the sine wave pattern Ls of the first wavelength irradiated onto range Rm1, and acquires a line pattern image Il by having the camera 31 capture the line pattern Ll of the second wavelength irradiated onto range Rm2. In this configuration, the main control unit 51 can distinguish between the sine wave pattern Ls and the line pattern Ll, which are irradiated and imaged simultaneously, based on the differences in their respective wavelengths (colors), and obtain three-dimensional data Ds of the range Rm1 from the sine wave pattern Ls irradiated onto the range Rm1, and obtain three-dimensional data Dl of the range Rm2 from the line pattern Ll irradiated onto the range Rm2.

[0086] 13B is a diagram schematically illustrating a second example of parallel measurement in which the phase shift method and the light section method are performed in parallel. In the example of FIG. 13B, it is assumed that ranges Rm1 and Rm2 are calculated in advance and stored in the storage unit 54.

[0087] In the example of FIG. 13B , the projector 33A projects a sine wave pattern Ls(N) onto the range Rm1, while the camera 31 captures the sine wave pattern Ls(N) to acquire a sine wave pattern image Is(N). The main control unit 51 then acquires three-dimensional data Ds representing the three-dimensional shape of the workpiece 10 (board surface 11A and components 13) within the range Rm1 based on the sine wave pattern image Is(N) using a phase shift method. Furthermore, a projector 33B, which is different from the projector 33A, projects a line pattern Ll onto the range Rm2, while the camera 31 captures the line pattern Ll to acquire a line pattern image Il. The main control unit 51 then acquires three-dimensional data Dl representing the three-dimensional shape of the workpiece 10 (solder 15) within the range Rm2 using a light-section method based on the line pattern image Il. Here, the period during which the projector 33A projects the sine wave pattern Ls overlaps with the period during which the projector 33B projects the line pattern Ll.

[0088] 13B, unlike the example of FIG. 8A, the range Rm2 where the solder 15 exists is not surrounded by the range Rm1, but is located outside the range Rm1. Therefore, the range Rm2 exists outside the range where the projector 33A projects the sine wave pattern Ls, and the projector 33B projects the line pattern Ll onto the range Rm2. Therefore, the sine wave pattern Ls and the line pattern Ll projected onto the workpiece 10 are separated from each other and do not overlap each other.

[0089] 13B , the main control unit 51 causes the projector 33 to project the line pattern Ll onto the range Rm2 during the period in which the projector 33 is causing the sine wave pattern Ls onto the range Rm1. With this configuration, it is possible to efficiently acquire the three-dimensional data Ds and Dl for the range Rm1 and the range Rm2, respectively, while simultaneously emitting the sine wave pattern Ls and the line pattern Ll.

[0090] Furthermore, a projector 33A (first projector) and a projector 33B (second projector) are provided. The main control unit 51 controls the projector 33A to project a sine wave pattern Ls onto a range Rm1, and also controls the projector 33B to project a line pattern Ll onto a range Rm2 outside the range onto which the projector 33A projects the sine wave pattern Ls. In this configuration, the sine wave pattern Ls and the line pattern Ll, which are projected simultaneously, are spaced apart on the workpiece 10, so that three-dimensional data Ds of the range Rm1 can be obtained from the sine wave pattern Ls projected onto the range Rm1, and three-dimensional data Dl of the range Rm2 can be obtained from the line pattern Ll projected onto the range Rm2.

[0091] Fig. 14 is a flowchart showing a second example of three-dimensional measurement. The flowchart of Fig. 14 is executed under the control of the main control unit 51 in accordance with the substrate inspection program 91. Note that parts common to the second example of Fig. 14 and the first example of Fig. 7 are given the same reference numerals, and explanations thereof will be omitted as appropriate.

[0092] In step S121, the main control unit 51 determines whether the current three-dimensional measurement is the first, based on whether or not three-dimensional measurement has been performed on a workpiece 10 of the same type as the workpiece 10 that is held on the transfer stage 2 and is the target of the current three-dimensional measurement. If there is no such record and the current three-dimensional measurement is the first for the corresponding type (if "YES" in step S121), the main control unit 51 executes steps S101 to S106.

[0093] In step S101, the inspection target range Rt of the workpiece 10 is positioned within the imaging field of view R3. As shown in FIG. 8A , the inspection target range Rt is set so as to overlap with the board 11, the component 13, and the solder 15. Then, the main control unit 51 acquires the three-dimensional shape of the inspection target range Rt using the phase shift method (step S102). This acquires three-dimensional data Ds of the workpiece 10 in the range Rm1. In step S111, the main control unit 51 confirms the maximum height Zh and minimum height Zl of the workpiece 10 based on the three-dimensional data Ds and stores them in the memory unit 54.

[0094] In step S103, the main control unit 51 determines whether or not a failure range Rf exists, which is a range in which the calculation to obtain the three-dimensional data Ds of the inspection target range Rt has failed. If it is determined that a failure range Rf does not exist ("NO" in step S103), the main control unit 51 proceeds to step S106, where it determines whether the inspection has ended. However, in this example, the inspection target range Rt overlaps with the solder 15. Therefore, the calculation to obtain the three-dimensional data Ds of the inspection target range Rt is successful in the range where the solder 15 does not exist, but fails in the range where the solder 15 does exist. Therefore, the main control unit 51 determines that a failure range Rf exists ("YES" in step S103) and proceeds to step S112.

[0095] In step S112, the main control unit 51 sets the interval Pl between the multiple line patterns Ll to be projected onto the workpiece 10 by the projector 33 based on the maximum height Zh and the minimum height Zl. In other words, the interval Pl is set so as to satisfy the above inequality 2. Then, while simultaneously projecting the multiple line patterns Ll, ​​the main control unit 51 acquires the three-dimensional shape of the workpiece 10 in the failure range Rf (in other words, range Rm2) by the light-section method (step S104). This allows three-dimensional data Ds of the workpiece 10 in the failure range Rf (range Rm2) to be acquired.

[0096] In this way, three-dimensional data Ds indicating the three-dimensional shape of the workpiece 10 (i.e., the board surface 11A and the components 13) in range Rm1 and three-dimensional data Dl indicating the three-dimensional shape of the workpiece 10 (i.e., the solder 15) in range Rm2 are obtained. In step S105, the main control unit 51 combines the three-dimensional data Ds and the three-dimensional data Dl to generate combined three-dimensional data Dt indicating the three-dimensional shape of the workpiece 10 (i.e., the board surface 11A, the components 13, and the solder 15) in the inspection target range Rt. In the following step S106, in the same manner as above, the main control unit 51 determines whether or not to end the inspection.

[0097] If the main control unit 51 determines in step S121 that three-dimensional measurement has been performed on a workpiece 10 of the same type as the workpiece 10 to be measured this time (if the determination is "NO"), then the main control unit 51 determines in step S122 whether measurement using the light-section method is necessary for the current three-dimensional measurement (step S122). If it is confirmed that no failure range Rf occurred in the phase-shift method measurement performed in the previous three-dimensional measurement, it is determined that the light-section method is unnecessary ("NO" in step S122), and the main control unit 51 executes steps S101 to S106 as described above. On the other hand, if it is confirmed that a failure range Rf occurred in the phase-shift method measurement performed in the previous three-dimensional measurement, it is determined that the light-section method is necessary ("NO" in step S122), and the main control unit 51 executes steps S123 to S127.

[0098] In step S123, the inspection range Rt of the workpiece 10 is positioned within the imaging field of view R3. As shown in Fig. 8A, the inspection range Rt is set so as to overlap the board 11, components 13, and solder 15. The main control unit 51 also sets the maximum height Zh and minimum height Zl based on the maximum height Zh and minimum height Zl indicated by the three-dimensional data Ds measured in the previous three-dimensional measurement (step S124). In other words, the interval Pl is set so as to satisfy the above inequality 2.

[0099] In step S125, the main control unit 51 executes the phase shift method and the light section method in parallel while irradiating the workpiece 10 with the sine wave pattern Ls and the line pattern Ll. Specifically, as described above with reference to FIG. 13A, three-dimensional data Ds of the workpiece 10 in the range Rm1 and three-dimensional data Dl of the workpiece 10 in the range Rm2 (failure range Rf) are acquired. In step S126, the main control unit 51 combines the three-dimensional data Ds and the three-dimensional data Dl to generate combined three-dimensional data Dt representing the three-dimensional shape of the workpiece 10 (i.e., the board surface 11A, the components 13, and the solder 15) in the inspection target range Rt. In the following step S127, similar to step S106, the main control unit 51 determines whether to end the inspection.

[0100] Note that in the description here, as in the example of Fig. 13A, a case has been shown in which the sine wave pattern Ls and the line pattern Ll are projected from the same projector 33. However, as in the example of Fig. 13B, even in the case in which the sine wave pattern Ls is projected from the projector 33A and the line pattern Ll is projected from the projector 33B, the three-dimensional measurement of Fig. 14 can be similarly performed.

[0101] In the embodiment described above, the mounted board inspection device 1 corresponds to an example of a "three-dimensional measuring device" of the present invention, the workpiece 10 corresponds to an example of a "workpiece" of the present invention, the board 11 corresponds to an example of a "board" of the present invention, the component 13 corresponds to an example of a "component" of the present invention, the solder 15 corresponds to an example of a "solder" of the present invention, the reference plate 19 corresponds to an example of a "reference member" of the present invention, the reference plane 191 corresponds to an example of a "reference plane" of the present invention, the transfer stage 2 corresponds to an example of a "workpiece holding unit" of the present invention, the camera 31 corresponds to an example of an "imaging unit" of the present invention, and the projector 33 corresponds to an example of a "projector" of the present invention. the projector 33 corresponds to an example of a "light irradiation unit" of the present invention, the projector 33 corresponds to an example of a "projector" of the present invention, the projector 33A corresponds to an example of a "first projector" of the present invention, the projector 33B corresponds to an example of a "second projector" of the present invention, the main control unit 51 corresponds to an example of a "phase shift measurement control unit" of the present invention, the main control unit 51 corresponds to an example of a "light section measurement control unit" of the present invention, the main control unit 51 corresponds to an example of a "data synthesis unit" of the present invention, the main control unit 51 corresponds to an example of a "computer" of the present invention, and the storage unit 54 corresponds to a "height information storage unit" of the present invention. ", the storage unit 54 corresponds to an example of a "measurement range storage unit" of the present invention, the server computer 9 corresponds to an example of a "recording medium" of the present invention, the substrate inspection program 91 corresponds to an example of a "three-dimensional measurement program" of the present invention, the three-dimensional data Ds and the three-dimensional data Dl correspond to an example of "three-dimensional data" of the present invention, the composite three-dimensional data Dt corresponds to an example of "composite three-dimensional data" of the present invention, the light L corresponds to an example of "light" of the present invention, the sine wave pattern Ls corresponds to an example of a "periodic pattern" of the present invention, and the sine wave pattern image Is corresponds to an example of a "periodic pattern" of the present invention. line pattern image II corresponds to an example of a "line pattern image" of the present invention, line pattern image II1 corresponds to an example of a "first line pattern image" of the present invention, line pattern image II2 corresponds to an example of a "second line pattern image" of the present invention, line pattern Ll corresponds to an example of a "line pattern" of the present invention, line pattern Ll1 corresponds to an example of a "first line pattern" of the present invention, line pattern Ll2 corresponds to an example of a "second line pattern" of the present invention, range Rm1 corresponds to an example of a "first measurement range" of the present invention,The range Rm2 corresponds to an example of the "second measurement range" of the present invention, the maximum height Zh corresponds to an example of the "maximum height" of the present invention, the minimum height Zl corresponds to an example of the "minimum height" of the present invention, the Y direction corresponds to an example of the "extension direction" of the present invention, and the X direction corresponds to an example of the "orthogonal direction" of the present invention.

[0102] The present invention is not limited to the above embodiment, and various modifications can be made to the above without departing from the spirit of the present invention. For example, instead of specifying the failure range Rf, a range Rm2 in which the solder 15 exists may be stored in advance in the storage unit 54. In this case, in step S103, it is determined whether the range Rm2 is stored in the storage unit 54. If the range Rm2 is stored, three-dimensional data Dl indicating the three-dimensional shape of the workpiece 10 (i.e., the solder 15) in the range Rm2 is acquired by the light-section method (step S104).

[0103] That is, in this modified example, a memory unit 54 (measurement range memory unit) is provided that stores the range Rm2 (second measurement range). Then, the main control unit 51 (light-section measurement control unit) acquires three-dimensional data Dl in the range Rm2 stored in the memory unit 54. In this configuration, by storing the range Rm2 in which the phase shift method does not function effectively in the memory unit 54, it is possible to appropriately deal with the workpiece 10 that includes the range Rm2 in which the phase shift method does not function effectively.

[0104] Furthermore, when setting the interval Pl between multiple line patterns Ll simultaneously projected from one projector 33, a different projector 33 may be used. In this modification, three-dimensional data Ds of the workpiece 10 is acquired in advance by the phase shift method based on the sine wave pattern Ls projected from projector 33B, and the interval Pl is set based on the maximum height Zh and minimum height Zl obtained from the three-dimensional data Ds. Then, multiple line patterns Ll are projected onto the workpiece 10 from projector 33A at the interval Pl.

[0105] Furthermore, in the above-described light-section method, the projector 33 is controlled to change the position at which the projector 33 projects the line pattern Ll in the X direction within the range Rm2. However, the position of the line pattern Ll may be changed in the X direction relative to the range Rm2 by moving the transport stage 2 in the X direction using the drive mechanism 4. In this case, it is preferable to synchronize the movement of the transport stage 2 in the X direction with the imaging timing (exposure timing) of the solid-state imaging element 32 so that the positional relationship between the pixels of the solid-state imaging element 32 that capture the sine wave pattern Ls when performing the phase shift method and the pixels of the solid-state imaging element 32 that capture the line pattern Ll when performing the light-section method does not shift in the X direction.

[0106] Furthermore, there are cases where the component 13 is a mirror-finished component, or where the electrodes of the component 13 are mirror-finished electrodes that specularly reflect light. When such a mirror-finished component or mirror-finished electrode is included in the workpiece 10, it may be impossible to obtain the three-dimensional data Ds indicating the three-dimensional shape of the mirror-finished component or mirror-finished electrode using the phase shift method. In such cases, a range Rm2 may be set for the mirror-finished component or mirror-finished electrode, and the three-dimensional data Dl may be obtained using the light-section method in the same manner as described above.

[0107] Furthermore, the magnitude relationship between ranges Rm1 and Rm2 is not limited to the above example, and range Rm2 may be wider than range Rm1.

[0108] Furthermore, the number and arrangement of the cameras 31 are not limited to the example in Fig. 1. Similarly, the number and arrangement of the projectors 33 are not limited to the example in Fig. 1.

[0109] REFERENCE SIGNS LIST 1...Mounted board inspection device 10...Workpiece 11...Board 13...Component 15...Solder 19...Reference plate 191...Reference plane 2...Transfer stage 31...Camera 33, 33A, 33B...Projector 51...Main control unit 54...Memory unit 9...Server computer 91...Board inspection program Ds...Three-dimensional data Dl...Three-dimensional data Dt...Synthesized three-dimensional data L...Light Ls...Sine wave pattern Is...Sine wave pattern image Il...Line pattern image Ll, Ll1, Ll2...Line pattern Rm1...Range (first measurement range) Rm2...Range (second measurement range) Zh...Maximum height Zl...Minimum height Y...Y direction X...X direction

Claims

a phase shift measurement control unit that causes the light irradiation unit to irradiate a periodic pattern, which is a periodically changing light pattern, and causes the imaging unit to capture the periodic pattern irradiated onto the workpiece, thereby acquiring a periodic pattern image, and acquires three-dimensional data showing a three-dimensional shape of the workpiece by a phase shift method based on the periodic pattern image; a light section measurement control unit that causes the light irradiation unit to irradiate a line pattern, which is a linearly extending light pattern, and causes the imaging unit to capture the line pattern irradiated onto the workpiece, thereby acquiring a line pattern image, and acquires three-dimensional data showing the three-dimensional shape of the workpiece by a light section method based on the line pattern image; and a data synthesis unit that synthesizes the three-dimensional data acquired by the phase shift measurement control unit and the three-dimensional data acquired by the light section measurement control unit, to generate synthetic three-dimensional data.

2. The three-dimensional measuring device of claim 1, wherein the phase shift measurement control unit causes the light irradiation unit to irradiate the periodic pattern onto a first measurement range of the workpiece, thereby acquiring three-dimensional data indicating the three-dimensional shape of the workpiece in the first measurement range; the light section measurement control unit causes the light irradiation unit to irradiate the line pattern onto a second measurement range of the workpiece different from the first measurement range, thereby acquiring three-dimensional data indicating the three-dimensional shape of the workpiece in the second measurement range; and the data synthesis unit synthesizes the three-dimensional data for the first measurement range acquired by the phase shift measurement control unit and the three-dimensional data for the second measurement range acquired by the light section measurement control unit to generate the synthesized three-dimensional data.

3. A three-dimensional measuring device according to claim 2, wherein the second measurement range is narrower than the first measurement range.

4. A three-dimensional measuring device as described in claim 2 or 3, further comprising a measurement range memory unit that stores the second measurement range, wherein the light section measurement control unit acquires three-dimensional data in the second measurement range stored in the measurement range memory unit.

5. A three-dimensional measuring device according to claim 2 or 3, wherein the light section measurement control unit sets a range in which the phase shift measurement control unit has failed to acquire three-dimensional data as the second measurement range.

6. A three-dimensional measuring device as described in claim 5, wherein the phase shift measurement control unit causes the light irradiation unit to irradiate the periodic pattern for both the first measurement range and the second measurement range, and the light section measurement control unit sets the second measurement range based on the success or failure of the result of acquiring three-dimensional data indicating the three-dimensional shape of the workpiece in the first measurement range and the second measurement range.

7. The three-dimensional measuring device according to any one of claims 1 to 6, wherein the work holding unit holds a reference member having a reference plane so that the reference plane is horizontal; the phase shift measurement control unit acquires a periodic pattern reference image by irradiating the periodic pattern onto the light irradiation unit and causing the imaging unit to image the periodic pattern irradiated onto the reference plane, and acquires the height of the reference plane by a phase shift method based on the periodic pattern reference image; the light section measurement control unit acquires a line pattern reference image by irradiating the line pattern onto the reference plane with the light irradiation unit and causing the imaging unit to image the line pattern irradiated onto the reference plane, and acquires the height of the reference plane by a light section method based on the line pattern reference image; and the data synthesis unit generates the synthetic three-dimensional data while correcting a difference between the height of the reference plane acquired by the phase shift measurement control unit and the height of the reference plane acquired by the light section measurement control unit.

8. The three-dimensional measuring device according to any one of claims 1 to 7, further comprising a height information storage unit which stores a maximum height and a minimum height of the workpiece in a height direction, wherein the light section measurement control unit causes the light irradiation unit to simultaneously irradiate a first line pattern and a second line pattern, each of which is the line pattern extending in a predetermined extension direction, with a gap between them in an orthogonal direction perpendicular to the extension direction, and acquires a first line pattern image by having the imaging unit capture the first line pattern irradiated onto the workpiece, and acquires three-dimensional data indicating a three-dimensional shape of the workpiece by a light section method based on the first line pattern image, and causes the imaging unit to capture the second line pattern irradiated onto the workpiece, to acquire a second line pattern image, and acquires three-dimensional data indicating the three-dimensional shape of the workpiece by the light section method based on the second line pattern image, wherein the light section measurement control unit controls the light irradiation unit so that a gap between the first line pattern and the second line pattern in the orthogonal direction is wider than a gap in the orthogonal direction between a position at the maximum height and a position at the minimum height of the line pattern.

9. The three-dimensional measuring device of claim 2 or 3, wherein the light irradiation unit has a projector capable of irradiating both the periodic pattern and the line pattern, the phase shift measurement control unit controls the projector to irradiate the periodic pattern onto the first measurement range, and the light section measurement control unit controls the projector to irradiate the line pattern onto the second measurement range.

10. A three-dimensional measuring device as described in claim 2 or 3, wherein during a period in which the phase shift measurement control unit causes the light irradiation unit to irradiate the periodic pattern onto the first measurement range, the light section measurement control unit causes the light irradiation unit to irradiate the line pattern onto the second measurement range.

11. The three-dimensional measuring device of claim 10, wherein the second measurement range is located inside the first measurement range and is surrounded by the first measurement range, the light irradiation unit irradiates the first measurement range and the second measurement range with the periodic pattern generated by light of a first wavelength so as to overlap, and irradiates the second measurement range with the periodic pattern generated by light of a second wavelength different from the first wavelength so as to overlap, the phase shift measurement control unit acquires the periodic pattern image by causing the imaging unit to capture the light of the first wavelength irradiated to the first measurement range, and the light section measurement control unit acquires the line pattern image by causing the imaging unit to capture the light of the second wavelength irradiated to the second measurement range.

12. The three-dimensional measuring device of claim 10, wherein the light irradiation unit has a first projector and a second projector, the phase shift measurement control unit causes the first projector to irradiate the periodic pattern onto the first measurement range, and the light section measurement control unit causes the second projector to irradiate the line pattern onto the second measurement range outside the range where the first projector is irradiating the periodic pattern.

13. A three-dimensional measuring device as described in claim 2 or 3, wherein the first measurement range is a range in which the light irradiated from the light irradiation unit is diffusely reflected, and the second measurement range is a range in which the light irradiated from the light irradiation unit is specularly reflected.

14. A three-dimensional measuring device as described in claim 2 or 3, wherein the workpiece has a board, a component placed on the board, and solder joining the component to the board, the first measurement range is a range in which the solder is not present, and the second measurement range is a range in which the solder is present.

15. A three-dimensional measurement method comprising the steps of: irradiating a workpiece with a periodic pattern, which is a periodically changing light pattern, capturing an image of the periodic pattern irradiated onto the workpiece to obtain a periodic pattern image, and obtaining three-dimensional data indicating the three-dimensional shape of the workpiece by a phase shift method based on the periodic pattern image; irradiating a workpiece with a line pattern, which is a linearly extending light pattern, capturing an image of the line pattern irradiated onto the workpiece to obtain a line pattern image, and obtaining three-dimensional data indicating the three-dimensional shape of the workpiece by a light section method based on the line pattern image; and combining the three-dimensional data obtained by the phase shift method and the three-dimensional data obtained by the light section method to generate combined three-dimensional data.

16. A three-dimensional measuring program for causing a computer to execute the three-dimensional measuring method according to claim 15.

17. A recording medium on which the three-dimensional measurement program according to claim 16 is recorded so as to be readable by a computer.

Images