Three-dimentional measuring apparatus and three-dimentional measuring method

Abstract

To make it possible to properly recognize an object to be recognized having high optical reflectance and an object to be recognized having low optical reflectance with the same measuring device even when both are included in one component.SOLUTION: A three-dimensional measuring apparatus includes: a projection device that projects pattern light having different cycles on a component having an electrode as a first object to be recognized and a second object to be recognized having lower optical reflectance than that of the electrode; an imaging device that separately images the first object to be recognized and the second object to be recognized on which the pattern light is projected; and a processing device that recognizes an image of the first object to be recognized on the basis of first image data of the first object to be recognized and recognizes an image of the second object to be recognized on the basis of second image data of the second object to be recognized. Measurement conditions including an imaging condition and an image recognition condition are different between the first object to be recognized and the second object to be recognized.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a three-dimensional measurement apparatus and a three-dimensional measurement method. [Background technology]

[0002] A phase shift method as disclosed in Patent Document 1 is known as a method for measuring three-dimensional objects. When measuring the three-dimensional shape of an object based on the phase shift method, a stripe pattern (pattern light) with a sinusoidal brightness distribution is projected onto the object while shifting the phase. The object onto which the stripe pattern is projected is imaged to obtain an image of the object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2001-124534 A Summary of the Invention [Problem to be solved by the invention]

[0004] One application of 3D measurement methods is the mounting of components on electronic devices. Recognition targets, such as the electrodes of components, are measured in 3D, and the components are mounted on boards and other devices based on the measurement results. Component electrodes have a high optical reflectance and are easy to recognize, but are prone to saturation (saturation of image brightness values) due to excessive optical reflection. On the other hand, components' resin parts, etc., have a lower optical reflectance than the electrodes, making it difficult to create contrast and therefore difficult to recognize. Therefore, when a single component contains recognition targets with high optical reflectance and those with low optical reflectance, it is necessary to provide separate dedicated measuring devices suitable for the recognition of each.

[0005] An aspect of the present invention aims to make it possible to appropriately recognize each recognition target using the same measurement device, even when a single component contains recognition targets with high light reflectance and recognition targets with low light reflectance. [Means for solving the problem]

[0006] According to a first aspect of the present invention, there is provided a three-dimensional measuring device comprising: a projection device that projects pattern light having different phases onto a part having a first recognition target and a second recognition target having a lower light reflectance than the first recognition target; an imaging device that separately images the first recognition target and the second recognition target onto which the pattern light is projected; and a processing device that performs image recognition of the first recognition target based on first image data of the first recognition target and image recognition of the second recognition target based on second image data of the second recognition target, wherein measurement conditions including imaging conditions and image recognition conditions are different for the first recognition target and the second recognition target.

[0007] According to a second aspect of the present invention, there is provided a three-dimensional measurement method comprising the steps of: projecting pattern light having different phases onto a part having a first recognition target and a second recognition target having a lower light reflectance than the first recognition target, and taking a first image of the first recognition target onto which the pattern light is projected; projecting the pattern light onto the part and taking a second image of the second recognition target onto which the pattern light is projected; and performing image recognition of the first recognition target based on first image data of the first recognition target, and performing image recognition of the second recognition target based on second image data of the second recognition target, wherein measurement conditions including imaging conditions and image recognition conditions are different for the first recognition target and the second recognition target. Effect of the Invention

[0008] According to an aspect of the present invention, even when one component includes a recognition target having a high optical reflectance and a recognition target having a low optical reflectance, each recognition target can be appropriately recognized by the same measurement device. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of an electronic component mounting system according to an embodiment. [Diagram 2] FIG. 2 is a schematic diagram illustrating an example of a three-dimensional measuring apparatus according to an embodiment. [Diagram 3]FIG. 3 is a schematic diagram illustrating an example of a three-dimensional measuring apparatus according to an embodiment. [Figure 4] FIG. 4 is a functional block diagram illustrating an example of a processing device according to the embodiment. [Diagram 5] FIG. 5 is a schematic diagram showing an example of pattern data according to the present embodiment. [Figure 6] FIG. 6A is a side view and FIG. 6B is a bottom view showing an example of a component according to an embodiment. [Figure 7] FIG. 7 is an explanatory diagram for explaining imaging of a component according to the embodiment. [Figure 8] FIG. 8 is a diagram showing a specific example of the measurement condition according to the embodiment. [Figure 9] FIG. 9 is a schematic diagram for explaining the image synthesis process according to the embodiment. [Figure 10] FIG. 10 is an explanatory diagram for explaining a recognition algorithm of the image recognition process executed by the processing device of the embodiment. [Figure 11] FIG. 11 is an explanatory diagram illustrating an example of the first complementation process of the image recognition by the processing device of the embodiment. [Figure 12] FIG. 12 is an explanatory diagram illustrating an example of the second complementation process of the image recognition by the processing device of the embodiment. [Figure 13] FIG. 13 is a flowchart showing the three-dimensional measurement method according to the embodiment. [Figure 14] FIG. 14 is a flowchart illustrating the operation of the electronic component mounting system according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings, but the present invention is not limited thereto. The components of the embodiments described below can be appropriately combined. In addition, some components may not be used.

[0011] In the following description, an XYZ orthogonal coordinate system is set, and the positional relationship of each part is described with reference to this XYZ orthogonal coordinate system. The direction parallel to the X-axis of a predetermined plane is defined as the X-axis direction, the direction parallel to the Y-axis perpendicular to the X-axis on the predetermined plane is defined as the Y-axis direction, and the direction parallel to the Z-axis perpendicular to the predetermined plane is defined as the Z-axis direction. The rotation direction around the Z-axis is defined as the θZ direction. In the following description, the predetermined plane is appropriately referred to as the XY plane. In this embodiment, the XY plane and the horizontal plane are parallel to each other.

[0012] [Embodiment] <Electronic component mounting system> An embodiment will now be described. Fig. 1 is a diagram that illustrates an example of an electronic component mounting system 100 according to an embodiment. Fig. 1 is a diagram of the electronic component mounting system 100 as viewed from the -Y direction.

[0013] 1, electronic component mounting system 100 includes a three-dimensional measuring device 1 and an electronic component mounting device 2. In electronic component mounting system 100, the three-dimensional measuring device 1 three-dimensionally measures the shape of a component C, and based on the measurement results, the electronic component mounting device 2 mounts the component C on a substrate P. In the following, the electronic component mounting device 2 will be described first, and then the three-dimensional measuring device 1 according to this embodiment will be described.

[0014] <Electronic component mounting equipment> The electronic component mounting apparatus 2 mounts a component C on a substrate P. The electronic component mounting apparatus 2 includes a mounting head 22 that holds a nozzle 11 that holds the component C, an electronic component supplying device 23 that supplies the component C, a substrate transporting device 24 that transports the substrate P on which the component C is to be mounted, and a control device 29 that controls the electronic component mounting apparatus 2.

[0015] The electronic component supplying device 23 has, for example, a moving mechanism that moves a tray holding a plurality of components C in a horizontal direction. The electronic component supplying device 23 supplies the components C to a supply position SP. The electronic component supplying device 23 may be a tape supply type component supplying device that supplies the components C held on a tape. In that case, the electronic component supplying device 23 includes a tape reel that supports the tape, and a feeder that transports the components C held on the tape unwound from the tape reel.

[0016] The board transport device 24 has a conveyor belt that transports the board P, and a clamp mechanism that fixes the position of the board P transported to the mounting position MP.

[0017] The mounting head 22 mounts the component C held by the nozzle 11 on the board P. The mounting head 22 is moved in each of the X-axis direction and the Y-axis direction by a head driving device 25. The nozzle 11 is moved in each of the Z-axis direction and the θZ direction relative to the mounting head 22 by a nozzle driving device 26 provided on the mounting head 22. The nozzle 11 can be moved in four directions, the X-axis direction, the Y-axis direction, the Z-axis direction, and the θZ direction, by the head driving device 25 and the nozzle driving device 26. The mounting head 22 can be moved between a supply position SP defined by the electronic component supply device 23 and a mounting position MP defined on the board P.

[0018] The mounting head 22 holds the component C supplied from the electronic component supply device 23 with the nozzle 11 at the supply position SP, and then transports it to the board P arranged at the mounting position MP. The mounting head 22 mounts the component C held by the nozzle 11 on the board P arranged at the mounting position MP.

[0019] The nozzle 11 detachably holds the component C. The nozzle 11 may be a suction nozzle that suctions and holds the component C, or a gripping nozzle that clamps and holds the component C.

[0020] After the mounting head 22 holds the component C with the nozzle 11 at the supply position SP, it places the component C at the measurement position TP of the three-dimensional measuring device 1 before transporting it to the substrate P. The measurement position TP of the three-dimensional measuring device 1 refers to a position where the pattern light PL (see FIG. 2) of the three-dimensional measuring device 1 can be irradiated.

[0021] The substrate transport device 24 is provided with a position measuring device 27 that measures the position of a through hole formed in the substrate P placed at the mounting position MP. Measurement data of the position measuring device 27 indicating the position of the through hole formed in the substrate P is output to the control device 29.

[0022] The control device 29 is connected to each of the head driving device 25, the nozzle driving device 26, the position measuring device 27, and a processing device 5 of the three-dimensional measuring device 1, which will be described later. Before mounting the component C on the substrate P, the control device 29 measures the three-dimensional shape of the component C using the three-dimensional measuring device 1. Based on the measurement results of the component C by the three-dimensional measuring device 1, the mounting head 22 adjusts the relative position between the component C and the substrate P, as will be described later, and mounts the component C on the substrate P.

[0023] <3D measuring device> As shown in Fig. 1, the three-dimensional measuring apparatus 1 includes a projection device 3 that projects pattern light PL onto a part C, an imaging device 4 that images the part C onto which the pattern light PL is projected, and a processing device 5 that performs image recognition based on the captured image data. In the example of Fig. 1, the three-dimensional measuring apparatus 1 is placed below (-Z direction) the measurement position TP. The three-dimensional measuring apparatus 1 is installed with the imaging direction facing upward (+Z direction) so as to image the bottom surface of the part C held by the nozzle 11.

[0024] The three-dimensional measuring device 1 is positioned on the movement path of the mounting head 22 between the supply position SP and the mounting position MP at a position where it can irradiate pattern light PL onto the component C held by the nozzle 11 and placed at the measurement position TP.

[0025] Fig. 2 is a schematic diagram showing an example of a three-dimensional measuring apparatus 1 according to an embodiment. Fig. 3 is a schematic diagram showing an example of a three-dimensional measuring apparatus 1 according to an embodiment. Fig. 2 is a diagram showing the three-dimensional measuring apparatus 1, which is a component of the three-dimensional measuring apparatus 1, as viewed from the same direction as Fig. 1. Fig. 3 is a diagram showing the three-dimensional measuring apparatus 1, which is a component of the three-dimensional measuring apparatus 1, as viewed from above in the vertical direction, i.e., from the +Z direction.

[0026] In the embodiment, the part C has a first recognition target R1 and a second recognition target R2 having a lower light reflectance than the first recognition target R1. The three-dimensional measuring device 1 performs three-dimensional measurement of both the first recognition target R1 and the second recognition target R2. Details of the part C, the first recognition target R1, and the second recognition target R2 will be described later.

[0027] 2, the projection device 3 projects pattern light PL having different phases onto a component C. The projection device 3 includes a light source 31 that generates light, a light modulation element 32 that optically modulates the light generated from the light source 31 to generate the pattern light PL, and an emission optical system 33 that emits the pattern light PL generated by the light modulation element 32.

[0028] The light modulation element 32 includes a digital mirror device (DMD). The light modulation element 32 may include a transmissive liquid crystal panel or a reflective liquid crystal panel. The light modulation element 32 generates the pattern light PL based on the pattern data output from the processing device 5. The projection device 3 emits the pattern light PL patterned based on the pattern data.

[0029] The projection device 3 projects the pattern light PL onto the component C from multiple directions separately. The projection device 3 includes a reflecting member 6 that changes the projection direction by reflecting the emitted light. The reflecting member 6 reflects the pattern light PL emitted from the projection device 3 and irradiates it onto the component C. The reflecting surface of the reflecting member 6 is flat.

[0030] The imaging device 4 has an imaging optical system 41 that forms an image of the pattern light PL reflected by the component C, and an imaging element 42 that acquires image data of the component C via the imaging optical system 41. The imaging element 42 is a solid-state imaging element including a CMOS image sensor (Complementary Metal Oxide Semiconductor Image Sensor) or a CCD image sensor (Charge Coupled Device Image Sensor).

[0031] In the embodiment, the imaging optical system 41 is a telecentric optical system. A telecentric optical system is an optical system in which the chief ray is parallel to the optical axis AX. Therefore, the light (chief ray) that enters the imaging field of view of the imaging device 4 and is imaged can be considered to be parallel to the optical axis AX.

[0032] In the embodiment, the imaging device 4 separately captures images of the first recognition target R1 and the second recognition target R2 onto which the pattern light PL is projected. The imaging device 4 captures the first recognition target R1 onto which the pattern light PL is projected, through the first imaging, to generate first image data 61 (see FIG. 7). The imaging device 4 captures the second recognition target R2 onto which the pattern light PL is projected, through the second imaging, to generate second image data 62 (see FIG. 7).

[0033] The processing device 5 includes a computer system and controls the projection device 3 and the imaging device 4. The processing device 5 has an arithmetic processing device including a processor such as a CPU (Central Processing Unit), a storage device including a memory and storage such as a ROM (Read Only Memory) or a RAM (Random Access Memory), and an input / output interface including an input / output circuit capable of inputting and outputting signals and data. The arithmetic processing device performs arithmetic processing according to a computer program stored in the storage device.

[0034] In the embodiment, the processing device 5 performs image recognition of the first recognition target R1 based on the first image data 61 of the first recognition target R1, and performs image recognition of the second recognition target R2 based on the second image data 62 of the second recognition target R2. In this specification, image recognition means acquiring at least one of the position and the shape of the recognition target by three-dimensional measurement. Performing image recognition includes performing three-dimensional measurement.

[0035] The three-dimensional measuring device 1 measures the three-dimensional shape of the recognition targets (first recognition target R1, second recognition target R2) of the part C held by the nozzle 11 and placed at the measurement position TP based on the pattern projection method. The projection device 3 irradiates the part C with pattern light PL, for example, stripe pattern light with a sinusoidal brightness distribution, while shifting the phase. As described above, the pattern light PL is irradiated onto the part C from multiple directions.

[0036] The imaging device 4 acquires image data of the part C onto which the pattern light PL is projected. The imaging device 4 acquires at least image data of the part C onto which the first pattern light PL1 is projected and image data of the part C onto which the second pattern light PL2 is projected. The incident direction of the first pattern light PL1 and the incident direction of the second pattern light PL2 incident on the part C are different.

[0037] 2, the first pattern light PL1 is pattern light PL emitted from the projection device 3 and directly irradiated onto the component C. The second pattern light PL2 is pattern light PL emitted from the projection device 3, reflected by the reflecting member 6, and irradiated onto the component C.

[0038] The processing device 5 controls the light modulation element 32 so as to change from one of a first irradiation state in which the first pattern light PL1 from the projection device 3 is irradiated onto the part C, to a second irradiation state in which the second pattern light PL2 from the reflecting member 6 is irradiated onto the part C, to the other.

[0039] When irradiating the first pattern light PL1 onto the component C, the processing device 5 controls the light modulation element 32 so that the pattern light PL is emitted from a first region 331 of the emission surface 33S of the emission optical system 33 and is not emitted from a second region 332 of the emission surface 33S. The pattern light PL emitted from the first region 331 is directly irradiated onto the component C as the first pattern light PL1 without passing through the reflecting member 6.

[0040] When irradiating the part C with the second pattern light PL2, the processing device 5 controls the light modulation element 32 so that the pattern light PL is emitted from the second region 332 of the emission surface 33S of the emission optical system 33 and is not emitted from the first region 331 of the emission surface 33S. The pattern light PL emitted from the second region 332 passes through the optical axis AX and is irradiated onto the reflecting member 6. The pattern light PL emitted from the second region 332 is irradiated onto the part C as the second pattern light PL2 via the reflecting member 6.

[0041] 1 and 2, the first region 331 is a half of the region of the emission surface 33S on the +X side of the optical axis of the emission optical system 33, and the second region 332 is a half of the region of the emission surface 33S on the -X side of the optical axis of the emission optical system 33. The pattern light PL emitted from the first region 331 is irradiated onto the component C in a state where it is held by the nozzle 11 and arranged at the measurement position TP. The pattern light PL emitted from the second region 332 is irradiated onto the reflecting surface of the reflecting member 6, reflected by the reflecting surface of the reflecting member 6, and then irradiated onto the component C in a state where it is held by the nozzle 11 and arranged at the measurement position TP.

[0042] 1 and 2, the projection device 3, the imaging device 4, and the reflecting member 6 are supported by a housing 7. The relative positions of the projection device 3, the imaging device 4, and the reflecting member 6 are fixed by the housing 7. The incident surface 41S of the imaging device 4 can face a component C arranged at a measurement position TP.

[0043] The reflecting member 6 is disposed at least partially around the optical axis AX of the imaging optical system 41 between the incident surface 41S and the component C disposed at the measurement position TP. The projection device 3 is disposed at least partially around the optical axis AX of the imaging optical system 41 between the incident surface 41S and the component C disposed at the measurement position TP. The reflecting member 6 is disposed at a position closer to the measurement position TP than the projection device 3.

[0044] 3, two projection devices 3 are arranged around the optical axis AX. In addition, two reflection members 6 are arranged around the optical axis AX.

[0045] The first reflecting member 6 is disposed on the -X side of the optical axis AX. The first projection device 3 is disposed on the +X side of the optical axis AX. The first projection device 3 can irradiate the first reflecting member 6 with the second pattern light PL2. The second pattern light PL2 reflected by the first reflecting member 6 is irradiated onto the component C disposed at the measurement position TP. In addition, the first projection device 3 can directly irradiate the first pattern light PL1 onto the component C disposed at the measurement position TP.

[0046] The second reflecting member 6 is disposed on the +Y side of the optical axis AX. The second projection device 3 is disposed on the -Y side of the optical axis AX. The second projection device 3 can irradiate the second reflecting member 6 with the second pattern light PL2. The second pattern light PL2 reflected by the second reflecting member 6 is irradiated onto the component C disposed at the measurement position TP. In addition, the second projection device 3 can directly irradiate the first pattern light PL1 onto the component C disposed at the measurement position TP.

[0047] In this way, the first and second projection devices 3 and the first and second reflecting members 6 can project the pattern light PL from four directions onto the part C placed at the measurement position TP. The imaging device 4 acquires image data of the part C onto which the first pattern light PL1 from the first projection device 3 is projected, image data of the part C onto which the second pattern light PL2 from the first reflecting member 6 is projected, image data of the part C onto which the first pattern light PL1 from the second projection device 3 is projected, and image data of the part C onto which the second pattern light PL2 from the second reflecting member 6 is projected.

[0048] As described above, in this embodiment, there are two pairs of projection devices 3 and reflecting members 6, one pair facing each other in the X-axis direction and one pair facing each other in the Y-axis direction, but the present invention is not limited to this and may have one pair, or three or more pairs. Furthermore, the three-dimensional measuring apparatus 1 may include multiple projection devices 3 provided independently (without reflecting members 6) for each projection direction.

[0049] In this embodiment, a three-dimensional measuring device 1 using a phase shift method including a projection device 3 consisting of a projection device 3 and a reflecting member 6, and an imaging device 4 is used as a preferred embodiment, but the present invention is not limited to this, and a form using an imaging image analysis method may also be used.

[0050] <Control device> 4 is a functional block diagram showing an example of a processing device 5 according to an embodiment. The processing device 5 includes an input / output unit 51, a pattern generating unit 52, an image data acquiring unit 53, a phase value calculating unit 54, and a three-dimensional shape calculating unit 55. The processing device 5 also includes a storage unit 56.

[0051] The input / output unit 51 is a functional unit that is realized by the input / output interface of the processing device 5 executing a 3D measurement program for causing the 3D measurement device 1 of the embodiment to execute the 3D measurement method of the embodiment.

[0052] The pattern generating unit 52, the image data acquiring unit 53, the phase value calculating unit 54 and the three-dimensional shape calculating unit 55 are functional units realized by the arithmetic processing unit of the processing device 5 executing a three-dimensional measuring program for the three-dimensional measuring device 1 according to the embodiment to execute the three-dimensional measuring method according to the embodiment. The arithmetic processing unit has a processor such as a CPU (Central Processing Unit) and a main memory including a non-volatile memory such as a ROM (Read Only Memory) and a volatile memory such as a RAM (Random Access Memory).

[0053] The storage unit 56 includes a non-volatile storage device such as a semiconductor memory. The storage unit 56 stores programs executed by the input / output unit 51, the pattern generation unit 52, the image data acquisition unit 53, the phase value calculation unit 54, and the three-dimensional shape calculation unit 55. The storage unit 56 stores part data which is design information of the part C. Information such as the shape and dimensions of the part C is recorded in the part data.

[0054] The pattern generation unit 52 generates pattern data. The pattern data generated by the pattern generation unit 52 is output to the light modulation element 32 via the input / output unit 51. The light modulation element 32 generates pattern light PL based on the pattern data generated by the pattern generation unit 52. The pattern data generated by the pattern generation unit 52 includes first pattern data for irradiating the part C with the first pattern light PL1 from the projection device 3 without the reflection member 6, and second pattern data for irradiating the part C with the second pattern light PL2 from the projection device 3 via the reflection member 6.

[0055] Fig. 5 is a schematic diagram showing an example of pattern data according to the present embodiment. In Fig. 5, a region that does not emit light is shown in black, and a region that emits light is shown in white. As shown in Fig. 5(A), when the first pattern light PL1 is irradiated onto the component C, the pattern generation unit 52 generates first pattern data and controls the light modulation element 32 so that the pattern light PL is emitted from the first region 331 of the emission surface 33S of the emission optical system 33 and the pattern light PL is not emitted from the second region 332 of the emission surface 33S.

[0056] As shown in Figure 5 (B), when the second pattern light PL2 is irradiated onto the component C, the pattern generation unit 52 generates second pattern data and controls the light modulation element 32 so that the pattern light PL is emitted from the second region 332 of the emission surface 33S of the emission optical system 33 and the pattern light PL is not emitted from the first region 331 of the emission surface 33S.

[0057] The image data acquisition unit 53 acquires image data from the imaging element 42 via the input / output unit 51. The image data acquisition unit 53 acquires image data of the part C onto which the first pattern light PL1 is projected by the first and second projection devices 3, and image data of the part C onto which the second pattern light PL2 from the first and second reflecting members 6 is projected.

[0058] The phase value calculation unit 54 calculates a phase value of each of the multiple pixels of the image data based on the luminance (pixel value) of the image data. The phase value calculation unit 54 calculates a phase value of a pixel of the image data corresponding to a same point based on the luminance of the same point of the multiple image data of the part C onto which each of the phase-shifted pattern lights PL is projected. The phase value calculation unit 54 calculates a phase value of each of the multiple pixels of the image data based on the luminance of the multiple points of the image data.

[0059] The three-dimensional shape calculation unit 55 calculates the position data of each of the multiple points of the part C corresponding to each of the multiple pixels of the image data based on the phase values ​​of each of the multiple pixels of the image data, and calculates the three-dimensional shape of the part C.

[0060] More specifically, the three-dimensional shape calculation unit 55 calculates the height (protruding length) of the first recognition target R1 and the second recognition target R2 based on the position data at the tip of the part C, and calculates the part height, which is the overall length of the part C in the vertical direction, as well as the height data of the first recognition target R1 and the height data of the second recognition target R2.

[0061] <Parts> 6A and 6B are a side view and a bottom view showing an example of a component C according to an embodiment. In one example, the component C is a connector component to be mounted on a substrate P. The component C has a body C1 and an electrode C2 extending from the body C1. The body C1 has a boss C3 for positioning and a screw hole C4 for fixing. The component C has a linear shape that is long in a predetermined direction as a whole.

[0062] The component C has two terminal portions C11 on which a plurality of electrodes C2 are arranged, and three fixing portions C12 for fixing the component C. The three fixing portions C12 are arranged in a straight line at intervals. One terminal portion C11 is arranged between the two fixing portions C12 at the center and one end, and another terminal portion C11 is arranged between the two fixing portions C12 at the center and the other end.

[0063] The body C1 is made of dark-colored resin. The boss C3 and the screw hole C4 are provided on the bottom surface C13 of the body C1. One boss C3 and one screw hole C4 are provided on each of the three fixing parts C12. The boss C3 is a cylindrical protrusion that protrudes downward from the bottom surface C13 of the body C1. The boss C3 is formed integrally with the body C1 and is made of dark-colored resin. The screw hole C4 is provided in the body C1 by fixing a metal female thread part to a mounting hole formed in the body C1.

[0064] The electrodes C2 are lead electrodes protruding from the body C1 of the component C. The electrodes C2 are made of metal. A large number of the electrodes C2 are arranged on each of the two terminal portions C11. Each electrode C2 protrudes diagonally downward from the bottom surface C13 of the body C1 at the terminal portion C11 so as to spread in the width direction of the component C. The tip (bottom end) of the electrode C2 is flat and approximately horizontal (parallel to the bottom surface C13 of the body C1) so as to be connectable to the wiring pattern of the substrate P.

[0065] In the embodiment, the first recognition target R1 is a tip portion of a lead electrode (i.e., electrode C2) protruding from a body C1 of a component C. The first recognition target R1 is a tip portion (lower surface portion) of each of the multiple electrodes C2. The second recognition target R2 is a tip portion of a boss C3 protruding from a body C1 made of a dark-colored resin. The second recognition target R2 is a tip portion (lower surface portion) of each of the three bosses C3.

[0066] In the example shown in FIG. 6, the part C is a long part extending in a predetermined direction, and the longitudinal dimension of the part C is larger than the field of view of the imaging device 4. Therefore, the three-dimensional measuring device 1 divides the entire part C into a plurality of regions, and performs image recognition (three-dimensional measurement) for each divided region. The number of divided regions of the part C depends on the size of the part C and the field of view size of the imaging device 4. FIG. 7 is an explanatory diagram for explaining imaging of the part C according to the embodiment. FIG. 7 shows an example in which the entire part C is divided into three divided regions. In FIG. 7, the entire part C is divided into divided regions CA1, CA2, and CA3. The divided region CA1 is about 1 / 3 of the region including one end of the part C. The divided region CA2 is about 1 / 3 of the region in the center of the part C. The divided region CA3 is about 1 / 3 of the region including the other end of the part C. The dashed rectangular portion shown in FIG. 7 indicates the field of view range 4S of the imaging device 4. In Figure 7, for ease of explanation, the field of view 4S is shown as being located in three locations: divided areas CA1, CA2, and CA3. In reality, however, component C is moved by mounting head 22 so that each of divided areas CA1, CA2, and CA3 falls within the field of view 4S.

[0067] The imaging device 4 separately images the first recognition target R1 and the second recognition target R2 with the divided area CA1 arranged within the visual field range 4S. The imaging device 4 separately images the first recognition target R1 and the second recognition target R2 with the divided area CA2 arranged within the visual field range 4S. The imaging device 4 separately images the first recognition target R1 and the second recognition target R2 with the divided area CA3 arranged within the visual field range 4S. As a result, first image data 61 of the first recognition target R1 in the divided areas CA1, CA2, and CA3 are acquired, and second image data 62 of the second recognition target R2 in the divided areas CA1, CA2, and CA3 are acquired.

[0068] In the embodiment, the first image data 61 and the second image data 62 are images captured of the same range of the component C, but are images captured under different imaging conditions according to measurement conditions 70 described below.

[0069] <Measurement conditions> The electrode C2 including the first recognition target R1 is made of metal as a whole, and therefore has a relatively high light reflectance. The boss C3 including the second recognition target R2 is made of dark-colored resin integrally formed with the body C1, and therefore has a relatively low light reflectance. In FIG. 7, the dark-colored resin part with low light reflectance (body C1) is colored to explain the difference in light reflectance. However, although the boss C3 (second recognition target R2) is a part with low light reflectance, coloring is omitted from the viewpoint of visibility in the figure. Metal parts such as the electrode C2 have high light reflectance, and therefore saturation (saturation of pixel values) is likely to occur when imaged. A pixel where saturation occurs cannot obtain a correct pixel value (brightness value), and therefore a phase value (height data) cannot be obtained. Therefore, in the three-dimensional measurement of the first recognition target R1, it is important not to generate saturation in the first image data 61. On the other hand, the body C1 including the second recognition target R2 is formed of dark-colored resin so as not to interfere with the recognition of the electrode C2, and is difficult to create contrast. Therefore, even if the pattern light PL is projected onto the boss C3, it is difficult to obtain a contrast (brightness distribution) that allows the phase value to be calculated. In this way, the first recognition target R1 and the second recognition target R2 are prone to saturation on the one hand and it is important to suppress excess brightness, while it is difficult to obtain contrast on the other hand and it is important to improve brightness, so it is usually difficult to perform 3D measurement using the same imaging system and the same optical system.

[0070] In the embodiment, the measurement conditions 70 (see FIG. 8) including the imaging conditions and the image recognition conditions are different between the first recognition target R1 and the second recognition target R2. This enables the same three-dimensional measurement device 1 (projection device 3 and imaging device 4) to perform three-dimensional measurements of both the first recognition target R1 and the second recognition target R2.

[0071] FIG. 8 is a diagram showing a specific example of the measurement condition 70 according to the embodiment. In the example of FIG. 8, the measurement condition 70 includes a camera gain, a threshold, and the number of combined images. The camera gain is the sensitivity of the imaging device 4 (image sensor 42). The higher the camera gain, the higher the luminance value (pixel value) in the image data for the same amount of incident light. The threshold is a threshold applied to threshold processing that divides the recognition target area and the background area in the image. The recognition target area is the area of ​​the first recognition target R1 or the area of ​​the second recognition target R2. The background is the area other than the recognition target. The threshold is set to a value between the range of luminance values ​​of the recognition target and the range of luminance values ​​of the background. The number of combined images is the number of image data to be combined in the combination process described later. The greater the number of combined images, the higher the luminance value (pixel value) of the combined image.

[0072] The measurement conditions 70 include a first measurement condition 70A for the first recognition target R1 and a second measurement condition 70B for the second recognition target R2. In the first measurement condition 70A, the camera gain is set to a low value (or a standard value). In the second measurement condition 70B for the second recognition target R2, the camera gain of the imaging device 4 is higher than that in the first measurement condition 70A for the first recognition target R1. This increases the luminance value (pixel value) of the second recognition target R2, making it easier to create contrast.

[0073] The first measurement condition 70A has a high threshold value (or a standard value). The processing device 5 cuts out an image portion for image recognition from the first image data 61 by threshold processing using the threshold value defined in the first measurement condition 70A. The threshold value of the first measurement condition 70A is set so that an image portion including the entire first recognition target R1 is cut out by the threshold processing. The processing device 5 cuts out an image portion for image recognition from the second image data 62 by threshold processing using the threshold value defined in the second measurement condition 70B. The threshold value of the second measurement condition 70B is set so that an image portion including the entire second recognition target R2 is cut out by the threshold processing. The second measurement condition 70B of the second recognition target R2 has a lower threshold value than the first measurement condition 70A of the first recognition target R1. This reduces the possibility that the area of ​​the second recognition target R2 is processed as a background area in the second image data 62 in which the luminance value of the second recognition target R2 is difficult to obtain.

[0074] In the first measurement condition 70A, the number of image combinations is set to a predetermined standard number. In the second measurement condition 70B of the second recognition target R2, the number of image combinations is greater than that of the first measurement condition 70A of the first recognition target R1. In one example, the number of image combinations in the first measurement condition 70A is set to one, and the number of image combinations in the second measurement condition 70B is set to a predetermined number of two or more. Therefore, the processing device 5 performs a combination process of a plurality of second image data 62 at least in the image recognition of the second recognition target R2. Note that the image combination process is performed when the number of image combinations is set to two or more. The number of image combinations of one means that image combination is not performed.

[0075] FIG. 9 is a schematic diagram for explaining the image synthesis process according to the embodiment. The projection device 3 projects a stripe pattern having a sine wave-shaped brightness distribution onto the part C while shifting the phase. The projection device 3 projects at least three types of stripe patterns having mutually different phases onto the part C. In one example, the projection device 3 projects four types of stripe patterns having phases differing by π / 2 onto the part C. That is, the stripe patterns include a first stripe pattern having a phase shift amount of 0° (initial phase=0), a second stripe pattern having a phase shift amount of 90° (initial phase=π / 2), a third stripe pattern having a phase shift amount of 180° (initial phase=π), and a fourth stripe pattern having a phase shift amount of 270° (initial phase=3π / 2). The projection device 3 sequentially projects the first stripe pattern, the second stripe pattern, the third stripe pattern, and the fourth stripe pattern onto the part C.

[0076] Here, image data before image synthesis is called a basic image. Image data after image synthesis is called a synthesized image. In each projection of a plurality of types of stripe patterns, the imaging device 4 captures the basic images the number of image synthesis frames set in the measurement condition 70. If the number of image synthesis frames in the second measurement condition 70B is N, the basic images are captured N times. FIG. 9 shows an example where N=4.

[0077] When the first stripe pattern is projected onto the part C, the image data acquisition unit 53 controls the imaging device 4 and the projection device 3 to image the part C onto which the first stripe pattern is projected N times. When the second stripe pattern is projected onto the part C, the image data acquisition unit 53 controls the imaging device 4 to image the part C onto which the second stripe pattern is projected N times. The same applies to the cases where the third stripe pattern is projected onto the part C and the fourth stripe pattern is projected onto the part C. The image data acquisition unit 53 acquires each of the first basic image imaged with the first stripe pattern, the second basic image imaged with the second stripe pattern, the third basic image imaged with the third stripe pattern, and the fourth basic image imaged with the fourth stripe pattern in the same number as the number of images to be combined N (=4).

[0078] The image data acquisition unit 53 combines the same number of basic images as the number of combined images N to generate a combined image corresponding to each of the multiple types of stripe patterns. The image data acquisition unit 53 combines N first basic images to generate a first combined image corresponding to the first stripe pattern. The image data acquisition unit 53 combines N second basic images to generate a second combined image corresponding to the second stripe pattern. The image data acquisition unit 53 combines N third basic images to generate a third combined image corresponding to the third stripe pattern. The image data acquisition unit 53 combines N fourth basic images to generate a fourth combined image corresponding to the fourth stripe pattern.

[0079] In an embodiment, synthesizing a plurality of basic images includes adding the plurality of basic images. The adding process of the plurality of basic images includes adding the luminance values ​​of the same pixel of the plurality of basic images. This results in a pixel with a high luminance value in the synthesized image. In each synthesized image, the luminance difference (i.e., contrast) between the light and dark parts of the stripe pattern becomes large. Therefore, it becomes easier to calculate the phase value even in the part of the second recognition target R2 where contrast is difficult to achieve.

[0080] The above measurement conditions 70 are stored in the storage unit 56. When performing three-dimensional measurement of the first recognition target R1, the processing device 5 acquires the first measurement conditions 70A from the storage unit 56 and performs imaging control and image recognition based on the first measurement conditions 70A. When performing three-dimensional measurement of the second recognition target R2, the processing device 5 acquires the second measurement conditions 70B from the storage unit 56 and performs imaging control and image recognition based on the second measurement conditions 70B.

[0081] <Image Recognition> 10 is an explanatory diagram for explaining a recognition algorithm of the image recognition process executed by the processing device 5 of the embodiment. The processing device 5 executes different recognition algorithms for the first recognition target R1 and the second recognition target R2 to perform image recognition.

[0082] When performing three-dimensional measurement of the first recognition target R1, the phase value calculation unit 54 executes an electrode recognition algorithm 71A. The electrode recognition algorithm 71A is optimized for recognizing the first recognition target R1, i.e., the tip of the electrode C2 of the component C. The electrode recognition algorithm 71A is suitable for recognizing the first recognition target R1 of a known shape based on the component data. The shape of the first recognition target R1 is, for example, rectangular.

[0083] When performing three-dimensional measurement of the second recognition target R2, the phase value calculation unit 54 executes the boss recognition algorithm 71B. The boss recognition algorithm 71B is optimized for recognizing the second recognition target R2, i.e., the tip of the boss C3 formed on the body C1 of the part C. The boss recognition algorithm 71B is suitable for recognizing the second recognition target R2 of a known shape based on the part data. The shape of the second recognition target R2 is, for example, a circular shape.

[0084] The electrode recognition algorithm 71A and the boss recognition algorithm 71B are stored in the storage unit 56. The processing device 5 selectively reads out the recognition algorithm corresponding to the recognition target from the storage unit 56 and executes image recognition. The phase value calculation unit 54 generates measurement data representing the phase value of each pixel constituting the recognition target (first recognition target R1 and second recognition target R2) by image recognition.

[0085] <Complementary processing> FIG. 11 is an explanatory diagram showing an example of the first complementation process of image recognition by the processing device 5 of the embodiment. As described above, in the embodiment, the projection device 3 can irradiate the pattern light PL from four directions, and four image data with different irradiation directions of the pattern light PL are acquired. That is, acquisition of the basic image shown in FIG. 9, generation of a composite image (only when performing a composite process), and calculation of a three-dimensional shape are performed four times for each direction. As a result, the processing device 5 generates four types of measurement data for each direction, as shown in FIG. 11. FIG. 11 shows examples of first direction measurement data 81A, second direction measurement data 81B, third direction measurement data 81C, and fourth direction measurement data 81D generated by image recognition of one recognition target.

[0086] In each measurement data, an error region 82 from which height data (or phase value) cannot be calculated may occur due to differences in the irradiation direction of the pattern light PL. The error region 82 is a saturation region that occurs when regular reflected light is directly incident on the imaging device 4, or a shadow region where light is blocked by other parts of the part C, depending on the relationship between the shape and orientation of the recognition target and the irradiation direction of the pattern light PL.

[0087] In the embodiment, when the processing device 5 fails to recognize any part of the image data obtained by the pattern light PL in the multiple directions, the processing device 5 performs image recognition based on the other image data that has been successfully recognized. In other words, even if an error region 82 occurs in any of the first direction measurement data 81A, the second direction measurement data 81B, the third direction measurement data 81C, and the fourth direction measurement data 81D, if the processing device 5 can calculate height data of the same region in the measurement data in the other directions, the processing device 5 performs a first complementation process to complement the measurement result (height data) based on the height data.

[0088] For example, assume that the circular shape shown in the upper part of Fig. 11 is the true shape of the recognition target (e.g., second recognition target R2). In each measurement data shown in the middle part of Fig. 11, the white area indicates the recognition area 83 where height data was calculated (recognition was successful). In each measurement data, the area surrounded by a dotted line adjacent to the recognition area 83 (the area of ​​difference between the true shape and the recognition area 83) is the error area 82. In each measurement data, the outside of the recognition area 83 and the error area 82 (black area) is the background part.

[0089] In the example of Fig. 11, an error region 82 is formed on the lower side in the first direction measurement data 81A, an error region 82 is formed on the upper side in the second direction measurement data 81B, an error region 82 is formed on the right side in the third direction measurement data 81C, and an error region 82 is formed on the left side in the fourth direction measurement data 81D. In the example of Fig. 11, an error region 82 is partially formed in each of the measurement data, and there is no measurement data that was able to acquire the true shape of the recognition target.

[0090] The processing device 5 synthesizes the measurement data in each direction by a method of calculating the logical sum of each recognition area 83 of the first direction measurement data 81A, the second direction measurement data 81B, the third direction measurement data 81C, and the fourth direction measurement data 81D, and generates the synthesized measurement data 80 shown in the lower part of FIG. 11. For example, the processing device 5 generates height data of a position (pixel) belonging to an error area 82 of the first direction measurement data 81A by using height data of the recognition area 83 of the second direction measurement data 81B, the third direction measurement data 81C, and the fourth direction measurement data 81D. When multiple height data are obtained for the same position, the processing device 5 calculates a representative value (for example, an average value) of the multiple height data and sets it as the height data of the position. When only one height data is obtained for the same position, the processing device 5 adopts the value of the height data as the height data of the position. As a result, in the synthesized measurement data 80, height data is obtained for the entire area corresponding to the true shape of the recognition target shown in the upper part of FIG. 11.

[0091] 12 is an explanatory diagram showing an example of the second complementation process of image recognition by the processing device 5 of the embodiment. The above-mentioned first complementation process is applied to the error region 82 where image recognition was successful in at least one of the measurement data in four directions and height data (phase value) was obtained. On the other hand, for example, when the tip of the electrode C2 (first recognition target R1) is mirror-like, there may be a case where height data (phase value) cannot be obtained in all of the measurement data in four directions due to the occurrence of saturation.

[0092] Therefore, in the embodiment, the processing device 5 executes a second complementation process when the processing fails to recognize the saturation region 84 due to the saturation region 84 being included in a part of the first image data 61. In the second complementation process, the processing device 5 determines whether or not the saturation region 84 is a part of the first recognition target R1 based on the shape data of the part C. Then, when the processing device 5 determines that the saturation region 84 is a part of the first recognition target R1, the processing device 5 executes a process of complementing the saturation region 84 using the recognition result of the first recognition target R1. The saturation region 84 is a pixel region in the image data (first image data 61) where the luminance value (pixel value) reaches the maximum value.

[0093] Specifically, as shown in FIG. 12, it is assumed that a saturation region 84 exists in a part of the first image data 61. In the first image data 61 in FIG. 12, the phase of the stripe pattern is different between the rectangular area in the center and the outer periphery. The rectangular area in the center indicates the first recognition target R1 (i.e., the lower surface of the electrode C2), and the outer periphery is the background region BG. In FIG. 12, a part of the left end of the first recognition target R1 in the figure is the saturation region 84. Since the pixel value (luminance value) of each pixel included in the saturation region 84 is the maximum value, the phase of the stripe pattern cannot be recognized. FIG. 12 illustrates the first image data 61 in one of the irradiation directions, but it is assumed that the saturation region 84 is formed at the same position in the first image data 61 in the other irradiation directions.

[0094] In this case, in the measurement data in each direction (first direction measurement data 81A to fourth direction measurement data 81D), height data (phase value) cannot be acquired in saturation region 84, so it becomes an error region 82. In FIG. 12, only first direction measurement data 81A is illustrated. In the first recognition target R1, the area other than error region 82 becomes recognition region 83 from which height data (phase value) can be acquired. If saturation region 84 becomes error region 82 in all measurement data in four directions (first direction measurement data 81A to fourth direction measurement data 81D), the above-mentioned first complementation process cannot be applied.

[0095] In this case, the processing device 5 performs a process of determining whether or not the saturation region 84 is a part of the first recognition target R1 based on the part data 72 stored in the storage unit 56. The processing device 5 identifies a part of the part C that is reflected in the first image data 61 based on the relationship between the position of the first recognition target R1 obtained from the part data 72, the position of the background region BG (such as the body C1), and the position of the imaging device 4. That is, the processing device 5 performs a calculation to determine whether or not the first recognition target R1 exists on the optical path extended along the optical axis AX (see FIG. 2) from a position corresponding to the pixel position of the saturation region 84, and if the first recognition target R1 exists, the processing device 5 determines that the saturation region 84 is a part of the first recognition target R1. If the first recognition target R1 does not exist on the optical path, the processing device 5 determines that the saturation region 84 is not a part of the first recognition target R1. In this embodiment, the imaging optical system 41 is a telecentric optical system, which differs from other optical systems in which the chief ray is not parallel to the optical axis AX in that correction of the lens angle of view, etc. is not required, thereby reducing the amount of calculation required for the determination process.

[0096] When the processing device 5 determines that the saturation region 84 is a part of the first recognition target R1, the processing device 5 complements the height data (phase value) of the saturation region 84 with the height data (phase value) of the other recognition region 83 of the first recognition target R1. As a result, in the measurement data 80 after the second complementation process, the entire height data of the first recognition target R1 including the saturation region 84 is calculated.

[0097] On the other hand, when the processing device 5 determines that the saturation region 84 is not a part of the first recognition target R1, the saturation region 84 remains as the error region 82. In this case, in the measurement data 80 after the second complementation process, the portion of the saturation region 84 is processed as the background region BG.

[0098] <3D measurement method> The operation of the three-dimensional measuring apparatus 1 according to the embodiment will be described below. Fig. 13 is a flowchart showing a three-dimensional measuring method according to the embodiment. The three-dimensional measuring method according to the embodiment, which is executed by the three-dimensional measuring apparatus 1, will be described with reference to Fig. 13.

[0099] As shown in FIG. 13, the three-dimensional measuring method according to the embodiment generally includes an imaging step S10, an image synthesis step S20, an image recognition step S30, and an interpolation processing step S40.

[0100] In the imaging step S10, the three-dimensional measuring device 1 performs a first imaging (step S11) of a first recognition target R1 of the part C and a second imaging (step S12) of a second recognition target R2. In Fig. 13, the processing is performed in the order of the first imaging (step S11) and the second imaging (step S12), but these processing may be performed in the reverse order, i.e., the second imaging and the first imaging.

[0101] First, the three-dimensional measuring device 1 performs a first image capture (step S11) in a state where any divided area (e.g., divided area CA1) of the component C is placed within the field of view 4S by the mounting head 22. The processing device 5 performs control to acquire the first image data 61 of the first recognition target R1 based on the first measurement condition 70A.

[0102] The processing device 5 controls the first and second projection devices 3 to emit the first pattern light PL1 and the second pattern light PL2 based on the first pattern data and the second pattern data created by the pattern generation unit 52, respectively, and irradiate them toward the component C. The image data acquisition unit 53 acquires the first image data 61 of the component C onto which the first pattern light PL1 and the second pattern light PL2 are projected in four directions, respectively, with a camera gain defined in the first measurement condition 70A. The image data acquisition unit 53 acquires each of the first image data 61 in the four directions, the number of which is equal to the number of image combinations defined in the first measurement condition 70A. The number N of image combinations in the first measurement condition 70A is, for example, 1 (no image combination).

[0103] The processing device 5 controls the first and second projection devices 3 to obtain first image data 61 in which four types of stripe patterns with different phases, from the first stripe pattern to the fourth stripe pattern, are projected. That is, for each of the four irradiation directions, the first image data 61 of the four phase patterns is obtained for each of the image synthesis number N (=1).

[0104] The three-dimensional measuring device 1 performs a second imaging (step S12) for the same divided area (for example, divided area CA1). The processing device 5 performs control to acquire second image data 62 of the second recognition target R2 based on the second measurement condition 70B.

[0105] The processing device 5 controls the first and second projection devices 3 to emit the first pattern light PL1 and the second pattern light PL2, respectively, and irradiate them toward the component C. The image data acquisition unit 53 acquires second image data 62 of the component C onto which the first pattern light PL1 and the second pattern light PL2 are respectively projected in four directions, with a camera gain defined in the second measurement condition 70B. The image data acquisition unit 53 acquires each of the second image data 62 in the four directions, the number of pieces of which is equal to the number of combined images N defined in the second measurement condition 70B. The number of combined images N in the second measurement condition 70B is, for example, four. As a result, a basic image of the second image data 62 is captured.

[0106] The processing device 5 controls the first and second projection devices 3 to obtain second image data 62 in which four types of stripe patterns with different phases, from the first stripe pattern to the fourth stripe pattern, are projected. That is, for each of the four irradiation directions, first image data 61 (basic images) of the four phase patterns are obtained for each of the image synthesis number N (=4).

[0107] The processing device 5 judges whether all image data to be acquired has been acquired (step S13). When the processing device 5 has completed imaging of the three divided areas CA1, CA2, and CA3 of the component C, it judges that all image data has been acquired. When there is a divided area that has not been imaged, the processing device 5 judges that acquisition of all image data has not been completed. In this case, the processing device 5 controls the mounting head 22 by the control device 29 to change the position of the component C so that the next divided area (e.g., divided area CA2) is positioned within the field of view range 4S (step S14).

[0108] The processing device 5 performs a first image capture (step S11) and a second image capture (step S12) for the divided area CA2. After step S13, the processing device 5 changes the image capture position so that the next divided area (e.g., divided area CA3) is positioned within the field of view range 4S (step S14). Similarly, the processing device 5 performs a first image capture (step S11) and a second image capture (step S12) for the divided area CA3. As a result, imaging is completed for all divided areas, and the processing device 5 proceeds to the image synthesis step S20 after step S13.

[0109] When the number of image combinations under the measurement conditions is two or more, the processing device 5 performs a combination process of the image data (image combination step S20). In the embodiment, since the number of image combinations N is set to 1 under the first measurement conditions 70A, the processing device 5 does not perform image combination on the first image data 61. The processing device 5 generates, as the first image data 61, four sets of images of the first to fourth stripe patterns for each irradiation direction.

[0110] Since the number of combined images N is set to 4 in the second measurement condition 70B, the processing device 5 performs image combination processing on the second image data 62. As shown in FIG. 9, the processing device 5 combines N basic images of the first to fourth stripe patterns to generate the first to fourth combined images. The processing device 5 performs the combination processing of each basic image of the first to fourth stripe patterns for each of the second image data 62 in the four irradiation directions. The processing device 5 generates four sets of the first to fourth combined images for each irradiation direction as the second image data 62.

[0111] The processing device 5 performs image recognition on the first image data 61 and the second image data 62 (image recognition step S30). The phase value calculation unit 54 calculates the phase value of each of the multiple pixels in the image data based on the luminance (pixel value) of each image data of the first stripe pattern to the fourth stripe pattern for each irradiation direction. The phase value calculation unit 54 cuts out a recognition target area for the first image data 61 using a threshold value specified in the first measurement condition 70A, and calculates height data (phase value) of the first recognition target R1 by executing the electrode recognition algorithm 71A. The processing device 5 cuts out a recognition target area for the second image data 62 using a threshold value specified in the second measurement condition 70B, and calculates the phase value of the second recognition target R2 by executing the boss recognition algorithm 71B.

[0112] The three-dimensional shape calculation unit 55 calculates height data for each pixel of the image data corresponding to each recognition target based on the phase value and the principle of triangulation. Here, the height data for each pixel of the image data and the height data for each point in the recognition target of part C correspond one-to-one. The height data for each point in the recognition target of part C indicates the coordinate value of each point in the recognition target of part C in three-dimensional space. The three-dimensional shape calculation unit 55 generates measurement data for the first recognition target R1 and measurement data for the second recognition target R2.

[0113] When an error region 82 exists in the measurement data of the first recognition target R1 and the measurement data of the second recognition target R2, the processing device 5 executes a complementation process for pixels belonging to the error region 82 (complementation process step S40). When an error region 82 exists in any part of the measurement data for each of the four irradiation directions, and there exists measurement data for another irradiation direction in which the same region has been successfully recognized, the processing device 5 executes a first complementation process. When the processing device 5 fails to recognize the saturation region 84 in all of the measurement data for the four irradiation directions due to the saturation region 84 being included in a part of the first image data 61, the processing device 5 executes a second complementation process in which the processing device 5 determines whether the saturation region 84 is part of the first recognition target R1, and executes a second complementation process in which the processing device 5 executes a process of complementing the saturation region 84 using the recognition result of the first recognition target R1 when it is determined that the saturation region 84 is part of the first recognition target R1. Note that, when the error region 82 (saturation region 84) does not exist, the processing device 5 does not execute the first and second complementation processes.

[0114] As a result, the processing device 5 calculates the three-dimensional shapes of the first recognition target R1 and the second recognition target R2 based on the height data at each point of the first recognition target R1 and the second recognition target R2.

[0115] <Implementation method> Fig. 14 is a flowchart for explaining the operation of electronic component mounting system 100 according to the embodiment. The operation of electronic component mounting system 100 according to the embodiment will be explained below. As shown in Fig. 14, electronic component mounting system 100 executes a detection step S60, a proximity step S70, and an insertion step S80.

[0116] The three-dimensional measuring device 1 recognizes the first recognition target R1 (electrode C2) and the second recognition target R2 (boss C3) of the component C by executing the three-dimensional measuring method according to the embodiment. The processing device 5 transmits measurement data of the first recognition target R1 and the second recognition target R2 to the control device 29. The control device 29 adjusts the relative position between the component C and the board P based on the height data of the first recognition target R1 and the second recognition target R2, and controls the mounting head 22 to mount the component C on the board P (detection step S60).

[0117] Based on commands from the three-dimensional measuring device 1, the control device 29 controls the head driving device 25 and nozzle driving device 26 of the mounting head 22 to bring the vertically lower tip of the component C held by the nozzle 11 close to the top surface of the substrate P (close step S70).

[0118] In the proximity step S70, the control device 29 controls the nozzle driving device 26 of the mounting head 22 to move the nozzle 11 vertically downward so that the vertically lower tip of the component C held by the nozzle 11 is brought close to the upper surface of the substrate P, based on the height data of the first recognition target R1 and the second recognition target R2 of the component C received from the 3D measuring device 1.

[0119] In the proximity step S70, the control device 29 also controls the head driving device 25 and nozzle driving device 26 of the mounting head 22 to move the mounting head 22 horizontally and rotate the nozzle 11 vertically so that the positions of the first recognition target R1 and the second recognition target R2 of the component C held by the nozzle 11 coincide with the electrode connection positions and the horizontal positions of the through holes formed on the substrate P, based on the measurement data of the three-dimensional shape of the component C held by the nozzle 11 and the measurement data of the position measuring device 27 indicating the positions of the through holes formed on the substrate P.

[0120] Based on a command from the three-dimensional measuring device 1, the control device 29 controls the nozzle driving device 26 of the mounting head 22 to insert the component C held by the nozzle 11 into a through-hole formed in the substrate P (insertion step S80). The control device 29 controls the nozzle driving device 26 of the mounting head 22 to move the nozzle 11 downward in the vertical direction. As a result, the three bosses C3 of the component C are inserted into the corresponding through-holes of the substrate P, and the component C is positioned on the substrate P. As a result, each electrode C2 of the component C is positioned at an electrode connection position of the substrate P. Note that each electrode C2 of the component C may be a pin-type electrode that is inserted into an electrode insertion hole of the substrate P.

[0121] <Effects> As described above, according to this embodiment, the three-dimensional measuring device 1 includes a projection device 3 that projects pattern light PL with different phases onto a part C having a first recognition target R1 and a second recognition target R2 having a lower light reflectance than the first recognition target R1, an imaging device 4 that separately images the first recognition target R1 and the second recognition target R2 onto which the pattern light PL is projected, and a processing device 5 that performs image recognition of the first recognition target R1 based on first image data 61 of the first recognition target R1 and image recognition of the second recognition target R2 based on second image data 62 of the second recognition target R2, and measurement conditions 70 including imaging conditions and image recognition conditions are different for the first recognition target R1 and the second recognition target R2. As a result, the first recognition target R1, which has a relatively high light reflectance, and the second recognition target R2, which has a low light reflectance, are imaged separately under measurement conditions suitable for each, so that first image data 61 suitable for image recognition of the first recognition target R1, which is prone to saturation, and second image data 62 suitable for image recognition of the second recognition target R2, which is difficult to create contrast, are obtained. Then, by performing image recognition of the first image data 61 and the second image data 62 under measurement conditions suitable for each, three-dimensional measurement of the first recognition target R1 and the second recognition target R2 can be performed. As a result, even if one part includes a recognition target with a high light reflectance and a recognition target with a low light reflectance, each recognition target can be appropriately recognized by the same measurement device.

[0122] According to this embodiment, the second measurement condition 70B, which is the measurement condition of the second recognition target R2, has a higher camera gain of the imaging device 4 than the first measurement condition 70A, which is the measurement condition of the first recognition target R1. The second measurement condition 70B has a lower threshold value for dividing the recognition target area and the background area in the image than the first measurement condition 70A. The second measurement condition 70B has a larger number of image composites than the first measurement condition 70A. As a result, the first image data 61 of the first recognition target R1, which is prone to saturation, can be generated with a relatively low gain, a high threshold value, and a small number of composites. As a result, saturation in the image data of the first recognition target R1 can be suppressed, and the first recognition target R1 and the background can be distinguished more accurately. The second image data 62 of the second recognition target R2, which is difficult to add contrast to, can be generated with a relatively high gain, a low threshold value, and a large number of composites. As a result, the image data of the second recognition target R2 is more likely to have contrast, and the second recognition target R2 can be distinguished more accurately from the background.

[0123] According to this embodiment, when the processing device 5 fails to recognize any part of the image data obtained by the pattern light PL in the multiple directions, the processing device 5 performs image recognition (first complementation processing) based on other image data that has been successfully recognized. As a result, even if image recognition fails for a part of the recognition target in the image data and an error region 82 occurs, if image recognition is successful using image data in other irradiation directions, the measurement data can be complemented to suppress the occurrence of the error region 82, which is a partial lack of measurement data.

[0124] According to this embodiment, when the processing device 5 fails to recognize the saturation region 84 due to the saturation region 84 being included in a part of the first image data 61, the processing device 5 determines whether the saturation region 84 is a part of the first recognition target R1 based on the shape data of the part C, and when it is determined that the saturation region 84 is a part of the first recognition target R1, the processing device 5 performs a process (second complementation process) of complementing the saturation region 84 using the recognition result of the first recognition target R1. As a result, even if the saturation region 84 occurs in a part of the first recognition target R1 with high light reflectance in the first image data 61 and measurement data cannot be obtained, the recognition result of the saturation region 84 can be complemented. As a result, even if the first recognition target R1 is prone to saturation due to its high light reflectance, the contour shape of the first recognition target R1 can be accurately acquired.

[0125] According to this embodiment, the first recognition target R1 is the tip of the electrode C2 protruding from the body C1 of the component C. The second recognition target R2 is the tip of the boss C3 protruding from the body C1 made of dark-colored resin. This allows image recognition to be performed together with the electrode C2 using the same measurement device even for a part of the body C1 made of dark-colored resin that is difficult to contrast, so as not to interfere with image recognition of the electrode C2.

[0126] [Other embodiments] In the above-described embodiment, an example was shown in which the three-dimensional measuring apparatus 1 was provided in an electronic component mounting system 100 equipped with an electronic component mounting apparatus 2, but the three-dimensional measuring apparatus 1 may also be provided in, for example, an inspection apparatus that inspects components C. [Explanation of symbols]

[0127] 1 ... 3D measurement device, 2 ... electronic component mounting device, 3 ... projection device, 4 ... imaging device, 4S ... field of view range, 5 ... processing device, 6 ... reflective member, 7 ... housing, 11 ... nozzle, 22 ... mounting head, 23 ... electronic component supply device, 24 ... substrate conveying device, 25 ... head driving device, 26 ... nozzle driving device, 27 ... position measurement device, 29 ... control device, 31 ... light source, 32 ... light modulation element, 33 ... emission optical system, 33S ... emission surface, 41 ... imaging optical system, 41S ... incidence surface, 42 ... imaging element, 51 ... input / output unit, 52 ... pattern generation unit, 53 ... image data acquisition unit, 54 ... phase value calculation unit, 55 ... 3D shape calculation unit, 56 ... memory unit, 61 ... first image data, 62 ... second image data, 70 ... measurement condition, 70A ... first measurement condition, 70B ... second measurement condition, 71A ... electrode recognition algorithm, 71B... boss recognition algorithm, 72... component data, 80... measurement data, 81A... first direction measurement data, 81B... second direction measurement data, 81C... third direction measurement data, 81D... fourth direction measurement data, 82... error area, 83... recognition area, 84... saturation area, 100... electronic component mounting system, 331... first area, 332... second area, AX... optical axis, BG... background area, C... component, C1... body, C2... electrode, C3... boss, C4... screw hole, C11... terminal part, C12... fixing part, C13... bottom surface, CA1, CA2, CA3... division area, MP... mounting position, N... number of images synthesized, P... board, PL... pattern light, PL1... first pattern light, PL2... second pattern light, R1... first recognition target, R2... second recognition target, SP... supply position, TP... measurement position.

Claims

1. A projection device that projects pattern light having different phases onto a part having a first recognition target and a second recognition target having a lower light reflectance than the first recognition target; an imaging device that separately images the first recognition target and the second recognition target onto which the pattern light is projected; a processing device that performs image recognition of the first recognition target based on first image data of the first recognition target, and performs image recognition of the second recognition target based on second image data of the second recognition target, Measurement conditions including imaging conditions and image recognition conditions are different between the first recognition target and the second recognition target. 3D measuring device.

2. the measurement condition of the second recognition target is higher than the measurement condition of the first recognition target, and a camera gain of the imaging device is higher. The three-dimensional measuring apparatus according to claim 1 .

3. The measurement condition of the second recognition target has a lower threshold value for dividing a recognition target region and a background region in an image than the measurement condition of the first recognition target. The three-dimensional measuring apparatus according to claim 1 .

4. The measurement condition of the second recognition target includes a larger number of times of imaging than the measurement condition of the first recognition target, The processing device performs a synthesis process on the plurality of second image data. The three-dimensional measuring apparatus according to claim 1 .

5. the projection device projects the pattern light onto the component from a plurality of directions separately, the imaging device captures images of the pattern light from the plurality of directions separately, when the processing device fails to recognize any part of the image data obtained by the pattern light in the multiple directions, the processing device performs image recognition based on other image data that has been successfully recognized. The three-dimensional measuring apparatus according to claim 1 .

6. When the processing device fails to recognize the saturation region due to a saturation region being included in a part of the first image data, determining whether the saturation region is a part of the first recognition target based on shape data of the part; When it is determined that the saturation region is a part of the first recognition target, a process of complementing the saturation region using a recognition result of the first recognition target is performed. The three-dimensional measuring apparatus according to claim 1 .

7. the first recognition target is a tip portion of an electrode protruding from a body of the component, the second recognition target is a tip portion of a boss protruding from the body made of a dark-colored resin; The three-dimensional measuring apparatus according to claim 1 .

8. A step of projecting pattern light having different phases onto a component having a first recognition target and a second recognition target having a lower light reflectance than the first recognition target, and taking a first image of the first recognition target onto which the pattern light is projected; projecting the pattern light onto the component and capturing a second image of the second recognition target onto which the pattern light is projected; performing image recognition on the first recognition target based on first image data of the first recognition target, and performing image recognition on the second recognition target based on second image data of the second recognition target; Measurement conditions including imaging conditions and image recognition conditions are different between the first recognition target and the second recognition target. 3D measurement method.

Images