Three-dimensional measurement device and three-dimensional measurement method

Abstract

The invention provides a three-dimensional measurement apparatus and a three-dimensional measurement method. Even if one member includes an object to be identified having a high light reflectance and an object to be identified having a low light reflectance, the object to be identified can be appropriately identified by the same measurement device. A three-dimensional measurement device includes: a projection device that projects pattern light having different phases onto a member having an electrode as a first recognition target and a second recognition target having a lower light reflectance than the electrode; an imaging device for imaging the first recognition object and the second recognition object on which the pattern light has been projected; and a processing device that performs image recognition on the first recognition object on the basis of the first image data of the first recognition object and performs image recognition on the second recognition object on the basis of the second image data of the second recognition object, in which measurement conditions including the imaging conditions and the image recognition conditions differ between the first recognition object and the second recognition object.

Description

Technical Field

[0001] The present invention relates to a three-dimensional measuring device and a three-dimensional measuring method. Background Art

[0002] As a method for measuring the three-dimensional shape of an object, the phase shift method disclosed in Patent Document 1 is known. When measuring the three-dimensional shape of an object based on the phase shift method, a stripe pattern (pattern light) with a sinusoidal brightness distribution is projected onto the object while being phase shifted. The object onto which the stripe pattern is projected is photographed to obtain an image of the object.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2001-124534 Summary of the invention

[0004] One of the uses of the three-dimensional measurement method is to install components in electronic devices. The identification object such as the electrode of the component is measured in three dimensions, and the component is installed on a substrate etc. based on the measurement results. The light reflectivity of the electrode of the component is high and easy to identify, but on the other hand, saturation (saturation of the brightness value of the image) is prone to occur due to excessive light reflection. On the other hand, the resin part of the component has a low light reflectivity compared to the electrode, and it is difficult to form a contrast, so it is difficult to identify. Therefore, in the case where a component contains an identification object with a high light reflectivity and an identification object with a low light reflectivity, it is necessary to set up a dedicated measurement device suitable for each identification.

[0005] An object of the aspects of the present invention is to enable appropriate identification of the identification objects by the same measuring device even when one component includes an identification object having a high light reflectance and an identification object having a low light reflectance.

[0006] According to a first embodiment of the present invention, there is provided a three-dimensional measuring device, comprising: a projection device for projecting pattern light having different phases onto a component having a first identification object and a second identification object having a lower light reflectance than the first identification object; a photographing device for photographing the first identification object and the second identification object onto which the pattern light is projected, respectively; and a processing device for performing image recognition on the first identification object based on first image data of the first identification object and performing image recognition on the second identification object based on second image data of the second identification object, wherein measurement conditions including photographing conditions and image recognition conditions are different for the first identification object and the second identification object.

[0007] According to the second scheme of the present invention, a three-dimensional measurement method is provided, which includes: a step of projecting pattern light with different phases onto a component having a first identification object and a second identification object with a lower light reflectivity than the first identification object, and performing a first shooting of the first identification object onto which the pattern light is projected; a step of projecting the pattern light onto the component, and performing a second shooting of the second identification object onto which the pattern light is projected; and a step of performing image recognition of the first identification object based on the first image data of the first identification object, and performing image recognition of the second identification object based on the second image data of the second identification object, wherein measurement conditions including shooting conditions and image recognition conditions are different for the first identification object and the second identification object.

[0008] According to the aspect of the present invention, even when an identification target having a high light reflectance and an identification target having a low light reflectance are included in one component, each identification target can be appropriately identified by the same measuring device. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is a diagram schematically showing an example of an electronic component mounting system according to an embodiment. Figure 2 It is a schematic diagram showing an example of a three-dimensional measuring device according to an embodiment. Figure 3 It is a schematic diagram showing an example of a three-dimensional measuring device according to an embodiment. Figure 4 This is a functional block diagram showing an example of a processing device according to an embodiment. Figure 5 It is a schematic diagram showing an example of pattern data according to the present embodiment. Figure 6 It is a side view (A) and a bottom view (B) which show an example of the component of embodiment. Figure 7 It is an explanatory diagram for explaining the photographing of the components of the embodiment. Figure 8 It is a diagram showing a specific example of measurement conditions in the embodiment. Fig. 9 It is a schematic diagram for explaining the image synthesis processing according to the embodiment. Fig.10 It is an explanatory diagram for explaining a recognition algorithm of an image recognition process executed by the processing device according to the embodiment. Fig.11 It is an explanatory diagram showing an example of the first complement processing of image recognition performed by the processing device according to the embodiment. Fig.12 It is an explanatory diagram showing an example of the second complement processing of image recognition performed by the processing device according to the embodiment. Fig.13 It is a flowchart showing the three-dimensional measurement method of the embodiment. Fig.14 This is a flowchart for explaining the operation of the electronic component mounting system according to the embodiment. DETAILED DESCRIPTION

[0010] Hereinafter, the embodiments of the present invention will be described with reference to the drawings, but the present invention is not limited thereto. The constituent elements of the embodiments described below may be combined as appropriate. In addition, some constituent elements may not be used.

[0011] In the following description, an XYZ orthogonal coordinate system is set, and the positional relationship of each part is described with reference to the XYZ orthogonal coordinate system. The direction parallel to the X axis of the specified plane is set as the X axis direction, the direction parallel to the Y axis orthogonal to the X axis in the specified plane is set as the Y axis direction, and the direction parallel to the Z axis orthogonal to the specified plane is set as the Z axis direction. The rotation direction centered on the Z axis is set as the θZ direction. In addition, in the following description, the specified plane is appropriately referred to as the XY plane. In the present embodiment, the XY plane is parallel to the horizontal plane.

[0012] [Implementation Method] <Electronic component mounting system> An implementation method will be described. Figure 1 This is a diagram schematically showing an example of the electronic component mounting system 100 according to the embodiment. Figure 1 This is a diagram of the electronic component mounting system 100 as viewed from the −Y direction.

[0013] like Figure 1 As shown, the electronic component mounting system 100 includes a three-dimensional measuring device 1 and an electronic component mounting device 2. The electronic component mounting system 100 performs three-dimensional measurement of the shape of a component C by the three-dimensional measuring device 1, and based on the measurement result, mounts the component C on a substrate P by the electronic component mounting device 2. Hereinafter, the electronic component mounting device 2 will be described first, and then the three-dimensional measuring device 1 of the present embodiment will be described.

[0014] <Electronic component mounting device> The electronic component mounting device 2 mounts the component C on the substrate P. The electronic component mounting device 2 includes: a mounting head 22 that holds the suction nozzle 11 for holding the component C; an electronic component supply device 23 that supplies the component C; a substrate conveying device 24 that conveys the substrate P on which the component C is to be mounted; and a control device 29 that controls the electronic component mounting device 2.

[0015] The electronic component supply device 23 has, for example, a moving mechanism that moves a tray holding a plurality of components C in a horizontal direction. The electronic component supply device 23 supplies the components C to the supply position SP. The electronic component supply device 23 may be a component supply device of a tape supply method that supplies the components C held on a tape. In this case, the electronic component supply device 23 includes: a tape reel that supports the tape; and a feeder that conveys the components C held on the tape drawn from the tape reel.

[0016] The substrate conveying device 24 includes a conveyor belt for conveying the substrate P and a clamp mechanism for fixing the position of the substrate P conveyed to the mounting position MP.

[0017] The mounting head 22 mounts the component C held by the suction nozzle 11 on the substrate P. The mounting head 22 moves in the X-axis direction and the Y-axis direction respectively by the head driving device 25. The suction nozzle 11 moves in the Z-axis direction and the θZ direction respectively relative to the mounting head 22 by the suction nozzle driving device 26 provided on the mounting head 22. The suction nozzle 11 can move in four directions, namely, the X-axis direction, the Y-axis direction, the Z-axis direction, and the θZ direction, by the head driving device 25 and the suction nozzle driving device 26. The mounting head 22 can move between the supply position SP specified by the electronic component supply device 23 and the mounting position MP specified by the substrate P.

[0018] The mounting head 22 holds the component C supplied from the electronic component supply device 23 by the suction nozzle 11 at the supply position SP and then conveys the component to the substrate P arranged at the mounting position MP. The mounting head 22 mounts the component C held by the suction nozzle 11 on the substrate P arranged at the mounting position MP.

[0019] The suction nozzle 11 detachably holds the component C. The suction nozzle 11 may be a suction nozzle for holding the component C by suction, or may be a gripping nozzle for holding the component C by clamping.

[0020] The mounting head 22 places the component C at the measurement position TP of the three-dimensional measuring device 1 after the component C is held by the suction nozzle 11 at the supply position SP and before being conveyed to the substrate P. The measurement position TP of the three-dimensional measuring device 1 is the pattern light PL (see Figure 2 ) can be irradiated.

[0021] The substrate transfer device 24 is provided with a position measuring device 27 for measuring the position of a through hole formed in the substrate P disposed at the mounting position MP. Measurement data of the position measuring device 27 indicating the position of the through hole formed in the substrate P is output to the control device 29 .

[0022] The control device 29 is connected to the head driving device 25, the nozzle driving device 26, the position measuring device 27, and the processing device 5 of the three-dimensional measuring device 1 described later. Before mounting the component C on the substrate P, the control device 29 causes the three-dimensional measuring device 1 to measure the three-dimensional shape of the component C. As described later, the mounting head 22 adjusts the relative position of the component C and the substrate P based on the measurement result of the component C measured by the three-dimensional measuring device 1, and mounts the component C on the substrate P.

[0023] <Three-dimensional measuring device> like Figure 1 As shown, the three-dimensional measuring device 1 includes: a projection device 3 for projecting pattern light PL onto a component C; a photographing device 4 for photographing the component C onto which the pattern light PL is projected; and a processing device 5 for performing image recognition based on the photographed image data. Figure 1 In the example in FIG. 1 , the three-dimensional measuring device 1 is disposed below the measurement position TP (in the −Z direction). The three-dimensional measuring device 1 is set with the imaging direction facing upward (in the +Z direction) so as to image the lower surface of the component C held by the suction nozzle 11 .

[0024] The three-dimensional measuring device 1 is disposed on the moving path of the mounting head 22 between the supply position SP and the mounting position MP at a position where the pattern light PL can be irradiated onto the component C held by the suction nozzle 11 and disposed at the measurement position TP.

[0025] Figure 2 It is a schematic diagram showing an example of the three-dimensional measuring device 1 according to the embodiment. Figure 3 It is a schematic diagram showing an example of the three-dimensional measuring device 1 according to the embodiment. Figure 2 It is from Figure 1 The diagram of the three-dimensional measuring device 1 showing the components of the three-dimensional measuring device 1 as viewed from the same direction. Figure 3 This is a diagram of the three-dimensional measuring device 1 as a component thereof, as viewed from the upper side in the vertical direction, that is, from the +Z direction.

[0026] In the embodiment, the component C includes a first identification object R1 and a second identification object R2 having a lower light reflectance than the first identification object R1. The three-dimensional measuring device 1 performs three-dimensional measurement of both the first identification object R1 and the second identification object R2. The details of the component C, the first identification object R1, and the second identification object R2 will be described later.

[0027] like Figure 2 As shown, the projection device 3 projects pattern light PL with different phases onto the component C. The projection device 3 includes: a light source 31 for generating light; a light modulator 32 for modulating the light generated by the light source 31 to generate pattern light PL; and an emission optical system 33 for emitting the pattern light PL generated by the light modulator 32.

[0028] The light modulator 32 includes a digital micromirror device (DMD). In addition, the light modulator 32 may include a transmissive liquid crystal panel or a reflective liquid crystal panel. The light modulator 32 generates pattern light PL based on pattern data output from the processing device 5. The projection device 3 emits pattern light PL patterned based on the pattern data.

[0029] The projection device 3 projects pattern light PL onto the component C from multiple directions. The projection device 3 includes a reflection member 6, which changes the projection direction by reflecting the emitted light. The reflection member 6 reflects the pattern light PL emitted from the projection device 3 and irradiates the component C. The reflection surface of the reflection member 6 is flat.

[0030] The imaging device 4 includes an imaging optical system 41 for imaging the pattern light PL reflected by the component C, and an imaging element 42 for acquiring image data of the component C via the imaging optical system 41. The imaging element 42 is a solid imaging element including a CMOS image sensor (Complementary Metal Oxide Semiconductor Image Sensor) or a CCD image sensor (Charge Coupled Device Image Sensor).

[0031] In the embodiment, the imaging optical system 41 is a telecentric optical system. A telecentric optical system is an optical system in which the principal ray is parallel to the optical axis AX. Therefore, the light (principal ray) incident on the imaging field of the imaging device 4 and imaged is considered to be parallel to the optical axis AX.

[0032] In the embodiment, the imaging device 4 captures the first identification object R1 and the second identification object R2 onto which the pattern light PL is projected. The imaging device 4 generates the first image data 61 (see FIG. 6 ) obtained by capturing the first identification object R1 onto which the pattern light PL is projected through the first imaging. Figure 7 ). The imaging device 4 generates the second image data 62 (see FIG. 1 ) obtained by imaging the second identification object R2 onto which the pattern light PL is projected, through the second imaging. Figure 7 ).

[0033] The processing device 5 includes a computer system, and controls the projection device 3 and the shooting device 4. The processing device 5 has: an operation processing device, including a processor such as a CPU (Central Processing Unit); a storage device, including a memory and a memory such as a ROM (Read Only Memory) or a RAM (Random Access Memory); and an input / output interface, including an input / output circuit capable of inputting and outputting signals and data. The operation processing device performs operation processing according to a computer program stored in the storage device.

[0034] In the embodiment, the processing device 5 performs image recognition on the first recognition object R1 based on the first image data 61 of the first recognition object R1, and performs image recognition on the second recognition object R2 based on the second image data 62 of the second recognition object R2. In addition, in this specification, image recognition refers to obtaining at least one of the position and shape of the recognition object by three-dimensional measurement. Performing image recognition includes performing three-dimensional measurement.

[0035] The three-dimensional measuring device 1 measures the three-dimensional shape of the identification object (the first identification object R1, the second identification object R2) of the component C held by the suction nozzle 11 and arranged at the measurement position TP based on the pattern projection method. The projection device 3 irradiates the component C with pattern light PL, for example, stripe pattern light with a sinusoidal brightness distribution, while shifting the phase. As described above, the pattern light PL irradiates the component C from multiple directions.

[0036] The imaging device 4 acquires image data of the component C onto which the pattern light PL is projected. The imaging device 4 acquires at least image data of the component C onto which the first pattern light PL1 is projected and image data of the component C onto which the second pattern light PL2 is projected. The incident direction of the first pattern light PL1 incident on the component C is different from the incident direction of the second pattern light PL2.

[0037] like Figure 2 As shown, the first pattern light PL1 is the pattern light PL emitted from the projection device 3 and directly irradiates the component. The second pattern light PL2 is the pattern light PL emitted from the projection device 3 and reflected by the reflection member 6 and irradiates the component C.

[0038] The processing device 5 controls the light modulation element 32 to change from a first irradiation state where the first pattern light PL1 from the projection device 3 irradiates the component C and a second irradiation state where the second pattern light PL2 from the reflection member 6 irradiates the component C to the other state.

[0039] When irradiating the first pattern light PL1 to the component C, the processing device 5 controls the light modulator 32 so that the pattern light PL is emitted from the first area 331 of the emission surface 33S of the emission optical system 33, and the pattern light PL is not emitted from the second area 332 of the emission surface 33S. The pattern light PL emitted from the first area 331 is directly irradiated to the component as the first pattern light PL1 without passing through the reflection member 6.

[0040] When irradiating the second pattern light PL2 to the component C, the processing device 5 controls the light modulator 32 so that the pattern light PL is emitted from the second area 332 of the emission surface 33S of the emission optical system 33, and the pattern light PL is not emitted from the first area 331 of the emission surface 33S. The pattern light PL emitted from the second area 332 passes through the optical axis AX and is irradiated to the reflective member 6. The pattern light PL emitted from the second area 332 is irradiated to the component C as the second pattern light PL2 via the reflective member 6.

[0041] exist Figure 1 and Figure 2 In the example shown, the first region 331 is a region of half of the emission surface 33S on the +X side with respect to the optical axis of the emission optical system 33, and the second region 332 is a region of half of the emission surface 33S on the -X side with respect to the optical axis of the emission optical system 33. The pattern light PL emitted from the first region 331 is irradiated to the component C in a state held by the suction nozzle 11 and arranged at the measurement position TP. The pattern light PL emitted from the second region 332 is irradiated to the reflection surface of the reflection member 6, and after being reflected by the reflection surface of the reflection member 6, it is irradiated to the component C in a state held by the suction nozzle 11 and arranged at the measurement position TP.

[0042] like Figure 1 and Figure 2 As shown, the projection device 3, the imaging device 4 and the reflection member 6 are supported by a housing 7. The relative positions of the projection device 3, the imaging device 4 and the reflection member 6 are fixed by the housing 7. The incident surface 41S of the imaging device 4 can face the component C arranged at the measurement position TP.

[0043] The reflecting member 6 is disposed at least partially around the optical axis AX of the imaging optical system 41 between the incident surface 41S and the component C disposed at the measurement position TP. The projection device 3 is disposed at least partially around the optical axis AX of the imaging optical system 41 between the incident surface 41S and the component C disposed at the measurement position TP. The reflecting member 6 is disposed at a position closer to the measurement position TP than the projection device 3.

[0044] like Figure 3 As shown in FIG. 1 , two projection devices 3 are arranged around the optical axis AX. In addition, two reflection members 6 are arranged around the optical axis AX.

[0045] The first reflecting member 6 is arranged on the -X side of the optical axis AX. The first projection device 3 is arranged on the +X side of the optical axis AX. The first projection device 3 can irradiate the second pattern light PL2 to the first reflecting member 6. The second pattern light PL2 reflected by the first reflecting member 6 is irradiated to the component C arranged at the measurement position TP. In addition, the first projection device 3 can directly irradiate the first pattern light PL1 to the component C arranged at the measurement position TP.

[0046] The second reflecting member 6 is arranged on the +Y side of the optical axis AX. The second projection device 3 is arranged on the -Y side of the optical axis AX. The second projection device 3 can irradiate the second pattern light PL2 to the second reflecting member 6. The second pattern light PL2 reflected by the second reflecting member 6 is irradiated to the component C arranged at the measurement position TP. In addition, the second projection device 3 can directly irradiate the first pattern light PL1 to the component C arranged at the measurement position TP.

[0047] In this way, the first and second projection devices 3 and the first and second reflection members 6 can project the pattern light PL from four directions to the component C arranged at the measurement position TP. The imaging device 4 obtains the image data of the component C onto which the first pattern light PL1 from the first projection device 3 is projected, the image data of the component C onto which the second pattern light PL2 from the first reflection member 6 is projected, the image data of the component C onto which the first pattern light PL1 from the second projection device 3 is projected, and the image data of the component C onto which the second pattern light PL2 from the second reflection member 6 is projected.

[0048] As described above, in the present embodiment, the projection device 3 and the reflection member 6 have two groups, one group is arranged opposite to each other in the X-axis direction, and the other group is arranged opposite to each other in the Y-axis direction, but the present invention is not limited to this, and may have one group or more than three groups. In addition, the three-dimensional measuring device 1 may also have a plurality of projection devices 3 arranged separately (without the reflection member 6) in each projection direction.

[0049] In addition, in this embodiment, a three-dimensional measuring device 1 using a phase shift method is used as a preferred three-dimensional measuring device. The three-dimensional measuring device 1 includes a projection device consisting of a projection device 3 and a reflection component 6, and a shooting device 4, but the present invention is not limited to this, and a three-dimensional measuring device using a shooting image analysis method can also be used.

[0050] <Control device> Figure 4 1 is a functional block diagram showing an example of the processing device 5 of the embodiment. The processing device 5 includes an input / output unit 51, a pattern generating unit 52, an image data acquiring unit 53, a phase value calculating unit 54, and a three-dimensional shape calculating unit 55. The processing device 5 also includes a storage unit 56.

[0051] The input / output unit 51 is a functional unit realized by executing a three-dimensional measurement program through the input / output interface of the processing device 5 . The three-dimensional measurement program is used to cause the three-dimensional measurement device 1 of the embodiment to execute the three-dimensional measurement method of the embodiment.

[0052] The pattern generating unit 52, the image data acquiring unit 53, the phase value calculating unit 54, and the three-dimensional shape calculating unit 55 are functional units realized by executing a three-dimensional measurement program by the arithmetic processing unit of the processing device 5, and the three-dimensional measurement program is used to make the three-dimensional measurement device 1 of the embodiment perform the three-dimensional measurement method of the embodiment. The arithmetic processing unit has a processor such as a CPU (Central Processing Unit), and a main memory including a non-volatile memory of a ROM (Read Only Memory) and a volatile memory of a RAM (Random Access Memory).

[0053] The storage unit 56 includes a nonvolatile storage device such as a semiconductor memory. The storage unit 56 stores programs executed in the input / output unit 51, the pattern generation unit 52, the image data acquisition unit 53, the phase value calculation unit 54, and the three-dimensional shape calculation unit 55. The storage unit 56 stores component data as design information of the component C. Information such as the shape and size of the component C is recorded in the component data.

[0054] The pattern generating unit 52 generates pattern data. The pattern data generated by the pattern generating unit 52 is output to the light modulating element 32 via the input / output unit 51. The light modulating element 32 generates pattern light PL based on the pattern data generated by the pattern generating unit 52. The pattern data generated by the pattern generating unit 52 includes: first pattern data for irradiating the first pattern light PL1 from the projection device 3 to the component C without passing through the reflection member 6; and second pattern data for irradiating the second pattern light PL2 from the projection device 3 to the component C via the reflection member 6.

[0055] Figure 5 Schematic diagram showing an example of pattern data of this embodiment. Figure 5 In the figure, black is used to represent the area where no light is emitted, and white is used to represent the area where light is emitted. Figure 5 As shown in (A), when the first pattern light PL1 is irradiated to component C, the pattern generating unit 52 generates the first pattern data and controls the light modulating element 32 so that the pattern light PL is emitted from the first area 331 of the emission surface 33S of the emission optical system 33, and the pattern light PL is not emitted from the second area 332 of the emission surface 33S.

[0056] like Figure 5As shown in (B), when the pattern generating unit 52 irradiates the second pattern light PL2 onto the component C, it generates the second pattern data and controls the light modulating element 32 so that the pattern light PL is emitted from the second area 332 of the emission surface 33S of the emission optical system 33, and the pattern light PL is not emitted from the first area 331 of the emission surface 33S.

[0057] The image data acquisition unit 53 acquires image data from the imaging element 42 via the input / output unit 51. The image data acquisition unit 53 acquires image data of the component C onto which the first pattern light PL1 is projected by the first and second projection devices 3 and image data of the component C onto which the second pattern light PL2 is projected from the first and second reflection members 6.

[0058] The phase value calculation unit 54 calculates the phase value of each of the plurality of pixels of the image data based on the brightness (pixel value) of the image data. The phase value calculation unit 54 calculates the phase value of the pixel of the image data corresponding to the point based on the brightness of the same point of the plurality of image data of the component C onto which each of the patterned lights PL after phase shift is projected. The phase value calculation unit 54 calculates the phase value of each of the plurality of pixels of the image data based on the brightness of each of the plurality of points of the image data.

[0059] The three-dimensional shape calculation unit 55 calculates the three-dimensional shape of the component C by calculating position data of each of the plurality of points of the component C corresponding to the plurality of pixels of the image data, based on the phase values ​​of the plurality of pixels of the image data.

[0060] In more detail, the three-dimensional shape calculation unit 55 calculates the height (protruding length) of the first identification object R1 and the second identification object R2 based on the position data of the front end of the component C, and calculates the overall length of the component C in the vertical direction, that is, the component height, the height data of the first identification object R1, and the height data of the second identification object R2.

[0061] <Components> Figure 6 1 is a side view (A) and a bottom view (B) showing an example of a component C of an embodiment. In one example, component C is a connector component mounted on a substrate P. Component C has a main body C1 and an electrode C2 extending from the main body C1. The main body C1 has a boss C3 for positioning and a threaded hole C4 for fixing. Component C has a straight shape that is long in a predetermined direction as a whole.

[0062] The component C has two terminal parts C11 on which a plurality of electrodes C2 are arranged and three fixing parts C12 for fixing the component C. The three fixing parts C12 are arranged linearly at intervals. One terminal part C11 is arranged between the two fixing parts C12 at the center and one end, and another terminal part C11 is arranged between the two fixing parts C12 at the center and the other end.

[0063] The main body C1 is made of a dark black resin. The boss C3 and the threaded hole C4 are provided on the lower surface C13 of the main body C1. The boss C3 and the threaded hole C4 are provided one each in the three fixing parts C12. The boss C3 is a cylindrical protrusion protruding downward from the lower surface C13 of the main body C1. The boss C3 is formed integrally with the main body C1 and is made of a dark black resin. The threaded hole C4 is provided in the main body C1 by fixing a metal internal thread part in a mounting hole formed in the main body C1.

[0064] The electrode C2 is a lead electrode protruding from the main body C1 of the component C. The electrode C2 is made of metal. A plurality of electrodes C2 are arranged in each of the two terminal portions C11. Each electrode C2 protrudes obliquely downward in a manner extending from the lower surface C13 of the main body C1 in the terminal portion C11 in the width direction of the component C. The front end (lower end) of the electrode C2 is flat and approximately horizontal (parallel to the lower surface C13 of the main body C1) so that it can be connected to the wiring pattern of the substrate P.

[0065] In the embodiment, the first identification object R1 is the front end of the lead electrode (i.e., electrode C2) protruding from the main body C1 of the component C. The first identification object R1 is the front end (lower surface) of each of the plurality of electrodes C2. The second identification object R2 is the front end of the boss C3 protruding from the main body C1 made of dark resin. The second identification object R2 is the front end (lower surface) of each of the three bosses C3.

[0066] exist Figure 6 In the example shown, the component C is a long strip extending in a predetermined direction, and the long side of the component C is larger than the field of view of the camera 4. Therefore, the three-dimensional measurement device 1 divides the entire component C into a plurality of regions, and performs image recognition (three-dimensional measurement) on each divided region. The number of divided regions of the component C corresponds to the size of the component C and the field of view of the camera 4. Figure 7 It is an explanatory diagram for explaining the photographing of the component C according to the embodiment. Figure 7 FIG. 4 shows an example of dividing the entire component C into three divided areas. Figure 7 , the entire component C is divided into divided areas CA1, CA2, and CA3. The divided area CA1 is an area including approximately 1 / 3 of one end of the component C. The divided area CA2 is an area including approximately 1 / 3 of the center of the component C. The divided area CA3 is an area including approximately 1 / 3 of the other end of the component C. Figure 7 The dotted line portion of the rectangle shown in FIG. 4 represents the field of view 4S of the camera 4. Figure 7 In the figure, for the sake of convenience, the field of view 4S is shown as being located at three locations of the divided areas CA1, CA2, and CA3, but in reality, the mounting head 22 moves the component C so that the divided areas CA1, CA2, and CA3 are respectively included in the field of view 4S.

[0067] The camera 4 takes pictures of the first recognition object R1 and the second recognition object R2 respectively when the segmented area CA1 is arranged in the field of view 4S. The camera 4 takes pictures of the first recognition object R1 and the second recognition object R2 respectively when the segmented area CA2 is arranged in the field of view 4S. The camera 4 takes pictures of the first recognition object R1 and the second recognition object R2 respectively when the segmented area CA3 is arranged in the field of view 4S. Thus, the first image data 61 of the first recognition object R1 in the segmented areas CA1, CA2 and CA3 are obtained respectively, and the second image data 62 of the second recognition object R2 in the segmented areas CA1, CA2 and CA3 are obtained respectively.

[0068] In the embodiment, the first image data 61 and the second image data 62 are images obtained by photographing the same range of the component C, but are images obtained by photographing under different photographing conditions according to the measurement conditions 70 described later.

[0069] <Measurement conditions> Since the entire electrode C2 including the first identification object R1 is made of metal, it has a relatively high light reflectivity. Since the boss C3 including the second identification object R2 is made of dark resin formed integrally with the body C1, it has a relatively low light reflectivity. Figure 7 In order to illustrate the difference in light reflectivity, the part with low light reflectivity (main body C1) made of dark resin is colored. However, although the boss C3 (the second identification object R2) is a part with low light reflectivity, coloring is omitted from the perspective of visual recognition of the figure. Since metal parts such as the electrode C2 have high light reflectivity, saturation (saturation of pixel values) is easily caused when shooting. For saturated pixels, the phase value (height data) cannot be obtained because the correct pixel value (brightness value) cannot be obtained. Therefore, in the three-dimensional measurement of the first identification object R1, it is important to prevent the first image data 61 from being saturated. On the other hand, in order not to hinder the identification of the electrode C2, the main body C1 including the second identification object R2 is made of dark resin, and it is difficult to give contrast. Therefore, even if the pattern light PL is projected onto the boss C3, it is difficult to obtain a contrast (brightness distribution) that can calculate the phase value. Thus, for the first identification object R1 and the second identification object R2, saturation easily occurs and it is important to suppress excess brightness, but contrast is difficult to form and it is important to increase brightness. Therefore, it is usually difficult to perform three-dimensional measurement using the same imaging system and the same optical system.

[0070] In the embodiment, the measurement conditions 70 (see Figure 8) is different between the first identification object R1 and the second identification object R2. Therefore, in the embodiment, the three-dimensional measurement of both the first identification object R1 and the second identification object R2 can be performed by the same three-dimensional measurement device 1 (projection device 3 and imaging device 4).

[0071] Figure 8 is a diagram showing a specific example of the measurement condition 70 according to the embodiment. Figure 8 In the example in , the measurement condition 70 includes a camera gain, a threshold value, and the number of images to be synthesized. The camera gain is the sensitivity of the imaging device 4 (the imaging element 42). The higher the camera gain, the higher the brightness value (pixel value) of the image data for the same amount of incident light. The threshold value is a threshold value applicable to a threshold processing that distinguishes between an identification object area and a background area in an image. The identification object area is an area of ​​the first identification object R1 and an area of ​​the second identification object R2. The background is an area outside the identification object. The threshold value is set to a value between the range of brightness values ​​of the identification object and the range of brightness values ​​of the background. The number of images to be synthesized is the number of image data synthesized in the synthesis processing described later. The more the number of synthesized sheets, the higher the brightness value (pixel value) of the synthesized image.

[0072] The measurement conditions 70 include a first measurement condition 70A for the first identification object R1 and a second measurement condition 70B for the second identification object R2. The camera gain of the first measurement condition 70A is set to a low value (or a standard value). The camera gain of the imaging device 4 is higher in the second measurement condition 70B of the second identification object R2 than in the first measurement condition 70A of the first identification object R1. As a result, the brightness value (pixel value) of the second identification object R2 becomes higher, and contrast is easily given.

[0073] The threshold value of the first measurement condition 70A is set to a high value (or a standard value). The processing device 5 cuts out the image portion for image recognition from the first image data 61 by threshold processing using the threshold value specified by the first measurement condition 70A. The threshold value of the first measurement condition 70A is set to cut out the image portion including the entire first recognition object R1 by threshold processing. The processing device 5 cuts out the image portion for image recognition from the second image data 62 by threshold processing using the threshold value specified by the second measurement condition 70B. The threshold value of the second measurement condition 70B is set to cut out the image portion including the entire second recognition object R2 by threshold processing. The second measurement condition 70B of the second recognition object R2 has a lower threshold value than the first measurement condition 70A of the first recognition object R1. As a result, in the second image data 62 where it is difficult to obtain the brightness value of the second recognition object R2, the possibility that the area of ​​the second recognition object R2 is processed as a background area can be reduced.

[0074] The number of images synthesized in the first measurement condition 70A is set to a prescribed standard number. The second measurement condition 70B of the second recognition object R2 has a larger number of images synthesized than the first measurement condition 70A of the first recognition object R1. In one example, the number of images synthesized in the first measurement condition 70A is 1, and the number of images synthesized in the second measurement condition 70B is a prescribed number of 2 or more. Therefore, the processing device 5 performs synthesis processing of a plurality of second image data 62 at least in the image recognition of the second recognition object R2. In addition, the image synthesis processing is performed when the number of images synthesized is set to 2 or more. The number of images synthesized being 1 means that image synthesis is not performed.

[0075] Fig. 9 : is a schematic diagram for illustrating the image synthesis processing of the embodiment. The projection device 3 projects the stripe pattern of the sinusoidal brightness distribution onto the component C while shifting the phase. The projection device 3 projects at least three stripe patterns with different phases onto the component C. In one example, the projection device 3 projects four stripe patterns with a phase difference of π / 2 onto the component C. That is, the stripe pattern includes a first stripe pattern with a phase shift of 0° (initial phase = 0), a second stripe pattern with a phase shift of 90° (initial phase = π / 2), a third stripe pattern with a phase shift of 180° (initial phase = π), and a fourth stripe pattern with a phase shift of 270° (initial phase = 3π / 2). The projection device 3 projects the first stripe pattern, the second stripe pattern, the third stripe pattern, and the fourth stripe pattern onto the component C in sequence.

[0076] Here, the image data before image synthesis is referred to as a base image. The image data after image synthesis is referred to as a synthesized image. The imaging device 4 captures the base images of the number of images synthesized set by the measurement condition 70 in the projection of each of the plurality of fringe patterns. When the number of images synthesized is N under the second measurement condition 70B, the base images are captured N times. Fig. 9 , an example where N=4 is shown.

[0077] When the first stripe pattern is projected onto component C, the image data acquisition unit 53 controls the imaging device 4 and the projection device 3 to capture component C onto which the first stripe pattern is projected N times. When the second stripe pattern is projected onto component C, the image data acquisition unit 53 controls the imaging device 4 to capture component C onto which the second stripe pattern is projected N times. The same is true when the third stripe pattern is projected onto component C and when the fourth stripe pattern is projected onto component C. The image data acquisition unit 53 acquires the first basic image captured by the first stripe pattern, the second basic image captured by the second stripe pattern, the third basic image captured by the third stripe pattern, and the fourth basic image captured by the fourth stripe pattern, which are the same number as the number of image synthesis sheets N (=4).

[0078] The image data acquisition unit 53 synthesizes the same number of basic images as the number of images to be synthesized N, and generates synthesized images corresponding to the plurality of stripe patterns. The image data acquisition unit 53 synthesizes N first basic images and generates a first synthesized image corresponding to the first stripe pattern. The image data acquisition unit 53 synthesizes N second basic images and generates a second synthesized image corresponding to the second stripe pattern. The image data acquisition unit 53 synthesizes N third basic images and generates a third synthesized image corresponding to the third stripe pattern. The image data acquisition unit 53 synthesizes N fourth basic images and generates a fourth synthesized image corresponding to the fourth stripe pattern.

[0079] In an embodiment, synthesizing a plurality of base images includes performing an addition process on the plurality of base images. The addition process of the plurality of base images includes adding the brightness values ​​of the same pixel of the plurality of base images. Thus, in the synthesized image, pixels with high brightness values ​​can be obtained. In each synthesized image, the brightness difference (i.e., contrast) between the bright and dark parts of the stripe pattern becomes larger. Therefore, it is easy to calculate the phase value even in the portion of the second identification object R2 where it is difficult to give contrast.

[0080] The above measurement conditions 70 are stored in the storage unit 56. When the processing device 5 performs three-dimensional measurement of the first recognition object R1, the first measurement condition 70A is obtained from the storage unit 56, and the image recognition is performed based on the first measurement condition 70A. When the processing device 5 performs three-dimensional measurement of the second recognition object R2, the second measurement condition 70B is obtained from the storage unit 56, and the image recognition is performed based on the second measurement condition 70B.

[0081] <Image Recognition> Fig.10 This is an explanatory diagram for explaining a recognition algorithm of an image recognition process executed by the processing device 5 of the embodiment. The processing device 5 performs image recognition by executing different recognition algorithms on the first recognition target R1 and the second recognition target R2.

[0082] The phase value calculation unit 54 executes the electrode recognition algorithm 71A when performing three-dimensional measurement of the first recognition object R1. The electrode recognition algorithm 71A is most suitable for recognizing the first recognition object R1, that is, the front end of the electrode C2 of the component C. The electrode recognition algorithm 71A is suitable for recognizing the first recognition object R1 of a known shape based on the component data. The shape of the first recognition object R1 is, for example, a rectangle.

[0083] The phase value calculation unit 54 executes the boss recognition algorithm 71B when performing three-dimensional measurement of the second recognition object R2. The boss recognition algorithm 71B is most suitable for recognizing the second recognition object R2, that is, the front end of the boss C3 formed on the body C1 of the component C. The boss recognition algorithm 71B is suitable for recognizing the second recognition object R2 of a known shape based on the component data. The shape of the second recognition object R2 is, for example, circular.

[0084] The electrode recognition algorithm 71A and the boss recognition algorithm 71B are stored in the storage unit 56. The processing device 5 selectively reads the recognition algorithm corresponding to the recognition object from the storage unit 56 and performs image recognition. The phase value calculation unit 54 generates measurement data representing the phase value of each pixel constituting the recognition object (the first recognition object R1 and the second recognition object R2) through image recognition.

[0085] <Completion processing> Fig.11 1 is an explanatory diagram showing an example of the first complement processing of image recognition performed by the processing device 5 of the embodiment. As described above, in the embodiment, the projection device 3 can irradiate the pattern light PL from four directions to obtain four image data with different irradiation directions of the pattern light PL. That is, Fig. 9 The acquisition of the basic image shown in FIG. 1 , the generation of the composite image (only when performing the composite process), and the calculation of the three-dimensional shape are performed four times in each direction. Fig.11 As shown, the processing device 5 generates four types of measurement data for each direction. Fig.11 An example of first direction measurement data 81A, second direction measurement data 81B, third direction measurement data 81C, and fourth direction measurement data 81D generated by image recognition of one recognition target is shown.

[0086] In each measurement data, due to the difference in the irradiation direction of the pattern light PL, an error area 82 where the height data (also called phase value) cannot be calculated may be generated. The error area 82 is a saturated area caused by direct incidence of regular reflected light on the camera 4, an area where light is blocked by other parts of the component C and becomes a shadow, etc., depending on the relationship between the shape and orientation of the recognition object and the irradiation direction of the pattern light PL.

[0087] In the embodiment, when a part of the image data based on the pattern light PL in multiple directions fails to be recognized, the processing device 5 performs image recognition based on other image data that have been successfully recognized. That is, even if an error area 82 is generated in any one of the first direction measurement data 81A, the second direction measurement data 81B, the third direction measurement data 81C, and the fourth direction measurement data 81D, when the height data of the same area in the measurement data in other directions can be calculated, the processing device 5 performs the first complementation processing of complementing the measurement result (height data) based on the height data.

[0088] For example, suppose Fig.11 The circle shown in the upper part of is the true shape of the identification object (for example, the second identification object R2). Fig.11 In each measurement data shown in the middle of FIG, the white area represents the recognition area 83 for which the height data can be calculated (recognition is successful). In each measurement data, the area surrounded by the dotted line adjacent to the recognition area 83 (the difference area between the true shape and the recognition area 83) is the error area 82. In each measurement data, the outside of the recognition area 83 and the error area 82 (the black area) is the background part.

[0089] exist Fig.11 In the example in FIG. 8 , the error region 82 is formed on the lower side of the figure in the first direction measurement data 81A, the error region 82 is formed on the upper side of the figure in the second direction measurement data 81B, the error region 82 is formed on the right side of the figure in the third direction measurement data 81C, and the error region 82 is formed on the left side of the figure in the fourth direction measurement data 81D. Fig.11 In the example in , an error region 82 is partially formed in any of the measurement data, and there is no measurement data that obtains the true shape of the recognition object.

[0090] The processing device 5 synthesizes the measurement data of each direction by obtaining the logical sum of each identification area 83 of the first direction measurement data 81A, the second direction measurement data 81B, the third direction measurement data 81C and the fourth direction measurement data 81D, and generates Fig.11The synthesized measurement data 80 is shown in the lower part of . The processing device 5 generates height data of the position (pixel) of the error area 82 belonging to the first direction measurement data 81A using, for example, the height data of the identification area 83 of the second direction measurement data 81B, the third direction measurement data 81C, and the fourth direction measurement data 81D. In a case where multiple height data can be obtained for the same position, the processing device 5 calculates a representative value (such as an average value, etc.) of the multiple height data as the height data of the position. In a case where only one height data can be obtained, the processing device 5 uses the value of the height data as the height data of the position. As a result, in the synthesized measurement data 80, a value equivalent to Fig.11 The upper part shows the overall height data of the area of ​​the recognized object's true shape.

[0091] Fig.12 1 is an explanatory diagram showing an example of the second complementation process of image recognition performed by the processing device 5 of the embodiment. The above-mentioned first complementation process is applied to the error area 82 where the image recognition is successful in at least one of the measurement data in the four directions and the height data (phase value) can be obtained. On the other hand, for example, in the case where the front end portion of the electrode C2 (the first recognition object R1) is mirror-like, due to saturation, it is possible that the height data (phase value) cannot be obtained in all the measurement data in the four directions.

[0092] Therefore, in the embodiment, when the recognition of the saturated region 84 fails because the saturated region 84 is included in a part of the first image data 61, the processing device 5 performs the second complementation process. In the second complementation process, the processing device 5 determines whether the saturated region 84 is a part of the first recognition target R1 based on the shape data of the component C. Then, when the processing device 5 determines that the saturated region 84 is a part of the first recognition target R1, it performs a process of complementing the saturated region 84 using the recognition result of the first recognition target R1. In addition, in the image data (first image data 61), the saturated region 84 is a pixel region where the brightness value (pixel value) reaches the maximum value.

[0093] Specifically, if Fig.12 As shown in FIG. 1 , it is assumed that a saturated region 84 exists in a portion of the first image data 61. Fig.12 In the first image data 61 in FIG. 1 , the phase of the stripe pattern is different in the rectangular range in the center and the peripheral portion. The rectangular range in the center represents the first recognition object R1 (i.e., the lower surface of the electrode C2), and the peripheral portion is the background area BG. Fig.12 In FIG. 8 , a portion of the left end of the first identification target R1 in the figure becomes a saturated area 84. Since the pixel value (luminance value) of each pixel included in the saturated area 84 is the maximum value, the phase of the stripe pattern cannot be identified. Fig.12Although the first image data 61 in any one irradiation direction is exemplified, the saturated region 84 is formed at the same position also in the first image data 61 in other irradiation directions.

[0094] In this case, in the measurement data of each direction (first direction measurement data 81A to fourth direction measurement data 81D), the height data (phase value) cannot be obtained in the saturation region 84, and thus it becomes an error region 82. Fig.12 In FIG. 1 , only the first direction measurement data 81A is illustrated. The area other than the error area 82 in the first recognition object R1 becomes the recognition area 83 from which the height data (phase value) can be obtained. When the saturation area 84 becomes the error area 82 in all the measurement data in the four directions (the first direction measurement data 81A to the fourth direction measurement data 81D), the first complementation process described above cannot be applied.

[0095] In this case, the processing device 5 performs a determination process on whether the saturated area 84 is a part of the first identification target R1 based on the component data 72 stored in the storage unit 56. The processing device 5 specifies the part of the component C reflected in the first image data 61 based on the relationship between the position of the first identification target R1 and the position of the background area BG (subject C1, etc.) obtained from the component data 72 and the position of the imaging device 4. That is, the processing device 5 performs a determination process from the position corresponding to the pixel position of the saturated area 84 along the optical axis AX (refer to Figure 2 ) is performed to determine whether the first identification object R1 exists on the optical path extending from the optical axis 104 to the optical axis 106. If the first identification object R1 exists, the saturated area 84 is determined to be a part of the first identification object R1. If the first identification object R1 does not exist on the optical path, the processing device 5 determines that the saturated area 84 is not a part of the first identification object R1. In the embodiment, the imaging optical system 41 is a telecentric optical system. Unlike other optical systems where the main ray and the optical axis AX are not parallel, the correction of the viewing angle of the lens, etc. is not required, so the amount of calculation for the judgment processing can be reduced.

[0096] When it is determined that the saturated region 84 is a part of the first identification object R1, the processing device 5 complements the height data (phase value) of the saturated region 84 with the value of the height data (phase value) of the other identification regions 83 of the first identification object R1. As a result, in the measurement data 80 after the second complementation process, the height data of the entire first identification object R1 including the saturated region 84 is calculated.

[0097] On the other hand, when it is determined that the saturated region 84 is not a part of the first recognition target R1, the processing device 5 maintains the saturated region 84 as the error region 82. In this case, in the measurement data 80 after the second complementation process, the saturated region 84 is processed as the background region BG.

[0098] <Three-dimensional measurement method> Hereinafter, the operation of the three-dimensional measuring device 1 according to the embodiment will be described. Fig.13 3D measurement method according to an embodiment of the present invention. Fig.13 A three-dimensional measuring method according to the embodiment executed by the three-dimensional measuring device 1 will be described.

[0099] like Fig.13 As shown, the three-dimensional measurement method of the embodiment generally includes a photographing step S10, an image synthesis step S20, an image recognition step S30, and a complement processing step S40.

[0100] In the imaging step S10, the three-dimensional measuring device 1 performs a first imaging of the first identification target R1 of the component C (step S11) and a second imaging of the second identification target R2 (step S12). Fig.13 In the embodiment, the processes are performed in the order of the first shooting (step S11) and the second shooting (step S12), but these processes may be performed in the reverse order, that is, the order of the second shooting and the first shooting.

[0101] First, the three-dimensional measuring device 1 performs a first imaging operation (step S11) with any segmented area (e.g., segmented area CA1) of the component C arranged within the field of view 4S by the mounting head 22. The processing device 5 controls the acquisition of the first image data 61 of the first recognition object R1 based on the first measurement condition 70A.

[0102] The processing device 5 controls the first and second projection devices 3 to emit the first pattern light PL1 and the second pattern light PL2 based on each of the first pattern data and the second pattern data generated by the pattern generating unit 52, and irradiate the component C. The image data acquiring unit 53 acquires the first image data 61 of the component C on which the first pattern light PL1 and the second pattern light PL2 in four directions are projected, respectively, with the camera gain specified in the first measurement condition 70A. The image data acquiring unit 53 acquires each of the first image data 61 in four directions with a number of sheets equal to the number of image synthesis sheets specified in the first measurement condition 70A. The number of image synthesis sheets N of the first measurement condition 70A is, for example, 1 sheet (image synthesis is not required).

[0103] The processing device 5 controls the first and second projection devices 3 to obtain the first image data 61 projecting four fringe patterns with different phases from the first fringe pattern to the fourth fringe pattern. That is, for each of the four irradiation directions, the first image data 61 of four phase patterns with the image synthesis number N (=1) are obtained respectively.

[0104] The three-dimensional measuring device 1 performs a second imaging operation on the same divided area (for example, divided area CA1) (step S12). The processing device 5 performs control to acquire the second image data 62 of the second identification target R2 based on the second measurement condition 70B.

[0105] The processing device 5 controls the first and second projection devices 3 to emit the first pattern light PL1 and the second pattern light PL2, respectively, and irradiate the component C. The image data acquisition unit 53 acquires the second image data 62 of the component C on which the first pattern light PL1 and the second pattern light PL2 are projected in four directions, respectively, with the camera gain specified in the second measurement condition 70B. The image data acquisition unit 53 acquires the second image data 62 in four directions with the number of images equal to the number of images N specified in the second measurement condition 70B. The number of images N of the second measurement condition 70B is, for example, 4. Thus, the base image of the second image data 62 is captured.

[0106] The processing device 5 controls the first and second projection devices 3 to obtain the second image data 62 projecting four fringe patterns with different phases from the first fringe pattern to the fourth fringe pattern. That is, in each of the four irradiation directions, the second image data 62 (base image) of four phase patterns with the number of image synthesis sheets N (=4 sheets) are obtained.

[0107] The processing device 5 determines whether all the image data that should be acquired have been acquired (step S13). When the imaging of the three segmented areas CA1, CA2, and CA3 of the component C is completed, the processing device 5 determines that all the image data have been acquired. In the case where there are segmented areas that have not been photographed, the processing device 5 determines that the acquisition of all the image data has not been completed. In this case, the processing device 5 controls the mounting head 22 through the control device 29 to change the position of the component C so that the next segmented area (for example, the segmented area CA2) is arranged within the field of view range 4S (step S14).

[0108] The processing device 5 performs the first shooting (step S11) and the second shooting (step S12) on the segmented area CA2. The processing device 5 passes step S13 and changes the shooting position so that the next segmented area (for example, segmented area CA3) is arranged within the field of view range 4S (step S14). Similarly, the processing device 5 performs the first shooting (step S11) and the second shooting (step S12) on the segmented area CA3. Thus, since the shooting is completed for all the segmented areas, the processing device 5 passes step S13 and enters the image synthesis step S20.

[0109] When the number of image synthesis is 2 or more in the measurement condition, the processing device 5 performs synthesis processing of these image data (image synthesis step S20). In the embodiment, since the number of image synthesis N is set to 1 in the first measurement condition 70A, the processing device 5 does not perform image synthesis on the first image data 61. The processing device 5 generates 4 sets of 4 images from the first stripe pattern to the fourth stripe pattern in each irradiation direction as the first image data 61.

[0110] Since the number of images to be synthesized N=4 is set in the second measurement condition 70B, the processing device 5 performs image synthesis processing on the second image data 62. Fig. 9 As shown, the processing device 5 synthesizes the N basic images from the first stripe pattern to the fourth stripe pattern to generate the first synthesized image to the fourth synthesized image. The processing device 5 performs the synthesis processing of the basic images from the first stripe pattern to the fourth stripe pattern on each second image data 62 of the four irradiation directions. The processing device 5 generates four sets of the first synthesized image to the fourth synthesized image as the second image data 62 in each irradiation direction.

[0111] The processing device 5 performs image recognition on the first image data 61 and the second image data 62 (image recognition step S30). The phase value calculation unit 54 calculates the phase value of each of the plurality of pixels in the image data based on the brightness (pixel value) of each image data from the first stripe pattern to the fourth stripe pattern in each irradiation direction. The phase value calculation unit 54 cuts out the recognition target area from the first image data 61 at the threshold value specified by the first measurement condition 70A, and calculates the height data (phase value) of the first recognition target R1 by executing the electrode recognition algorithm 71A. The processing device 5 cuts out the recognition target area from the second image data 62 at the threshold value specified by the second measurement condition 70B, and calculates the phase value of the second recognition target R2 by executing the boss recognition algorithm 71B.

[0112] The three-dimensional shape calculation unit 55 calculates the height data of each pixel of the image data corresponding to each identification object according to the principle of triangulation based on the phase value. Here, the height data of each pixel of the image data corresponds to the height data of each point of the identification object of the component C one by one. The height data of each point of the identification object of the component C represents the coordinate value of each point of the identification object of the component C in the three-dimensional space. The three-dimensional shape calculation unit 55 generates the measurement data of the first identification object R1 and the measurement data of the second identification object R2 respectively.

[0113] When the error region 82 exists in the measurement data of the first identification object R1 and the measurement data of the second identification object R2, the processing device 5 performs a complementation process of the pixels belonging to the error region 82 (complementation process step S40). When the error region 82 exists in a part of the measurement data of each of the four irradiation directions, the processing device 5 performs the first complementation process when there is measurement data of other irradiation directions in which the same region is successfully recognized. When the saturated region 84 is included in a part of the first image data 61 and the recognition of the saturated region 84 fails in all the measurement data of the four irradiation directions, the processing device 5 performs the second complementation process. The processing device 5 determines whether the saturated region 84 is a part of the first identification object R1. When it is determined that the saturated region 84 is a part of the first identification object R1, the processing device 5 performs the second complementation process, which performs a process of complementing the saturated region 84 using the recognition result of the first identification object R1. In addition, when the error region 82 (saturated region 84) does not exist, the processing device 5 does not perform the first complementation process and the second complementation process.

[0114] As a result, the processing device 5 calculates the three-dimensional shapes of the first identification target R1 and the second identification target R2 based on the height data of each point of the first identification target R1 and the second identification target R2.

[0115] <Installation method> Fig.14 1 is a flowchart for explaining the operation of the electronic component mounting system 100 according to the embodiment. The operation of the electronic component mounting system 100 according to the embodiment will be explained below. Fig.14 As shown, the electronic component mounting system 100 performs a detection step S60, an approach step S70, and an insertion step S80.

[0116] The three-dimensional measuring device 1 recognizes the first recognition object R1 (electrode C2) and the second recognition object R2 (boss C3) of the component C by executing the three-dimensional measuring method of the embodiment. The processing device 5 sends the measurement data of the first recognition object R1 and the second recognition object R2 to the control device 29. The control device 29 controls the mounting head 22 to adjust the relative position of the component C and the substrate P based on the height data of the first recognition object R1 and the second recognition object R2, and mounts the component C on the substrate P (detection step S60).

[0117] The control device 29 controls the head driving device 25 and the nozzle driving device 26 of the mounting head 22 based on the instructions from the three-dimensional measuring device 1, thereby making the front end of the lower side of the component C held by the suction nozzle 11 in the vertical direction approach the upper surface of the substrate P (approach step S70).

[0118] In the approach step S70, the control device 29 controls the nozzle driving device 26 of the mounting head 22 to move the nozzle 11 downward in the vertical direction based on the height data of the first identification object R1 and the second identification object R2 of the component C received from the three-dimensional measuring device 1, so that the front end of the lower side of the component C held by the suction nozzle 11 in the vertical direction approaches the upper surface of the substrate P.

[0119] In addition, in the approach step S70, the control device 29 controls the head driving device 25 and the nozzle driving device 26 of the mounting head 22 to move the mounting head 22 in the horizontal direction and rotate the nozzle 11 around the vertical direction based on the measurement data of the three-dimensional shape of the component C held by the nozzle 11 and the measurement data of the position measuring device 27 indicating the position of the through hole formed in the substrate P, so that the positions of the first identification object R1 and the second identification object R2 of the component C held by the nozzle 11 are consistent with the horizontal positions of the electrode connection positions and the through holes formed in the substrate P.

[0120] The control device 29 controls the nozzle driving device 26 of the mounting head 22 based on the instruction from the three-dimensional measuring device 1, thereby inserting the component C held by the nozzle 11 into the through hole formed in the substrate P (insertion step S80). The control device 29 controls the nozzle driving device 26 of the mounting head 22 to move the nozzle 11 downward in the vertical direction. As a result, the three bosses C3 of the component C are respectively inserted into the corresponding through holes of the substrate P, thereby positioning the component C on the substrate P. As a result, each electrode C2 of the component C is positioned at the electrode connection position of the substrate P. In addition, each electrode C2 of the component C may also be a pin-type electrode inserted into the electrode insertion hole of the substrate P.

[0121] <Effect> As described above, according to the present embodiment, the three-dimensional measuring device 1 includes: a projection device 3 for projecting pattern light PL of different phases onto a component C having a first identification object R1 and a second identification object R2 having a lower light reflectance than the first identification object R1; an imaging device 4 for imaging the first identification object R1 and the second identification object R2 onto which the pattern light PL is projected; and a processing device 5 for performing image recognition of the first identification object R1 based on the first image data 61 of the first identification object R1 and performing image recognition of the second identification object R2 based on the second image data 62 of the second identification object R2, wherein the measurement conditions 70 including the imaging conditions and the image recognition conditions are different for the first identification object R1 and the second identification object R2. Thus, the first identification object R1 having a relatively high light reflectance and the second identification object R2 having a relatively low light reflectance are respectively imaged under the measurement conditions suitable for each, so that the first image data 61 suitable for image recognition of the first identification object R1 which is easy to be saturated and the second image data 62 suitable for image recognition of the second identification object R2 which is difficult to give contrast can be obtained. Furthermore, the first image data 61 and the second image data 62 are image-recognized under respective measurement conditions, thereby enabling three-dimensional measurement of the first identification object R1 and the second identification object R2. Based on this, even when a component includes an identification object with high light reflectivity and an identification object with low light reflectivity, each identification object can be appropriately identified by the same measurement device.

[0122] According to the present embodiment, the second measurement condition 70B as the measurement condition of the second identification object R2 has a higher camera gain of the imaging device 4 than the first measurement condition 70A as the measurement condition of the first identification object R1. The second measurement condition 70B has a lower threshold for distinguishing the identification object area and the background area in the image than the first measurement condition 70A. The second measurement condition 70B has a larger number of image synthesis than the first measurement condition 70A. Thus, the first image data 61 of the first identification object R1, which is prone to saturation, can be generated with a relatively low gain, a high threshold, and a small number of synthesis. Thus, the saturation of the image data of the first identification object R1 can be suppressed, and the first identification object R1 and the background can be distinguished more accurately. The second image data 62 of the second identification object R2, which is difficult to give contrast, can be generated with a relatively high gain, a low threshold, and a large number of synthesis. Thus, the contrast of the image data of the second identification object R2 can be easily given, and the second identification object R2 and the background can be distinguished more accurately.

[0123] According to the present embodiment, when a part of the image data based on the pattern light PL in multiple directions fails to be recognized, the processing device 5 performs image recognition based on other image data that has been successfully recognized (first supplementation processing). Thus, even if the image recognition of a part of the recognition object in the image data fails and an error area 82 is generated, if the image recognition is successful in the image data in other irradiation directions, the partial loss of the measurement data, i.e., the generation of the error area 82, can be suppressed by supplementing the measurement data.

[0124] According to the present embodiment, when the recognition of the saturated region 84 fails because the saturated region 84 is included in a part of the first image data 61, the processing device 5 determines whether the saturated region 84 is a part of the first recognition object R1 based on the shape data of the component C. When it is determined that the saturated region 84 is a part of the first recognition object R1, the processing (second complementation processing) of complementing the saturated region 84 using the recognition result of the first recognition object R1 is performed. Thus, even when the measurement data cannot be obtained because the saturated region 84 is generated in the part of the first recognition object R1 with high light reflectivity in the first image data 61, the recognition result of the saturated region 84 can be complemented. Thus, even if the first recognition object R1 is easily saturated due to the high light reflectivity, the contour shape of the first recognition object R1 can be accurately obtained.

[0125] According to the present embodiment, the first identification object R1 is the front end portion of the electrode C2 protruding from the main body C1 of the component C. The second identification object R2 is the front end portion of the boss C3 protruding from the main body C1 made of dark resin. Thus, even if a part of the main body C1 is made of dark resin that is difficult to provide contrast so as not to hinder image recognition of the electrode C2, image recognition can be performed together with the electrode C2 by the same measurement device.

[0126] [Other embodiments] In the above embodiment, an example is shown in which the three-dimensional measuring device 1 is installed in the electronic component mounting system 100 including the electronic component mounting device 2 . However, the three-dimensional measuring device 1 may be installed in, for example, an inspection device that inspects the components C. Description of reference numerals:

[0127] 1: 3D measuring device; 2: electronic component mounting device; 3: projection device; 4: imaging device; 4S: field of view; 5: processing device; 6: reflection member; 7: housing; 11: nozzle; 22: mounting head; 23: electronic component supply device; 24: substrate conveying device; 25: head driving device; 26: nozzle driving device; 27: position measuring device; 29: control device; 31: light source; 32: light modulation element; 33: emission optical system; 33S: emission surface; 41: imaging optical system; 41S: incident surface; 42: imaging element; 51: input / output unit; 52: pattern generating unit; 53: image data acquiring unit; 54: phase value calculating unit; 55: 3D shape calculating unit; 56: storage unit; 61: first image data; 62: second image data; 70: measurement condition; 70A: first measurement condition; 70B: second measurement condition; 71A: electrode identification unit Algorithm; 71B: Algorithm for boss recognition; 72: Component data; 80: Measurement data; 81A: Measurement data in the first direction; 81B: Measurement data in the second direction; 81C: Measurement data in the third direction; 81D: Measurement data in the fourth direction; 82: Error area; 83: Recognition area; 84: Saturation area; 100: Electronic component mounting system; 331: First area; 332: Second area; AX: Optical axis; BG: Background area; C: Component; C1: Main body; C2: Electrode; C3: Boss; C4: Threaded hole; C11: Terminal part; C12: Fixing part; C13: Lower surface; CA1, CA2, CA3: Segmented area; MP: Mounting position; N: Number of images to be synthesized; P: Substrate; PL: Pattern light; PL1: First pattern light; PL2: Second pattern light; R1: First recognition object; R2: Second recognition object; SP: Supply position; TP: Measurement position.

Claims

1. A three-dimensional measuring device, characterized in that: have: A projection device projects pattern lights with different phases onto a component having a first identification object and a second identification object having a lower light reflectivity than the first identification object; a photographing device for photographing the first identification object and the second identification object respectively onto which the pattern light is projected; as well as a processing device that performs image recognition on the first recognition object based on the first image data of the first recognition object, and performs image recognition on the second recognition object based on the second image data of the second recognition object, The measurement conditions including the shooting conditions and the image recognition conditions are different between the first recognition target and the second recognition target.

2. The three-dimensional measuring device according to claim 1, characterized in that: The measurement condition of the second identification target is higher than the measurement condition of the first identification target in that the camera gain of the imaging device is higher.

3. The three-dimensional measuring device according to claim 1, characterized in that: The measurement condition of the second identification target has a lower threshold for distinguishing an identification target area from a background area in an image than the measurement condition of the first identification target.

4. The three-dimensional measuring device according to claim 1, characterized in that: The measurement condition of the second identification object has a greater number of photographing times than the measurement condition of the first identification object. The processing device performs synthesis processing of the plurality of second image data.

5. The three-dimensional measuring device according to any one of claims 1 to 4, characterized in that: The projection device projects the pattern light onto the component from multiple directions respectively. The photographing device uses the pattern lights from the multiple directions to respectively photograph. In the case where recognition of a portion of the image data based on the patterned light in the multiple directions fails, the processing device performs image recognition based on other image data that has been successfully recognized.

6. The three-dimensional measuring device according to any one of claims 1 to 4, characterized in that: When recognition of the saturated region fails because a part of the first image data includes a saturated region, the processing device determines whether the saturated region is a part of the first recognition target based on the shape data of the component. When it is determined that the saturated region is a part of the first recognition target, the processing device performs a process of completing the saturated region using a recognition result of the first recognition target.

7. The three-dimensional measuring device according to any one of claims 1 to 4, characterized in that: The first identification object is a tip end portion of an electrode protruding from a main body of the component, The second identification object is a front end portion of a boss protruding from the main body made of dark resin.

8. A three-dimensional measurement method, characterized in that: include: The step of projecting patterned light having different phases onto a component having a first identification object and a second identification object having a lower light reflectance than the first identification object, and performing a first photographing of the first identification object onto which the patterned light is projected; The step of projecting the pattern light onto the component and performing a second photographing of the second identification object onto which the pattern light is projected; as well as performing image recognition on the first recognition object based on the first image data of the first recognition object, and performing image recognition on the second recognition object based on the second image data of the second recognition object, The measurement conditions including the photographing conditions and the image recognition conditions are different between the first recognition target and the second recognition target.

Images