Component mounting device and component mounting method

Abstract

To suppress deterioration of productivity of a component mounting device.SOLUTION: A component mounting device comprises: a robot manipulator; a projection device that irradiates a stripe pattern to a lead component in a state where a body is held by a robot hand; an imaging device for performing an imaging; an arithmetic device that performs an imaging processing of imaging data; and a control device of the robot manipulator so that a lead of the lead component is inserted into a hole of the substrate on the basis of an imaging processing result of the arithmetic device. The arithmetic device includes: a three-dimentional image generation part that performs an arithmetic processing of the image data of the lead component on the basis of a phase shift method, and generates three-dimentional image data; a three-dimentional point group conversion part that converts the three-dimentional image data into the three-dimentional point group data; a segmentation part that segments the three-dimentional point group data and extracts lead point group data indicating a three-dimentional formation of a front surface of the lead from the three-dimentional point group data; and a lead state calculation part that analyzes a main component of the lead point group data and calculates the three-dimentional data of the lead.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The technology disclosed in this specification relates to a component mounting apparatus and a component mounting method.

Background Art

[0002] In the production of electronic devices, a component mounting apparatus for mounting components on a substrate is used. Patent Document 1 discloses a component mounting apparatus for mounting lead components on a substrate.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When the leads of a lead component are inserted into the holes of a substrate, the lead component is mounted on the substrate. Depending on the state of the leads, it may be difficult to insert the leads into the holes of the substrate, and as a result, the productivity of the component mounting apparatus may decrease. For example, when the leads are bent, it becomes difficult to insert the leads into the holes of the substrate, and as a result, there may be a situation where the mounting of the lead component has to be abandoned. In order to suppress the decrease in the productivity of the component mounting apparatus, a technology that can recognize the state of the leads and insert the leads into the holes of the substrate according to the state of the leads is desired.

Means for Solving the Problems

[0005] This specification discloses a component mounting device. The component mounting device includes a robot manipulator, a robot hand provided at the tip of the robot manipulator for holding the body of a lead component, a projection device for irradiating the lead component with stripe pattern light while the body is held by the robot hand, an imaging device for imaging the lead component irradiated with the stripe pattern light from a predetermined viewpoint, an arithmetic unit for performing image processing on the imaging data of the lead component captured by the imaging device, and a control unit for controlling the robot manipulator so that the leads of the lead component are inserted into the holes of the substrate based on the image processing result of the arithmetic unit. The arithmetic unit has a three-dimensional image generation unit that performs arithmetic processing on the imaging data of the lead component based on the phase shift method to generate three-dimensional image data, a three-dimensional point cloud conversion unit that converts the three-dimensional image data into three-dimensional point cloud data, a segmentation unit that segments the three-dimensional point cloud data and extracts lead point cloud data indicating the three-dimensional shape of the surface of the lead from the three-dimensional point cloud data, and a lead state calculation unit that performs principal component analysis on the lead point cloud data to calculate the three-dimensional data of the lead.

Advantages of the Invention

[0006] According to the component mounting device disclosed in this specification, a decrease in productivity is suppressed.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments according to the present disclosure will be described with reference to the drawings, but the present disclosure is not limited to the embodiments. The components of the embodiments described below can be combined as appropriate. Also, some components may not be used.

[0009] In the embodiment, a local coordinate system is set for the component mounting apparatus 1, and the positional relationship of each part will be described with reference to the local coordinate system. As the local coordinate system, an XYZ orthogonal coordinate system is set. In a predetermined plane, the direction parallel to the X-axis is defined as the X-axis direction. In the predetermined plane, the direction parallel to the Y-axis orthogonal to the X-axis is defined as the Y-axis direction. The direction parallel to the Z-axis orthogonal to each of the X-axis and the Y-axis is defined as the Z-axis direction. The rotational direction or tilting direction centered on the X-axis is defined as the θX direction. The rotational direction or tilting direction centered on the Y-axis is defined as the θY direction. The rotational direction or tilting direction centered on the Z-axis is defined as the θZ direction. The predetermined plane is the XY plane. The Z-axis is orthogonal to the predetermined plane. In the embodiment, the predetermined plane is assumed to be parallel to the horizontal plane. The Z-axis direction is the vertical direction. Note that the predetermined plane may be inclined with respect to the horizontal plane.

[0010] [Component mounting apparatus] FIG. 1 is a perspective view showing the component mounting apparatus 1 according to the embodiment. FIG. 2 is a side view showing the component mounting apparatus 1 according to the embodiment. As shown in FIGS. 1 and 2, the component mounting apparatus 1 includes a base 2, a component supply member 3, a substrate support member 4, a robot hand 5, a robot manipulator 6, and a three-dimensional measuring device 7.

[0011] The base 2 supports each of the component supply member 3, the substrate support member 4, the robot manipulator 6, and the three-dimensional measuring device 7.

[0012] The component supply member 3 supplies the lead components 100. In the embodiment, the component supply member 3 includes a tray on which the lead components 100 are arranged. A plurality of lead components 100 are arranged on the component supply member 3. The types of the plurality of lead components 100 may be the same or different.

[0013] The substrate support member 4 supports the substrate 200 on which the lead components 100 are mounted. The substrate support member 4 supports the substrate 200 such that the upper surface of the substrate 200 is parallel to the XY plane.

[0014] The robot hand 5 holds the lead component 100. The robot hand 5 is provided at the tip of the robot manipulator 6.

[0015] The robot manipulator 6 moves the robot hand 5. The robot manipulator 6 includes an articulated robot. In an embodiment, the robot manipulator 6 is a vertical articulated robot. Note that the robot manipulator 6 may be a horizontal articulated robot. The robot manipulator 6 has a base member 6A fixed to the base 2, a swivel member 6B supported by the base member 6A, a first arm 6C connected to the swivel member 6B, a second arm 6D connected to the first arm 6C, and a third arm 6E connected to the second arm 6D.

[0016] The swivel member 6B is supported by the base member 6A so as to be swivelable about the swivel axis TX. The swivel axis TX is parallel to the Z axis. The first arm 6C is connected to the swivel member 6B so as to be rotatable about the first rotation axis AX1. The first rotation axis AX1 is orthogonal to the Z axis. The second arm 6D is connected to the first arm 6C so as to be rotatable about the second rotation axis AX2. The second rotation axis AX2 is parallel to the first rotation axis AX1. The third arm 6E is connected to the second arm 6D so as to be rotatable about the third rotation axis AX3. The third rotation axis AX3 is parallel to the second rotation axis AX2. The robot hand 5 is attached to the third arm 6E.

[0017] The robot manipulator 6 has a swivel actuator that swivels the swivel member 6B, a first rotation actuator that rotates the first arm 6C, a second rotation actuator that rotates the second arm 6D, and a third rotation actuator that rotates the third arm 6E.

[0018] The three-dimensional measuring device 7 measures the lead component 100 held by the robot hand 5. The three-dimensional measuring device 7 detects the position of the lead component 100 in the local coordinate system based on the phase shift method.

[0019] [Robot Hand] FIG. 3 is a perspective view showing the robot hand 5 according to the embodiment. The robot hand 5 has a connecting member 5A attached to the third arm 6E, a rotating member 5B supported by the connecting member 5A, and a pair of moving members 5C supported by the rotating member 5B.

[0020] The rotating member 5B is rotatably supported by the connecting member 5A about the rotation axis RX. The rotation axis RX is orthogonal to the third rotation axis AX3. The pair of moving members 5C move in directions approaching and separating from each other. A grip portion 5D is provided at the lower end portion of the moving member 5C. The pair of grip portions 5D approach and separate from each other.

[0021] The robot hand 5 has a rotation actuator that rotates the rotating member 5B and a grip actuator that moves the pair of moving members 5C closer to or away from each other.

[0022] With the lead component 100 disposed between the pair of grip portions 5D, when the pair of grip portions 5D approach each other, the lead component 100 is held by the grip portions 5D. When the pair of grip portions 5D separate from each other, the lead component 100 is released from the grip portions 5D.

[0023] A force sensor 8 is disposed on one of the moving members 5C. The force sensor 8 can detect the load applied to the grip portion 5D.

[0024] [Lead component] FIG. 4 is a side view showing the lead component 100 held by the robot hand 5 according to the embodiment. FIG. 5 is a view of the lead component 100 according to the embodiment as seen from below.

[0025] The lead component 100 has a body 101 and a plurality of leads 110 protruding from the body 101.

[0026] The body 101 includes a housing made of synthetic resin. An element such as a coil is arranged in the internal space of the body 101. The lead 110 is a metal protrusion. The lead 110 is connected to, for example, an element arranged in the internal space of the body 101.

[0027] The lead 110 protrudes downward from the lower surface of the body 101. With the lead component 100 mounted on the substrate 200, the lower surface of the body 101 faces the upper surface of the substrate 200.

[0028] The robot hand 5 holds the body 101 of the lead component 100. A pair of grip portions 5D holds the lead component 100 by sandwiching the body 101.

[0029] [Three-dimensional measuring device] FIG. 6 is a perspective view showing a three-dimensional measuring device 7 according to the embodiment. As shown in FIG. 6, the three-dimensional measuring device 7 measures the three-dimensional shape of the lead component 100 with the body 101 held by the robot hand 5.

[0030] The three-dimensional measuring device 7 includes a projection device 7A, an imaging device 7B, and an arithmetic device 7C. In the embodiment, each of the projection device 7A and the imaging device 7B is housed in a housing 7D. Each of the projection device 7A and the imaging device 7B is fixed to the housing 7D. A transparent member 7E is arranged at an opening at the upper end of the housing 7D. A glass plate is exemplified as the transparent member 7E.

[0031] The projection device 7A irradiates the lead component 100 with stripe pattern light in a state where the body 101 is held by the robot hand 5. The projection device 7A includes a light source, a light modulation element that modulates light emitted from the light source to generate stripe pattern light, and an emission optical system that emits the stripe pattern light generated by the light modulation element. Examples of the light modulation element include a digital mirror device (DMD), a transmissive liquid crystal panel, or a reflective liquid crystal panel.

[0032] The imaging device 7B images the lead component 100 irradiated with the striped pattern light from a predetermined viewpoint. The viewpoint of the imaging device 7B refers to the relative imaging position and imaging angle of the imaging device 7B with respect to the lead component 100. The imaging device 7B includes an imaging optical system that forms an image of the striped pattern light reflected by the lead component 100, and an image sensor that acquires image data of the lead component 100 via the imaging optical system. Examples of the image sensor include a CMOS image sensor (Complementary Metal Oxide Semiconductor Image Sensor) or a CCD image sensor (Charge Coupled Device Image Sensor).

[0033] The arithmetic device 7C performs image processing on the imaging data of the lead component 100 imaged by the imaging device 7B. The arithmetic device 7C includes a computer system. The arithmetic device 7C includes a processor such as a CPU (Central Processing Unit), a memory such as a ROM (Read Only Memory) or a RAM (Random Access Memory), and an input / output interface including an input / output circuit capable of inputting and outputting signals and data.

[0034] The three-dimensional measurement device 7 measures the three-dimensional shape of the lead component 100 held by the robot hand 5 based on the phase shift method.

[0035] The projection device 7A irradiates the lead component 100 while phase-shifting, for example, the striped pattern light with a sinusoidal brightness distribution. The lead component 100 held by the robot hand 5 is disposed above the transparent member 7E. The striped pattern light emitted from the projection device 7A is irradiated onto the lead component 100 through the transparent member 7E.

[0036] The imaging device 7B images the lead component 100 irradiated with stripe pattern light. The imaging device 7B images the lead component 100 through the transparent member 7E. The imaging device 7B images the lead component 100 from below the lead component 100. The viewing point of the imaging device 7B is defined below the lead component 100. When the robot manipulator 6 operates and the position and orientation of the lead component 100 held by the robot hand 5 change, the relative position and relative angle between the lead component 100 and the imaging device 7B change. When the relative position and relative angle between the lead component 100 and the imaging device 7B change, the relative viewing point of the imaging device 7B with respect to the lead component 100 changes.

[0037] The arithmetic unit 7C performs image processing on the imaging data of the lead component 100 imaged by the imaging device 7B based on the phase shift method to calculate the three-dimensional data of the lead 110. The three-dimensional data of the lead 110 includes the amount of bend of the lead 110 and the coordinates of the tip of the lead 110 in a three-dimensional space defined in a local coordinate system. When the angle of the lead 110 with respect to the body 101 in the design value is defined as the ideal angle and the actual angle of the lead 110 with respect to the body 101 is defined as the actual angle, the amount of bend of the lead 110 refers to the difference between the ideal angle and the actual angle.

[0038] [Operation of Robot Manipulator] FIG. 7 is a diagram for explaining the operation of the robot manipulator 6 according to the embodiment. The component mounting device 1 includes a control device 9 that controls the robot manipulator 6. The control device 9 includes a computer system. As shown in FIG. 7, the substrate 200 is provided with holes 210 into which the leads 110 of the lead component 100 are inserted. The control device 9 controls the robot manipulator 6 based on the image processing result of the arithmetic unit 7C of the three-dimensional measurement device 7 so that the leads 110 of the lead component 100 are inserted into the holes 210 of the substrate 200.

[0039] [Arithmetic Unit] FIG. 8 is a block diagram showing the component mounting device 1 according to the embodiment. As shown in FIG. 8, the component mounting device 1 includes an arithmetic unit 7C and a control device 9.

[0040] The arithmetic unit 7C includes a three-dimensional image generation unit 11, a three-dimensional point group conversion unit 12, a three-dimensional point group integration unit 13, a segmentation unit 14, a candidate point group generation unit 15, a lead state calculation unit 16, and an output unit 17.

[0041] The three-dimensional image generation unit 11 acquires imaging data of the lead component 100 from the imaging device 7B. The three-dimensional image generation unit 11 performs arithmetic processing on the imaging data of the lead component 100 based on the phase shift method to generate three-dimensional image data Da of the lead component 100.

[0042] In the embodiment, the control device 9 controls the robot manipulator 6 so that the lead component 100 is imaged by the imaging device 7B from each of a plurality of viewpoints. The three-dimensional image generation unit 11 generates a plurality of three-dimensional image data Da of the lead component 100 as seen from each of the plurality of viewpoints based on the plurality of imaging data captured from each of the plurality of viewpoints.

[0043] The three-dimensional point group conversion unit 12 converts the three-dimensional image data Da generated by the three-dimensional image generation unit 11 into three-dimensional point group data Db. The three-dimensional point group data Db indicates the three-dimensional shape of the surface of the lead component 100. The three-dimensional point group data Db is an aggregate of a plurality of measurement points by the three-dimensional measuring device 7 on the surface of the lead component 100. The position of each of the plurality of measurement points is defined by an X coordinate, a Y coordinate, and a Z coordinate.

[0044] As described above, in the embodiment, the three-dimensional image generation unit 11 generates a plurality of three-dimensional image data Da based on a plurality of imaging data captured from each of a plurality of viewpoints. The three-dimensional point group conversion unit 12 generates a plurality of three-dimensional point group data Db of the lead component 100 as seen from each of the plurality of viewpoints based on the plurality of three-dimensional image data Da.

[0045] The three-dimensional point group integration unit 13 aligns the plurality of three-dimensional point group data Db generated by the three-dimensional point group conversion unit 12 based on a specified algorithm to generate integrated point group data Dc.

[0046] As a specified algorithm, the ICP (Iterative Closest Point) matching algorithm is exemplified. The three-dimensional point cloud integration unit 13 aligns a plurality of three-dimensional point cloud data Db in a three-dimensional space based on an existing algorithm such as the ICP matching algorithm, and generates integrated point cloud data Dc.

[0047] The segmentation unit 14 segments the three-dimensional point cloud data Db and extracts lead point cloud data Dd from the three-dimensional point cloud data Db. The lead point cloud data Dd indicates the three-dimensional shape of the surface of the lead 110. The surface of the lead 110 includes the outer peripheral surface and the lower end surface of the lead 110. By segmentation, in the three-dimensional point cloud data Db, the measurement points corresponding to the lead 110 and the measurement points corresponding to the background or noise are separated. That is, by segmentation, the measurement points corresponding to the lead 110 are extracted, and the measurement points corresponding to the background or noise are removed.

[0048] Segmentation refers to a process of separating a target point cloud from the point cloud constituting the three-dimensional point cloud data Db according to specified rules. In the embodiment, the segmentation unit 14 captures the feature points of the three-dimensional point cloud data Db and classifies the measurement points constituting the three-dimensional point cloud data Db into a plurality of groups. The segmentation unit 14 segments the three-dimensional point cloud data Db based on a specified segmentation method.

[0049] As the segmentation method, the LCCP (Locally Convex Connected Patches) method or the region growing method is exemplified. The segmentation unit 14 extracts the lead point cloud data Dd from the three-dimensional point cloud data Db based on an existing segmentation method such as the LCCP method or the region growing method.

[0050] In an embodiment, segmenting the three-dimensional point group data Db includes segmenting the integrated point group data Dc generated by the three-dimensional point group integration unit 13. The segmentation unit 14 segments the integrated point group data Dc generated by the three-dimensional point group integration unit 13, and extracts lead point group data Dd from the integrated point group data Dc.

[0051] Based on the design values of the lead component 100, the viewpoints of the imaging device 7B when the lead component 100 is imaged, and the specified connection conditions, the candidate point group generation unit 15 connects a plurality of segment point groups De to generate candidate point group data Df that is a candidate for the lead 110. Although the lead point group data Dd is extracted from the integrated point group data Dc by segmentation, the lead point group data Dd may be separated into a plurality of segment point groups De. When the lead point group data Dd is separated into a plurality of segment point groups De by segmentation, it may become unclear which segment point group De corresponds to the lead 110 and which segment point group De does not correspond to the lead 110. Therefore, the candidate point group generation unit 15 extracts a plurality of segment point groups De predicted to correspond to the lead 110 based on the design values of the lead component 100, the viewpoints of the imaging device 7B when the lead component 100 is imaged, and the specified connection conditions, and connects these plurality of segment point groups De to generate candidate point group data Df that is a candidate for the lead 110.

[0052] The design values of the lead component 100 are known data. The design values of the lead component 100 are stored in advance in the candidate point group generation unit 15. Examples of the design values of the lead component 100 include the length of the lead 110, the relative position between the body 101 and the lead 110, the angle (ideal angle) of the lead 110 with respect to the body 101, and the interval between a plurality of leads 110.

[0053] When the lead component 100 is imaged by the imaging device 7B, the candidate point group generation unit 15 can calculate the viewpoint of the imaging device 7B when the lead component 100 is imaged based on the control command output from the control device 9 to the robot manipulator 6. When a sensor capable of detecting the position and orientation of the robot manipulator 6 is provided on the robot manipulator 6, the candidate point group generation unit 15 can calculate the viewpoint of the imaging device 7B when the lead component 100 is imaged based on the detection value of the sensor.

[0054] The specified connection conditions are predefined. The specified connection conditions are stored in advance in the candidate point group generation unit 15. In the embodiment, the specified connection conditions include a first connection condition and a second connection condition. The first connection condition includes that the relative distance between two segment point groups De to be connected is equal to or less than a predefined specified distance. The second connection condition includes that when the segment point group De after connection is surrounded by an axis-aligned bounding box (AABB), the area of the smallest face of the axis-aligned bounding box satisfies the condition of [α < β × γ], where α is the area of the smallest face of the axis-aligned bounding box, β is the diameter of the lead 110, and γ is a coefficient. The axis-aligned bounding box refers to the boundary figure of a rectangular parallelepiped with sides parallel to the X-axis, Y-axis, and Z-axis of the three-dimensional space defined in the local coordinate system.

[0055] The lead state calculation unit 16 performs principal component analysis on the lead point group data Dd to calculate the three-dimensional data of the lead 110. The lead state calculation unit 16 calculates the amount of bend of the lead 110 and the coordinates of the tip of the lead 110 in the three-dimensional space defined in the local coordinate system as the three-dimensional data of the lead 110.

[0056] Principal component analysis refers to an analysis method in which the principal components are determined by maximizing the variance of the first principal component and, for the second and subsequent principal components, selecting them to maximize the variance under the condition of being orthogonal to the previous principal components.

[0057] In an embodiment, performing principal component analysis on the lead point group data Dd includes performing principal component analysis on the candidate point group data Df generated by the candidate point group generation unit 15. The lead state calculation unit 16 performs principal component analysis on the candidate point group data Df generated by the candidate point group generation unit 15 to calculate three-dimensional data of the lead 110.

[0058] In an embodiment, the lead state calculation unit 16 extracts, as the lead 110, a principal component that approximates the principal axis vector of the principal component of the candidate point group data Df and the coordinates of the tip of the lead 110 to the principal axis vector of the lead 110 and the coordinates of the tip of the lead 110 predicted based on the design value of the lead 110 and the viewpoint of the imaging device 7B when imaging the lead component 100.

[0059] The output unit 17 outputs the three-dimensional data of the lead 110 calculated by the lead state calculation unit 16 to the control device 9. The image processing result of the arithmetic device 7C includes the three-dimensional data of the lead 110 calculated by the lead state calculation unit 16. The control device 9 controls the robot manipulator 6 based on the three-dimensional data of the lead 110 so that the lead 110 is inserted into the hole 210 of the substrate 200.

[0060] [Image Processing Method] FIG. 9 is a flowchart showing an image processing method of the arithmetic device 7C according to the embodiment. To mount the lead component 100 on the substrate 200, the control device 9 controls the robot manipulator 6 so that the robot hand 5 approaches the component supply member 3. The robot hand 5 that has moved to the component supply member 3 holds the body 101 of the lead component 100 arranged on the component supply member 3. After the body 101 of the lead component 100 is held by the robot hand 5, the control device 9 controls the robot manipulator 6 so that the lead component 100 held by the robot hand 5 is measured by the three-dimensional measuring device 7. That is, the control device 9 controls the robot manipulator 6 so that the lead component 100 held by the robot hand 5 is arranged above the three-dimensional measuring device 7 as described with reference to FIG. 6.

[0061] In an embodiment, the control device 9 controls the robot manipulator 6 so that the lead component 100 is imaged by the imaging device 7B from each of a plurality of viewpoints. That is, the control device 9 controls the robot manipulator 6 so that the position and angle of the lead component 100 held by the robot hand 5 change above the three-dimensional measuring device 7. The imaging device 7B images the lead component 100 from each of a plurality of different viewpoints.

[0062] The three-dimensional image generation unit 11 acquires imaging data of the lead component 100 from the imaging device 7B (step S1).

[0063] The three-dimensional image generation unit 11 performs arithmetic processing on the imaging data of the lead component 100 based on the phase shift method to generate three-dimensional image data Da of the lead component 100. The three-dimensional image generation unit 11 generates a plurality of three-dimensional image data Da of the lead component 100 as viewed from each of a plurality of viewpoints based on the plurality of imaging data captured from each of the plurality of viewpoints (step S2).

[0064] The three-dimensional point cloud conversion unit 12 converts the three-dimensional image data Da into three-dimensional point cloud data Db. The three-dimensional point cloud conversion unit 12 generates a plurality of three-dimensional point cloud data Db of the lead component 100 as viewed from each of a plurality of viewpoints based on the plurality of three-dimensional image data Da generated in step S2 (step S3).

[0065] FIG. 10 is a diagram schematically showing an example of the three-dimensional point cloud data Db according to the embodiment. When the lead component 100 is imaged from a plurality of viewpoints, the three-dimensional point cloud conversion unit 12 generates a plurality of three-dimensional point cloud data Db of the lead component 100 as viewed from each of the plurality of viewpoints. For example, when the lead component 100 is imaged from three viewpoints, as shown in FIG. 10, the three-dimensional point cloud conversion unit 12 generates three-dimensional point cloud data Db1 of the lead component 100 as viewed from the first viewpoint, three-dimensional point cloud data Db2 of the lead component 100 as viewed from the second viewpoint, and three-dimensional point cloud data Db3 of the lead component 100 as viewed from the third viewpoint.

[0066] The three-dimensional point cloud integration unit 13 aligns the plurality of three-dimensional point cloud data Db generated in step S3 based on a specified algorithm to generate integrated point cloud data Dc (step S4).

[0067] FIG. 11 is a diagram schematically showing an example of the integrated point cloud data Dc according to the embodiment. As shown in FIG. 10, when three three-dimensional point cloud data Db (Db1, Db2, Db3) are generated, the three-dimensional point cloud integration unit 13 aligns the three three-dimensional point cloud data Db generated in step S3 in a three-dimensional space based on an existing algorithm such as the ICP matching algorithm to generate integrated point cloud data Dc.

[0068] The segmentation unit 14 segments the integrated point cloud data Dc generated in step S4 to extract lead point cloud data Dd from the integrated point cloud data Dc (step S5).

[0069] FIG. 12 is a diagram schematically showing an example of the lead point cloud data Dd according to the embodiment. The segmentation unit 14 segments the integrated point cloud data Dc based on an existing segmentation method such as the LCCP method or the region expansion method to extract lead point cloud data Dd from the integrated point cloud data Dc. As shown in FIG. 12, the lead point cloud data Dd is extracted by segmentation. Also, by segmentation, measurement points corresponding to the background or noise are removed.

[0070] Although the lead point cloud data Dd is extracted from the integrated point cloud data Dc by the segmentation in step S5, the lead point cloud data Dd may be separated into a plurality of segment point clouds De. The candidate point cloud generation unit 15 connects the plurality of segment point clouds De based on the design value of the lead component 100, the viewpoint of the imaging device 7B when the lead component 100 is imaged, and a specified connection condition to generate candidate point cloud data Df that is a candidate for the lead 110 (step S6).

[0071] FIG. 13 is a diagram for explaining the segment point group De according to the embodiment. As shown in FIG. 13, although the lead point group data Dd is extracted from the integrated point group data Dc by segmentation, the lead point group data Dd may be separated into a plurality of segment point groups De. In the example shown in FIG. 13, the segment point group De includes a first segment point group De1, a second segment point group De2, a third segment point group De3, and a fourth segment point group De4. When the lead point group data Dd is separated into a plurality of segment point groups De by segmentation, it may become unclear which segment point group De corresponds to the lead 110 and which segment point group De does not correspond to the lead 110. Therefore, the candidate point group generation unit 15 extracts a plurality of segment point groups De predicted to correspond to the lead 110 based on the design value of the lead component 100, the viewpoint of the imaging device 7B when the lead component 100 is imaged, and a specified connection condition, and connects these plurality of segment point groups De to generate candidate point group data Df that is a candidate for the lead 110.

[0072] As described above, in the embodiment, the specified connection condition includes a first connection condition and a second connection condition. The first connection condition includes that the relative distance between two segment point groups De to be connected is equal to or less than a specified distance determined in advance. The second connection condition includes that when the minimum area of the segment point group De after connection is surrounded by an axis-parallel bounding box, with the area of the minimum face being α, the diameter of the lead 110 being β, and the coefficient being γ, the condition [α < β × γ] is satisfied.

[0073] The two segment point groups De to be connected are determined based on the design value of the lead component 100 and the viewpoint of the imaging device 7B when the lead component 100 is imaged. The candidate point group generation unit 15 predicts the extending direction of the actual lead 110 based on the design value of the lead component 100 and the viewpoint of the imaging device 7B when the lead component 100 is imaged. The candidate point group generation unit 15 determines a plurality of segment point groups De arranged along the extending direction of the actual lead 110 as the connection targets.

[0074] In the example shown in FIG. 13, the first segment point group De1, the second segment point group De2, the third segment point group De3, and the fourth segment point group De4 are arranged such that the predicted actual lead 110 extends along the extending direction. Therefore, the first segment point group De1, the second segment point group De2, the third segment point group De3, and the fourth segment point group De4 are determined as connection targets. More specifically, the first segment point group De1 and the second segment point group De2 adjacent to the first segment point group De1 are determined as connection targets. The second segment point group De2 and the third segment point group De3 adjacent to the second segment point group De2 are determined as connection targets. The third segment point group De3 and the fourth segment point group De4 adjacent to the third segment point group De3 are determined as connection targets.

[0075] In the example shown in FIG. 13, focusing on the first connection condition, the first segment point group De1 and the second segment point group De2 are close to each other, and the relative distance between the first segment point group De1 and the second segment point group De2 is equal to or less than a specified distance. The second segment point group De2 and the third segment point group De3 are close to each other, and the relative distance between the second segment point group De2 and the third segment point group De3 is equal to or less than a specified distance. On the other hand, the third segment point group De3 and the fourth segment point group De4 are separated from each other, and the relative distance between the third segment point group De3 and the fourth segment point group De4 exceeds the specified distance. Therefore, in the example shown in FIG. 13, the fourth segment point group De4 is determined not to satisfy the first connection condition.

[0076] In the example shown in FIG. 13, focusing on the second connection condition, when the first segment point group De1, the second segment point group De2, and the third segment point group De3 are connected, the area α of the lower surface, which is the minimum surface Aa of the axis-parallel bounding box Ba surrounding the three segment point groups De (De1, De2, De3) after connection, is less than [β × γ]. On the other hand, when the first segment point group De1, the second segment point group De2, the third segment point group De3, and the fourth segment point group De4 are connected, the area α of the lower surface, which is the minimum surface Ab of the axis-parallel bounding box Bb surrounding the four segment point groups De (De1, De2, De3, De4) after connection, is greater than [β × γ]. Therefore, in the example shown in FIG. 13, it is determined that the fourth segment point group De4 does not satisfy the second connection condition either.

[0077] Also, the extending direction of the three segment point groups De (De1, De2, De3) after connection matches the extending direction of the predicted actual lead 110. On the other hand, the extending direction of the four segment point groups De (De1, De2, De3, De4) after connection does not match the extending direction of the predicted actual lead 110.

[0078] Thus, in the case of the example shown in FIG. 13, the candidate point group generation unit 15 predicts that the first segment point group De1, the second segment point group De2, and the third segment point group De3 correspond to the lead 110, and predicts that the fourth segment point group De4 does not correspond to the lead 110.

[0079] The candidate point group generation unit 15 extracts a plurality of segment point groups De (De1, De2, De3) predicted to correspond to the lead 110, connects those plurality of segment point groups De (De1, De2, De3), and generates candidate point group data Df that is a candidate for the lead 110 (step S6).

[0080] In the example shown in FIG. 13, the candidate point group data Df is composed of the first segment point group De1, the second segment point group De2, and the third segment point group De3.

[0081] The lead state calculation unit 16 performs principal component analysis on the candidate point group data Df generated in step S6 to calculate the three-dimensional data of the lead 110 (step S7).

[0082] In the embodiment, the lead state calculation unit 16 extracts, as the lead 110, a principal component that approximates the principal axis vector of the principal component of the candidate point group data Df and the coordinates of the tip of the lead 110 to the principal axis vector of the predicted lead 110 and the coordinates of the tip of the lead 110 based on the design value of the lead 110 and the viewpoint of the imaging device 7B when imaging the lead component 100. As shown in FIG. 13, the predicted principal axis vector of the lead 110 includes the above-described extending direction of the predicted actual lead 110.

[0083] The lead state calculation unit 16 calculates the amount of bending of the lead 110 and the coordinates of the tip of the lead 110 in a three-dimensional space defined by a local coordinate system as the three-dimensional data of the lead 110.

[0084] The output unit 17 outputs the three-dimensional data of the lead 110 calculated in step S7 to the control device 9 (step S8).

[0085] Based on the three-dimensional data of the lead 110, the control device 9 controls the robot manipulator 6 so that the lead 110 is inserted into the hole 210 of the substrate 200.

[0086] [Insertion of Lead] Each of FIGS. 14 and 15 is a diagram for explaining the operation of inserting the lead 110 of the lead component 100 according to the embodiment into the hole 210 of the substrate 200. In the example shown in FIGS. 14 and 15, the lead component 100 has two leads 110. The lead 110 includes a first lead 111 and a second lead 112. The hole 210 of the substrate 200 includes a first hole 211 into which the first lead 111 is inserted and a second hole 212 into which the second lead 112 is inserted.

[0087] In an embodiment, the robot manipulator 6 is an articulated robot. Therefore, the robot manipulator 6 can tilt the lead component 100 with respect to the upper surface of the substrate 200. The robot manipulator 6 can arbitrarily adjust the angle formed by the upper surface of the substrate 200 and the lower surface of the body 101 held by the robot hand 5.

[0088] As shown in FIG. 14, for example, when the second lead 112 is bent, the control device 9, based on the image processing result of the arithmetic device 7C, controls the robot manipulator 6 so that the second lead 112 is inserted into the second hole 212 before the first lead 111 is inserted into the first hole 211. The robot manipulator 6 inserts the second lead 112 into the second hole 212 while tilting the lead component 100.

[0089] After the second lead 112 is inserted into the second hole 212, the control device 9 moves the lead component 100 in the -X direction until the tip of the first lead 111 faces the first hole 211 of the substrate 200. Thereby, the second lead 112 is corrected. After the second lead 112 is corrected until the first lead 111 faces the first hole 211 of the substrate 200, as shown in FIG. 15, the control device 9 controls the robot manipulator 6 so that the first lead 111 is inserted into the first hole 211 with the second lead 112 disposed in the second hole 212.

[0090] [Computer System] FIG. 16 is a block diagram showing a computer system 1000 according to an embodiment. Each of the above-described arithmetic unit 7C and control unit 9 includes the computer system 1000. The computer system 1000 has a processor 1001 such as a CPU (Central Processing Unit), a main memory 1002 including a non-volatile memory such as a ROM (Read Only Memory) and a volatile memory such as a RAM (Random Access Memory), a storage 1003, and an interface 1004 including an input / output circuit. The functions of each of the arithmetic unit 7C and the control unit 9 are stored in the storage 1003 as a computer program. The processor 1001 reads the computer program from the storage 1003, expands it in the main memory 1002, and executes the above-described processing according to the computer program. Note that the computer program may be distributed to the computer system 1000 via a network.

[0091] According to the above-described embodiments, the computer program causes the computer system 1000 to irradiate the lead component 100 with stripe pattern light while holding the body 101 of the lead component 100 with the robot hand 5 provided at the tip of the robot manipulator 6, image the lead component 100 irradiated with the stripe pattern light from a predetermined viewpoint, perform image processing on the imaging data of the lead component 100, and control the robot manipulator 6 based on the image processing result so that the lead 110 of the lead component 100 is inserted into the hole 210 of the substrate 200. In the image processing, according to the above-described embodiments, the computer program causes the computer system 1000 to perform arithmetic processing on the imaging data of the lead component 100 based on the phase shift method to generate three-dimensional image data Da, convert the three-dimensional image data Da into three-dimensional point cloud data Db, segment the three-dimensional point cloud data Db, and extract lead point cloud data Dd indicating the three-dimensional shape of the surface of the lead 110 from the three-dimensional point cloud data Db, and perform principal component analysis on the lead point cloud data Dd to calculate the three-dimensional data of the lead 110.

[0092] [Effect] As described above, according to the embodiment, the component mounting apparatus 1 includes a robot manipulator 6, a robot hand 5, and a three-dimensional measuring apparatus 7. The three-dimensional measuring apparatus 7 can recognize the amount of bending of the lead 110 and the coordinates of the tip of the lead 110 in the three-dimensional space defined by the local coordinate system as the state of the lead 110. The robot manipulator 6 and the robot hand 5 can insert the lead 110 into the hole 210 of the substrate 200 according to the state of the lead 110. Therefore, a decrease in the productivity of the component mounting apparatus 1 is suppressed.

[0093] The arithmetic unit 7C of the three-dimensional measuring device 7 performs arithmetic processing on the imaging data of the lead component 100 based on the phase shift method to generate three-dimensional image data Da, a three-dimensional image generation unit 11, a three-dimensional point cloud conversion unit 12 that converts the three-dimensional image data Da into three-dimensional point cloud data Db, and a segmentation unit 14 that segments the three-dimensional point cloud data Db and extracts lead point cloud data Dd indicating the three-dimensional shape of the surface of the lead 110 from the three-dimensional point cloud data Db, and a lead state calculation unit 16 that performs principal component analysis on the lead point cloud data Dd to calculate three-dimensional data of the lead 110. By the segmentation by the segmentation unit 14, measurement points corresponding to the background or noise are removed from the three-dimensional point cloud data Db. By the principal component analysis by the lead state calculation unit 16, a principal axis vector representing the state of the lead 110 is calculated. Therefore, the three-dimensional measuring device 7 can appropriately recognize the state of the lead 110.

[0094] When the lead point cloud data Dd is separated into a plurality of segment point clouds De by segmentation, the candidate point cloud generation unit 15 generates candidate point cloud data Df that is a candidate for the lead 110 by connecting the plurality of segment point clouds De based on the design value of the lead component 100, the viewpoint of the imaging device 7B when the lead component 100 is imaged, and a specified connection condition. The three-dimensional measuring device 7 can appropriately recognize the state of the lead 110 based on the candidate point cloud data Df.

[0095] The three-dimensional image generation unit 11 generates a plurality of three-dimensional image data Da based on a plurality of imaging data captured from each of a plurality of viewpoints. The three-dimensional point cloud conversion unit 12 generates a plurality of three-dimensional point cloud data Db based on the plurality of three-dimensional image data Da. The three-dimensional point cloud integration unit 13 aligns the plurality of three-dimensional point cloud data based on a specified algorithm to generate integrated point cloud data Dc. Thereby, even if the positioning accuracy of the robot manipulator 6 is insufficient in the imaging of the lead component 100 by the imaging device 7B, the integrated point cloud data Dc is generated, so that the error of the three-dimensional point cloud data Db due to the positioning accuracy of the robot manipulator 6 is absorbed.

Explanation of Signs

[0096] 1... Component mounting device, 2... Base, 3... Component supply member, 4... Substrate support member, 5... Robot hand, 5A... Connecting member, 5B... Rotating member, 5C... Moving member, 5D... Gripping part, 6... Robot manipulator, 6A... Base member, 6B... Swiveling member, 6C... First arm, 6D... Second arm, 6E... Third arm, 7... Three-dimensional measuring device, 7A... Projection device, 7B... Imaging device, 7C... Computing device, 7D... Housing, 7E... Transparent member, 8... Force sensor, 9... Control device, 11... Three-dimensional image generation unit, 12... Three-dimensional point group conversion unit, 13... Three-dimensional point group integration unit, 14... Segmentation unit, 15... Candidate point group generation unit, 16... Lead state calculation unit, 17... Output unit, 100... Lead component, 101... Body, 110... Lead, 111... First lead, 112... Second lead, 200... Substrate, 210... Hole, 211... First hole, 212... Second hole, 1000... Computer system, 1001... Processor, 1002... Main memory, 1003... Storage, 1004... Interface, Aa... Minimum surface, Ab... Minimum surface, AX1... First rotation axis, AX2... Second rotation axis, AX3... Third rotation axis, Ba... Axis-parallel bounding box, Bb... Axis-parallel bounding box, Da... Three-dimensional image data, Db... Three-dimensional point group data, Db1... Three-dimensional point group data, Db2... Three-dimensional point group data, Db3... Three-dimensional point group data, Dc... Integrated point group data, Dd... Lead point group data, De... Segment point group, De1... First segment point group, De2... Second segment point group, De3... Third segment point group, De4... Fourth segment point group, Df... Candidate point group data, RX... Rotation axis, TX... Swivel axis.

Claims

1. A robot manipulator, a robot hand provided at the tip of the robot manipulator and holding the body of the lead component, a projection device that irradiates the lead component with stripe pattern light while the body is held by the robot hand, an imaging device that images the lead component irradiated with the stripe pattern light from a predetermined viewpoint, an arithmetic unit that performs image processing on the imaging data of the lead component imaged by the imaging device, a control device that controls the robot manipulator so that the lead of the lead component is inserted into the hole of the substrate based on the image processing result of the arithmetic unit, and the arithmetic unit includes a three-dimensional image generation unit that performs arithmetic processing on the imaging data of the lead component based on the phase shift method to generate three-dimensional image data, a three-dimensional point cloud conversion unit that converts the three-dimensional image data into three-dimensional point cloud data, a segmentation unit that segments the three-dimensional point cloud data and extracts lead point cloud data indicating the three-dimensional shape of the surface of the lead from the three-dimensional point cloud data, a lead state calculation unit that performs principal component analysis on the lead point cloud data to calculate three-dimensional data of the lead, a component mounting device.

2. By the segmentation, the lead point cloud data is separated into a plurality of segment point clouds, the arithmetic unit includes a candidate point cloud generation unit that generates candidate point cloud data that is a candidate for the lead by connecting the plurality of segment point clouds based on the design value of the lead component, the viewpoint of the imaging device when the lead component is imaged, and a specified connection condition, performing principal component analysis on the lead point cloud data includes performing principal component analysis on the candidate point cloud data, The component mounting device according to Claim 1.

3. The control device controls the robot manipulator so that the lead component is imaged by the imaging device from each of a plurality of viewpoints, the three-dimensional image generation unit generates a plurality of three-dimensional image data based on a plurality of imaging data imaged from each of a plurality of viewpoints, the three-dimensional point cloud conversion unit generates a plurality of three-dimensional point cloud data based on the plurality of three-dimensional image data, the arithmetic unit includes a three-dimensional point cloud integration unit that aligns the plurality of three-dimensional point cloud data based on a specified algorithm to generate integrated point cloud data Segmenting the three-dimensional point cloud data includes segmenting the integrated point cloud data. The component mounting device according to claim 1 or claim 2.

4. The lead state calculation unit calculates, as the three-dimensional data of the lead, the amount of bending of the lead in three-dimensional space and the coordinates of the tip of the lead. The component mounting device according to any one of claims 1 to 3.

5. Irradiating a striped pattern light on the lead component while holding the body of the lead component with a robot hand provided at the tip of a robot manipulator; Imaging the lead component irradiated with the striped pattern light from a predetermined viewpoint; Performing image processing on the imaging data of the lead component; Controlling the robot manipulator so that the lead of the lead component is inserted into the hole of the substrate based on the image processing result, including: Performing the image processing includes: Performing arithmetic processing on the imaging data of the lead component based on the phase shift method to generate three-dimensional image data; Converting the three-dimensional image data into three-dimensional point cloud data; Segmenting the three-dimensional point cloud data and extracting lead point cloud data indicating the three-dimensional shape of the surface of the lead from the three-dimensional point cloud data; Performing principal component analysis on the lead point cloud data to calculate the three-dimensional data of the lead, including: Component mounting method.

Images