3D measuring device, component mounting device, and 3D measuring method

Abstract

To speed up the process of measuring the three-dimensional shape of components.SOLUTION: A three dimensional measurement device includes: a model point cloud data generation unit that generates model point cloud data indicating three dimensional point cloud data of a model of a component mounted on a board; a detected point cloud data generation unit that generates detected point cloud data indicating three dimensional point cloud data of the component based on multiple detection data of the component detected from each of multiple viewpoints through three dimensional sensors; an offset data calculation unit that calculates offset data for aligning detected point cloud data with model point cloud data for each of multiple detected point cloud data; and an integrated point cloud data generation unit that generates integrated point cloud data for the component by integrating multiple detected point cloud data offset based on the offset data.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The technology disclosed in this specification relates to a three-dimensional measuring device, a component mounting device, and a three-dimensional measuring method.

Background Art

[0002] In the technical field related to component mounting devices, an electronic component mounting device equipped with a three-dimensional measuring device as disclosed in Patent Document 1 is known.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In order to efficiently produce electronic devices using a component mounting device, it is desired to speed up the measurement process of the three-dimensional shape of components.

Means for Solving the Problems

[0005] This specification discloses a three-dimensional measuring device. The three-dimensional measuring device includes a model point cloud data generation unit that generates model point cloud data indicating three-dimensional point cloud data of a model of a component mounted on a substrate, a detection point cloud data generation unit that generates detection point cloud data indicating three-dimensional point cloud data of a component based on a plurality of detection data of the component detected from each of a plurality of viewpoints by a three-dimensional sensor, an offset data calculation unit that calculates offset data for aligning the detection point cloud data with the model point cloud data for each of the plurality of detection point cloud data, and an integrated point cloud data generation unit that integrates the plurality of detection point cloud data offset based on the offset data to generate integrated point cloud data of the component.

Effects of the Invention

[0006] The technology disclosed herein enables faster measurement of the three-dimensional shape of a part. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 is a perspective view showing a component mounting apparatus according to an embodiment. [Figure 2] Figure 2 is a side view showing a component mounting apparatus according to an embodiment. [Figure 3] Figure 3 is a perspective view showing a robot hand according to this embodiment. [Figure 4] Figure 4 is a side view showing a component held by a robot hand according to this embodiment. [Figure 5] Figure 5 is a view of the component according to the embodiment, seen from below. [Figure 6] Figure 6 is a perspective view showing a three-dimensional measuring device according to an embodiment of this device. [Figure 7] Figure 7 is a diagram illustrating the operation of the robot manipulator according to the embodiment. [Figure 8] Figure 8 is a block diagram of a component mounting apparatus according to an embodiment. [Figure 9] Figure 9 is a flowchart showing the image processing method of the computing device according to the embodiment. [Figure 10] Figure 10 is a schematic diagram showing an example of model point cloud data according to the embodiment. [Figure 11] Figure 11 is a schematic diagram showing an example of detected point cloud data according to the embodiment. [Figure 12] Figure 12 is a diagram illustrating the offset data according to the embodiment. [Figure 13] Figure 13 is a schematic diagram showing an example of integrated point cloud data according to the embodiment. [Figure 14] Figure 14 is a diagram illustrating the operation of inserting the leads of a leaded component according to the embodiment into holes in a substrate. [Figure 15]FIG. 15 is a diagram for explaining an operation of inserting a lead of a lead component according to an embodiment into a hole of a substrate. [Figure 16] FIG. 16 is a block diagram showing a computer system according to an embodiment. MODE FOR CARRYING OUT THE INVENTION

[0008] Hereinafter, embodiments according to the present disclosure will be described with reference to the drawings, but the present disclosure is not limited to the embodiments. The components of the embodiments described below can be combined as appropriate. Also, some components may not be used.

[0009] In an embodiment, a local coordinate system is set in the component mounting apparatus 1, and the positional relationship of each part will be described while referring to the local coordinate system. As the local coordinate system, an XYZ orthogonal coordinate system is set. A direction parallel to the X-axis in a predetermined plane is defined as the X-axis direction. A direction parallel to the Y-axis orthogonal to the X-axis in the predetermined plane is defined as the Y-axis direction. A direction parallel to the Z-axis orthogonal to each of the X-axis and the Y-axis is defined as the Z-axis direction. A rotational direction or an inclination direction centered on the X-axis is defined as the θX direction. A rotational direction or an inclination direction centered on the Y-axis is defined as the θY direction. A rotational direction or an inclination direction centered on the Z-axis is defined as the θZ direction. The predetermined plane is the XY plane. The Z-axis is orthogonal to the predetermined plane. In the embodiment, the predetermined plane is assumed to be parallel to the horizontal plane. The Z-axis direction is the vertical direction. Note that the predetermined plane may be inclined with respect to the horizontal plane.

[0010] [Component Mounting Apparatus] FIG. 1 is a perspective view showing a component mounting apparatus 1 according to an embodiment. FIG. 2 is a side view showing the component mounting apparatus 1 according to an embodiment. The component mounting apparatus 1 mounts a component 100 on a substrate 200.

[0011] As shown in FIGS. 1 and 2, the component mounting apparatus 1 includes a base 2, a component supply member 3, a substrate support member 4, a robot hand 5, a robot manipulator 6, and a three-dimensional measuring device 7.

[0012] The base 2 supports each of the component supply member 3, the substrate support member 4, the robot manipulator 6, and the three-dimensional measuring device 7.

[0013] The component supply member 3 supplies the components 100. In an embodiment, the component supply member 3 includes a tray on which the components 100 are arranged. A plurality of components 100 are arranged on the component supply member 3. The types of the plurality of components 100 may be the same or different.

[0014] The substrate support member 4 supports the substrate 200 on which the components 100 are mounted. The substrate support member 4 supports the substrate 200 such that the upper surface of the substrate 200 is parallel to the XY plane.

[0015] The robot hand 5 holds the component 100. The robot hand 5 is provided at the tip of the robot manipulator 6.

[0016] The robot manipulator 6 moves the robot hand 5. The robot manipulator 6 holds the component 100 via the robot hand 5. The robot manipulator 6 is driven while holding the component 100 to be mounted on the substrate 200. The robot manipulator 6 includes an articulated robot. In an embodiment, the robot manipulator 6 is a vertically articulated robot. Note that the robot manipulator 6 may be a horizontally articulated robot. The robot manipulator 6 has a base member 6A fixed to the base 2, a swivel member 6B supported by the base member 6A, a first arm 6C connected to the swivel member 6B, a second arm 6D connected to the first arm 6C, and a third arm 6E connected to the second arm 6D.

[0017] The swivel member 6B is supported by the base member 6A so as to be able to rotatably around the swivel axis TX. The swivel axis TX is parallel to the Z axis. The first arm 6C is connected to the swivel member 6B so as to be able to rotatably around the first rotation axis AX1. The first rotation axis AX1 is perpendicular to the Z axis. The second arm 6D is connected to the first arm 6C so as to be able to rotatably around the second rotation axis AX2. The second rotation axis AX2 is parallel to the first rotation axis AX1. The third arm 6E is connected to the second arm 6D so as to be able to rotatably around the third rotation axis AX3. The third rotation axis AX3 is parallel to the second rotation axis AX2. The robot hand 5 is attached to the third arm 6E.

[0018] The robot manipulator 6 includes a swivel actuator for swiveling a swivel member 6B, a first rotation actuator for rotating the first arm 6C, a second rotation actuator for rotating the second arm 6D, and a third rotation actuator for rotating the third arm 6E.

[0019] The 3D measuring device 7 measures the part 100 held by the robot manipulator 6 via the robot hand 5. The 3D measuring device 7 measures the three-dimensional shape of the part 100 based on the phase shift method. The 3D measuring device 7 also detects the position and angle of the part 100 in the local coordinate system based on the phase shift method.

[0020] [Robot Hand] Figure 3 is a perspective view showing a robot hand 5 according to an embodiment. The robot hand 5 includes a connecting member 5A attached to a third arm 6E, a rotating member 5B supported by the connecting member 5A, and a pair of movable members 5C supported by the rotating member 5B.

[0021] The rotating member 5B is supported by the connecting member 5A so as to be rotatable about the rotation axis RX. The rotation axis RX is perpendicular to the third rotation axis AX3. The pair of movable members 5C move in directions toward and toward each other. A grip portion 5D is provided at the lower end of the movable member 5C. The pair of grip portions 5D move toward and toward each other.

[0022] The robot hand 5 includes a rotary actuator that rotates a rotating member 5B and a grip actuator that moves a pair of movable members 5C closer together or further apart from each other.

[0023] With the part 100 positioned between a pair of grip portions 5D, the part 100 is held by the grip portions 5D as the pair of grip portions 5D move closer to each other. The part 100 is released from the grip portions 5D as the pair of grip portions 5D move further apart from each other.

[0024] A force sensor 8 is positioned on one of the moving members 5C. The force sensor 8 can detect the load applied to the grip portion 5D.

[0025] [parts] Figure 4 is a side view showing a component 100 held by a robot hand 5 according to the embodiment. Figure 5 is a view of the component 100 according to the embodiment from below.

[0026] The component 100 has a body portion 101 and a plurality of lead portions 110 that protrude from the body portion 101.

[0027] The body portion 101 includes a housing made of synthetic resin. An element, such as a coil, is arranged in the internal space of the body portion 101. The lead portion 110 is a metal projection that protrudes from the body portion 101. The lead portion 110 is connected to an element, such as one, arranged in the internal space of the body portion 101.

[0028] The lead portion 110 protrudes downward from the lower surface of the body portion 101. With the component 100 mounted on the circuit board 200, the lower surface of the body portion 101 and the upper surface of the circuit board 200 face each other.

[0029] The robot hand 5 holds the body portion 101 of the part 100. The pair of grip portions 5D hold the part 100 by sandwiching the body portion 101 between them.

[0030] [3D measuring device] Figure 6 is a perspective view showing a three-dimensional measuring device 7 according to an embodiment. As shown in Figure 6, the three-dimensional measuring device 7 measures the three-dimensional shape of the part 100 while the body portion 101 is held by the robot hand 5.

[0031] The 3D measuring device 7 includes a projection device 7A, an imaging device 7B, and a computing device 7C. In this embodiment, the projection device 7A and the imaging device 7B are each housed in a housing 7D. The projection device 7A and the imaging device 7B are each fixed to the housing 7D. A transparent member 7E is placed in the opening at the upper end of the housing 7D. A glass plate is exemplified as the transparent member 7E.

[0032] The projection device 7A irradiates the part 100 with striped pattern light while the body 101 is held by the robot hand 5. The projection device 7A includes a light source, an optical modulation element that modulates the light emitted from the light source to generate striped pattern light, and an emission optical system that emits the striped pattern light generated by the optical modulation element. Examples of the optical modulation element include a digital mirror device (DMD), a transmissive liquid crystal panel, or a reflective liquid crystal panel.

[0033] The imaging device 7B images the component 100 illuminated by the striped pattern light from a predetermined viewpoint. The viewpoint of the imaging device 7B refers to the relative imaging position and imaging angle of the imaging device 7B with respect to the component 100. The imaging device 7B includes an imaging optical system that forms an image of the striped pattern light reflected by the component 100, and an image sensor that acquires image data of the component 100 via the imaging optical system. Examples of image sensors include a CMOS image sensor (Complementary Metal Oxide Semiconductor Image Sensor) or a CCD image sensor (Charge Coupled Device Image Sensor).

[0034] In this embodiment, the imaging device 7B functions as a three-dimensional sensor for detecting the component 100. The imaging data of the component 100 captured by the imaging device 7B functions as detection data of the component 100 detected by the three-dimensional sensor.

[0035] The computing unit 7C performs measurement processing of the three-dimensional shape of the part 100 based on the imaging data of the part 100 captured by the imaging device 7B. The computing unit 7C performs measurement processing of the three-dimensional shape of the part 100 based on the phase shift method. The measurement processing of the computing unit 7C includes image processing of the imaging data of the part 100 captured by the imaging device 7B. The computing unit 7C includes a computer system. The computing unit 7C has a processor such as a CPU (Central Processing Unit), memory such as ROM (Read Only Memory) or RAM (Random Access Memory), and an input / output interface including input / output circuits capable of inputting and outputting signals and data.

[0036] The projection device 7A illuminates the part 100 with striped pattern light, for example, a sinusoidal brightness distribution, while shifting its phase. The part 100, held by the robot hand 5, is positioned above the transparent member 7E. The striped pattern light emitted from the projection device 7A illuminates the part 100 through the transparent member 7E.

[0037] The imaging device 7B images the part 100 illuminated by the striped pattern light. The imaging device 7B images the part 100 through the transparent member 7E. The imaging device 7B images the part 100 from below. The viewpoint of the imaging device 7B is defined below the part 100. As the robot manipulator 6 operates and the position and angle of the part 100 held by the robot hand 5 change, the relative position and relative angle between the part 100 and the imaging device 7B change. As the relative position and relative angle between the part 100 and the imaging device 7B change, the relative viewpoint of the imaging device 7B with respect to the part 100 changes.

[0038] The computing unit 7C processes the image data of the part 100 captured by the imaging device 7B based on the phase shift method to calculate the three-dimensional data of the lead portion 110. The three-dimensional data of the lead portion 110 includes the amount of curvature of the lead portion 110 and the coordinates of the tip of the lead portion 110 in a three-dimensional space defined by the local coordinate system. If the design angle of the lead portion 110 relative to the body portion 101 is defined as the ideal angle and the actual angle of the lead portion 110 relative to the body portion 101 is defined as the actual angle, the amount of curvature of the lead portion 110 refers to the difference between the ideal angle and the actual angle.

[0039] [Robot manipulator operation] Figure 7 is a diagram illustrating the operation of a robot manipulator 6 according to an embodiment. The component mounting apparatus 1 includes a control device 9 that controls the robot manipulator 6. The control device 9 includes a computer system. As shown in Figure 7, the substrate 200 is provided with holes 210 into which the lead portions 110 of the component 100 are inserted. The control device 9 controls the robot manipulator 6 so that the lead portions 110 of the component 100 are inserted into the holes 210 of the substrate 200, based on the image processing results of the computing device 7C of the 3D measuring apparatus 7.

[0040] [Arithmetic device] Figure 8 is a block diagram showing a component mounting apparatus 1 according to an embodiment. As shown in Figure 8, the component mounting apparatus 1 has a calculation unit 7C and a control device 9.

[0041] The computing unit 7C includes a model data storage unit 11, a model point cloud data generation unit 12, a detected point cloud data generation unit 13, an offset data calculation unit 14, an integrated point cloud data generation unit 15, and an output unit 16.

[0042] The model data storage unit 11 stores model data representing the model of part 100. The model data of part 100 includes design data for part 100. The design data for part 100 includes 3D CAD (Computer Aided Design) data.

[0043] The model data for part 100 is known data. The model data for part 100 is pre-stored in the model data storage unit 11. The model data for part 100 includes the length of the lead portion 110, the relative position of the body portion 101 and the lead portion 110, the angle of the lead portion 110 relative to the body portion 101 (ideal angle), and the spacing between the multiple lead portions 110.

[0044] Note that the model data for part 100 does not have to be design data. If part 100 with an ideal shape (the shape according to the design values) and ideal dimensions (the dimensions according to the design values) is measured by a 3D measuring device 7, the 3D data of part 100 with the ideal shape and ideal dimensions may be used as the model data for part 100.

[0045] The model point cloud data generation unit 12 generates model point cloud data Da, which represents the three-dimensional point cloud data of the part 100 model, based on the model data of the part 100 stored in the model data storage unit 11. The model point cloud data Da represents the three-dimensional shape of the surface of the part 100 model. The model point cloud data Da is a collection of multiple measurement points on the surface of the part 100 model measured by the three-dimensional measuring device 7. The position of each of the multiple measurement points is defined by the X coordinate, Y coordinate, and Z coordinate.

[0046] The detected point cloud data generation unit 13 generates detected point cloud data Db, which represents the three-dimensional point cloud data of the part 100, based on multiple imaging data of the part 100 captured from each of multiple viewpoints by the imaging device 7B.

[0047] In this embodiment, the detected point cloud data generation unit 13 acquires imaging data of the part 100 from the imaging device 7B. The detected point cloud data generation unit 13 processes the acquired imaging data of the part 100 based on the phase shift method to generate three-dimensional image data of the part 100. The detected point cloud data generation unit 13 converts the generated three-dimensional image data into detected point cloud data Db, which is three-dimensional point cloud data. The detected point cloud data Db represents the three-dimensional shape of the surface of the part 100. The detected point cloud data Db is a collection of multiple measurement points on the surface of the part 100 by the three-dimensional measurement device 7. The position of each of the multiple measurement points is defined by the X coordinate, Y coordinate, and Z coordinate.

[0048] In this embodiment, the control device 9 controls the robot manipulator 6 so that the part 100 is imaged by the imaging device 7B from each of the multiple viewpoints. The detection point cloud data generation unit 13 generates multiple detection point cloud data Db of the part 100 as seen from each of the multiple viewpoints, based on the multiple imaging data captured from each of the multiple viewpoints.

[0049] The offset data calculation unit 14 calculates offset data Df for each of the multiple detected point cloud data Db for aligning the detected point cloud data Db with the model point cloud data Da, based on a predetermined algorithm. The offset data Df includes the amount of movement Δx in the X-axis direction of the detected point cloud data Db, the amount of movement Δy in the Y-axis direction of the detected point cloud data Db, the amount of movement Δz in the Z-axis direction of the detected point cloud data Db, the amount of rotation Δθx in the θX direction of the detected point cloud data Db, the amount of rotation Δθy in the θY direction of the detected point cloud data Db, and the amount of rotation Δθz in the θZ direction of the detected point cloud data Db, for aligning the detected point cloud data Db with the model point cloud data Da.

[0050] As a standard algorithm, the ICP (Iterative Closest Point) matching algorithm is given as an example. The offset data calculation unit 14 calculates offset data Df for each of the multiple detected point cloud data Db for aligning the detected point cloud data Db with the model point cloud data Da, based on an existing algorithm such as the ICP matching algorithm.

[0051] The integrated point cloud data generation unit 15 integrates multiple detected point cloud data Db offset based on offset data Df to generate integrated point cloud data Dc of part 100. The integrated point cloud data Dc is the 3D point cloud data of part 100. The integrated point cloud data generation unit 15 aligns each of the multiple detected point cloud data Db with the model point cloud data Da in 3D space so that each of the multiple detected point cloud data Db matches the model point cloud data Da, and generates integrated point cloud data Dc.

[0052] The output unit 16 outputs the integrated point cloud data Dc of the component 100 generated by the integrated point cloud data generation unit 15 to the control device 9. The measurement processing result of the arithmetic unit 7C includes the integrated point cloud data Dc of the lead portion 110 calculated by the integrated point cloud data generation unit 15. The integrated point cloud data Dc of the lead portion 110 includes the amount of curvature of the lead portion 110 and the coordinates of the tip of the lead portion 110. Based on the integrated point cloud data Dc of the lead portion 110, the control device 9 controls the robot manipulator 6 so that the lead portion 110 is inserted into the hole 210 of the substrate 200.

[0053] [Image processing method] Figure 9 is a flowchart illustrating the image processing method of the computing device 7C according to the embodiment. In order to mount the component 100 onto the substrate 200, the control device 9 controls the robot manipulator 6 so that the robot hand 5 approaches the component supply member 3. The robot hand 5, having moved to the component supply member 3, holds the body portion 101 of the component 100 that is placed on the component supply member 3. After the body portion 101 of the component 100 is held by the robot hand 5, the control device 9 controls the robot manipulator 6 so that the component 100 held by the robot hand 5 is measured by the 3D measuring device 7. That is, as described with reference to Figure 6, the control device 9 controls the robot manipulator 6 so that the component 100 held by the robot hand 5 is positioned above the 3D measuring device 7.

[0054] In this embodiment, the control device 9 controls the robot manipulator 6 so that the part 100 is imaged by the imaging device 7B from each of a plurality of viewpoints. That is, the control device 9 controls the robot manipulator 6 so that the position and angle of the part 100 held by the robot hand 5 changes above the 3D measuring device 7. The imaging device 7B images the part 100 from each of a plurality of different viewpoints.

[0055] The model point cloud data generation unit 12 generates model point cloud data Da based on the model data of the component 100 stored in the model data storage unit 11 (step S1).

[0056] Figure 10 is a schematic diagram showing an example of model point cloud data Da according to an embodiment. As shown in Figure 10, in the model point cloud data Da, measurement points are provided on the entire surface of the body portion 101 and the entire surface of the lead portion 110. There are no missing measurement points in the model point cloud data Da. In the model point cloud data Da, the measurement points are ideally arranged.

[0057] The detection point cloud data generation unit 13 acquires multiple imaging data of the part 100 captured from each of the multiple viewpoints by the imaging device 7B. Based on the multiple imaging data of the part 100 captured from each of the multiple viewpoints, the detection point cloud data generation unit 13 generates detection point cloud data Db of the part 100 as seen from each of the multiple viewpoints (step S2).

[0058] Figure 11 is a schematic diagram showing an example of detected point cloud data Db according to the embodiment. When part 100 is imaged from multiple viewpoints, the detected point cloud data generation unit 13 generates multiple detected point cloud data Db for part 100 as seen from each of the multiple viewpoints. For example, when part 100 is imaged from three viewpoints, as shown in Figure 11, the detected point cloud data generation unit 13 generates detected point cloud data Db1 for part 100 as seen from the first viewpoint, detected point cloud data Db2 for part 100 as seen from the second viewpoint, and detected point cloud data Db3 for part 100 as seen from the third viewpoint.

[0059] Depending on the relative imaging position and angle of the imaging device 7B with respect to the part 100, the imaging device 7B may not be able to image the entire part 100. In other words, depending on the viewpoint of the imaging device 7B, blind spots may occur on the part 100, and there may be areas of the part 100 that cannot be imaged by the imaging device 7B. As shown in Figure 11, in the detected point cloud data Db, measurement points may not be provided on at least a portion of the surface of the body part 101. Also, in the detected point cloud data Db, measurement points may not be provided on at least a portion of the surface of the lead part 110. In other words, there may be gaps in the measurement points in the detected point cloud data Db.

[0060] Note that the detected point cloud data Db shown in Figure 11 is just one example. Two detected point cloud data Db may be generated from each of two viewpoints, or multiple detected point cloud data Db may be generated from each of four or more arbitrary viewpoints.

[0061] The offset data calculation unit 14 calculates offset data Df for each of the multiple detected point cloud data Db to align the detected point cloud data Db with the model point cloud data Da, based on a predetermined algorithm (step S3).

[0062] Figure 12 is a diagram illustrating the offset data Df according to the embodiment. As shown in Figure 12, the offset data Df includes the X-axis movement Δx of the detected point cloud data Db, the Y-axis movement Δy of the detected point cloud data Db, the Z-axis movement Δz of the detected point cloud data Db, the θX-axis rotation Δθx of the detected point cloud data Db, the θY-axis rotation Δθy of the detected point cloud data Db, and the θZ-axis rotation Δθz of the detected point cloud data Db, for aligning the detected point cloud data Db with the model point cloud data Da. The position and angle of the model point cloud data Da in the local coordinate system are predetermined as a reference position and reference angle. The position and angle of the detected point cloud data Db in the local coordinate system can be calculated, for example, based on the drive amount of the robot manipulator 6. Therefore, the offset data calculation unit 14 can calculate offset data Df for aligning the detected point cloud data Db with the model point cloud data Da, based on the position and angle of the model point cloud data Da in the local coordinate system and the position and angle of the detected point cloud data Db in the local coordinate system.

[0063] The offset data calculation unit 14 calculates offset data Df based on the feature portion of the part 100. Examples of feature portions of the part 100 include the corners of the body portion 101, or the boundary between the body portion 101 and the lead portion 110. The offset data calculation unit 14 calculates offset data Df so that the feature portion of the model point cloud data Da and the feature portion of the detected point cloud data Db are aligned.

[0064] Figure 12 shows the offset data Df used to align the detected point cloud data Db1 of part 100 as seen from the first viewpoint with the model point cloud data Da. The offset data calculation unit 14 generates offset data Df for each of the multiple detected point cloud data Db generated in step S2.

[0065] The integrated point cloud data generation unit 15 integrates multiple detected point cloud data Db that have been offset based on the offset data Df to generate integrated point cloud data Dc for part 100. That is, the integrated point cloud data generation unit 15 offsets each of the multiple detected point cloud data Db generated in step S2 based on the offset data Df calculated in step S3 to integrate the multiple detected point cloud data Db (step S4).

[0066] Figure 13 schematically shows an example of integrated point cloud data Dc according to the embodiment. Integrated point cloud data Dc is generated by integrating multiple detected point cloud data Db based on an existing algorithm such as the ICP matching algorithm. As explained with reference to Figure 11, even if there are missing measurement points in the detected point cloud data Db, the missing measurement points are mutually compensated for by the multiple detected point cloud data Db when multiple detected point cloud data Db are integrated. As a result, as shown in Figure 13, in the integrated point cloud data Dc, measurement points are provided on the entire surface of the body portion 101 and the entire surface of the lead portion 110. In the integrated point cloud data Dc, there are virtually no missing measurement points.

[0067] Note that the integrated point cloud data Dc shown in Figure 13 illustrates an example where one of the lead portions 110 of part 100 is slightly bent.

[0068] The output unit 16 outputs the integrated point cloud data Dc generated in step S4 to the control device 9. The integrated point cloud data Dc corresponds to the measurement processing result of the 3D measuring device 7 (step S5).

[0069] The control device 9 controls the robot manipulator 6 so that the component 100 is mounted on the substrate 200, based on the integrated point cloud data Dc output from the output unit 16. The integrated point cloud data Dc includes the amount of bending of the lead portion 110 and the coordinates of the tip of the lead portion 110. Based on the integrated point cloud data Dc, the control device 9 controls the robot manipulator 6 so that the lead portion 110 is inserted into the hole 210 in the substrate 200.

[0070] [Inserting the lead] Figures 14 and 15 are diagrams illustrating the operation of inserting the lead portion 110 of a component 100 according to an embodiment into a hole 210 of a substrate 200. In the example shown in Figures 14 and 15, the component 100 has two lead portions 110. The lead portion 110 includes a first lead portion 111 and a second lead portion 112. The hole 210 of the substrate 200 includes a first hole 211 into which the first lead portion 111 is inserted and a second hole 212 into which the second lead portion 112 is inserted.

[0071] In this embodiment, the robot manipulator 6 is a multi-joint robot. Therefore, the robot manipulator 6 can tilt the part 100 relative to the upper surface of the substrate 200. The robot manipulator 6 can arbitrarily adjust the angle between the upper surface of the substrate 200 and the lower surface of the body portion 101 held by the robot hand 5.

[0072] As shown in Figure 14, for example, if the second lead portion 112 is bent, the control device 9 controls the robot manipulator 6 based on the image processing result of the computing device 7C so that the second lead portion 112 is inserted into the second hole 212 before the first lead portion 111 is inserted into the first hole 211. The robot manipulator 6 inserts the second lead portion 112 into the second hole 212 while tilting the part 100.

[0073] After the second lead portion 112 is inserted into the second hole 212, the control device 9 moves the component 100 in the -X direction until the tip of the first lead portion 111 faces the first hole 211 of the substrate 200. This straightens the second lead portion 112. After the second lead portion 112 has been straightened until the first lead portion 111 faces the first hole 211 of the substrate 200, as shown in Figure 15, the control device 9 controls the robot manipulator 6 so that the first lead portion 111 is inserted into the first hole 211 with the second lead portion 112 positioned in the second hole 212.

[0074] [Computer System] Figure 16 is a block diagram showing a computer system 1000 according to an embodiment. The arithmetic unit 7C and the control unit 9 described above each include the computer system 1000. The computer system 1000 has a processor 1001 such as a CPU (Central Processing Unit), a main memory 1002 including non-volatile memory such as ROM (Read Only Memory) and volatile memory such as RAM (Random Access Memory), a storage 1003, and an interface 1004 including input / output circuits. The respective functions of the arithmetic unit 7C and the control unit 9 are stored in the storage 1003 as computer programs. The processor 1001 reads the computer program from the storage 1003, loads it into the main memory 1002, and executes the above-described processing according to the computer program. The computer program may be distributed to the computer system 1000 via a network.

[0075] The computer program can cause the computer system 1000 to perform the following actions according to the above-described embodiment: generate model point cloud data Da showing the 3D point cloud data of a model of a component 100 mounted on a circuit board 200; generate detected point cloud data Db showing the 3D point cloud data of a component 100 based on multiple detection data (image data) of the component 100 detected (imaged) from each of multiple viewpoints by the imaging device 7B, which is a 3D sensor; calculate offset data Df for each of the multiple detected point cloud data Db to align the detected point cloud data Db with the model point cloud data Da; and integrate the multiple detected point cloud data Db offset based on the offset data Df to generate integrated point cloud data Dc of the component 100.

[0076] [effect] As described above, the 3D measuring device 7 includes a model point cloud data generation unit 12 that generates model point cloud data Da showing the 3D point cloud data of the model of the component 100 mounted on the substrate 200; a detection point cloud data generation unit 13 that generates detection point cloud data Db showing the 3D point cloud data of the component 100 based on multiple detection data (image data) of the component 100 detected (imaged) from each of multiple viewpoints by the imaging device 7B, which is a 3D sensor; an offset data calculation unit 14 that calculates offset data Df for each of the multiple detection point cloud data Db to align the detection point cloud data Db with the model point cloud data Da; and an integrated point cloud data generation unit that integrates the multiple detection point cloud data Db offset based on the offset data Df to generate integrated point cloud data Dc of the component 100.

[0077] According to the embodiment, when integrating multiple detected point cloud data Db, model point cloud data Da is referenced. Each of the multiple detected point cloud data Db is integrated based on the model point cloud data Da. If there is no model point cloud data Da, a common overlapping portion is required in each of the multiple detected point cloud data Db for integration. Therefore, it is necessary to acquire a large number of detected point cloud data Db. To acquire a large number of detected point cloud data Db, the part 100 must be imaged from many different viewpoints. In other words, the number of images taken by the imaging device 7B must be increased. As a result, the measurement process of the 3D shape of the part 100 takes time. If the number of detected point cloud data Db is reduced in order to shorten the time required for measurement processing, the common overlapping portion will decrease, making it difficult to integrate multiple detected point cloud data Db.

[0078] In this embodiment, each of the multiple detected point cloud data Db is integrated based on the model point cloud data Da. Therefore, even if the number of detected point cloud data Db is small, it is possible to integrate the detected point cloud data Db to generate integrated point cloud data Dc. Consequently, the measurement process of the 3D shape of part 100 is accelerated.

[0079] The 3D measuring device 7 includes a model data storage unit 11 that stores model data representing the model of the part 100. The model point cloud data generation unit 12 generates model point cloud data Da based on the model data. Since the model data is predetermined, the model point cloud data generation unit 12 can properly generate the model point cloud data Da.

[0080] Component 100 has a body portion 101 and lead portions 110 which are multiple protruding portions that extend from the body portion 101. The integrated point cloud data Dc includes the amount of curvature of the lead portions 110 and the coordinates of the tips of the lead portions 110. Based on the integrated point cloud data Dc, the control device 9 can insert the lead portions 110 of component 100 into the holes 210 of the substrate 200.

[0081] [Other embodiments] In the above embodiment, the 3D sensor is defined as an imaging device 7B that captures a part 100 onto which a striped pattern light is projected by a projection device 7A. The 3D sensor only needs to be able to detect the 3D shape of the part 100. The 3D sensor may be, for example, a laser scanner or a stereo camera.

[0082] In the above embodiment, the protruding portion of the component 100 is a metal lead portion 110. The protruding portion may be made of synthetic resin. For example, a boss portion made of synthetic resin may protrude from the body portion 101 as the protruding portion, and the boss portion may be inserted into the hole 210 of the substrate 200. [Explanation of Symbols]

[0083] 1...Component mounting device, 2...Base, 3...Component supply member, 4...Substrate support member, 5...Robot hand, 5A...Connecting member, 5B...Rotating member, 5C...Moving member, 5D...Grip section, 6...Robot manipulator, 6A...Base member, 6B...Swivel member, 6C...First arm, 6D...Second arm, 6E...Third arm, 7...3D measuring device, 7A...Projection device, 7B...Imaging device, 7C...Calculation unit, 7D...Housing, 7E...Transparent member, 8...Force sensor, 9...Control device, 11...Model data storage unit, 12...Model point cloud data generation unit, 13...Detected point cloud data generation unit, 14...Offset data calculation unit Output section, 15...Integrated point cloud data generation section, 16...Output section, 100...Parts, 101...Body section, 110...Lead section, 111...First lead section, 112...Second lead section, 200...Substrate, 210...Hole, 211...First hole, 212...Second hole, 1000...Computer system, 1001...Processor, 1002...Main memory, 1003...Storage, 1004...Interface, AX1...First rotation axis, AX2...Second rotation axis, AX3...Third rotation axis, Da...Model point cloud data, Db...Detected point cloud data, Dc...Integrated point cloud data, Df...Offset data, RX...Rotation axis, TX...Swivel axis.

Claims

1. A model point cloud data generation unit generates model point cloud data that shows the 3D point cloud data of the components to be mounted on the circuit board, A detection point cloud data generation unit generates detection point cloud data showing the 3D point cloud data of the part based on multiple detection data of the part detected from each of multiple viewpoints by a 3D sensor, An offset data calculation unit calculates offset data for each of the multiple detected point cloud data to align the detected point cloud data with the model point cloud data, The system includes an integrated point cloud data generation unit that integrates a plurality of detected point cloud data offset based on the offset data to generate integrated point cloud data of the part. Three-dimensional measuring device.

2. It includes a model data storage unit that stores model data indicating the model of the aforementioned part, The model point cloud data generation unit generates the model point cloud data based on the model data. The three-dimensional measuring device according to claim 1.

3. The aforementioned part has a body portion and a plurality of protrusions that protrude from the body portion. The integrated point cloud data includes the amount of curvature of the protrusion and the coordinates of the tip of the protrusion. The three-dimensional measuring device according to claim 1.

4. A robotic manipulator that operates while holding components mounted on a circuit board, A three-dimensional measuring device according to claim 1 for measuring the three-dimensional shape of the aforementioned part, The system includes a control device that controls the robot manipulator so that the component is mounted on the substrate based on the measurement processing results of the three-dimensional measuring device. Component mounting equipment.

5. To generate model point cloud data that shows the 3D point cloud data of the components mounted on the circuit board, Based on multiple detection data of the part detected from each of multiple viewpoints by a 3D sensor, detection point cloud data representing the 3D point cloud data of the part is generated. The process involves calculating offset data for each of the multiple detected point cloud data sets to align the detected point cloud data with the model point cloud data, This includes integrating a plurality of the detected point cloud data offset based on the offset data to generate integrated point cloud data of the part, Three-dimensional measurement methods.

Images