Soldering device and soldering system, and processing device

Abstract

To provide a soldering device for emitting processing light melting a solder arranged in a circuit board.SOLUTION: A soldering device includes: a light irradiation device which has a galvanometer mirror and emits processing light through the galvanometer mirror; a detection device for detecting light from a circuit board, and creating at least one data of image data and shape data; a robot arm which is provided with the light irradiation device and the detection device, and has a driving part for moving the light irradiation device and the detection device; and a control device for controlling the direction of the galvanometer mirror so that the processing light from the light irradiation device displaced together with the detection device is emitted to the same position, on the basis of at least the one data changing with the displacement of the detection device.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to the technical field of a soldering apparatus and a soldering system that perform soldering by irradiating processing light, and a processing apparatus that processes an object by irradiating processing light. [Background technology]

[0002] As an example of this type of device, a device has been proposed that projects laser light from a laser head attached to a robot arm toward a portion to be soldered (see Patent Document 1). Another related technique is Patent Document 2. A technical problem with this type of device is, for example, how to properly solder a substrate having a three-dimensional shape (i.e., a 3D substrate). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] US Publication No. 2001 / 0054637 [Patent Document 2] US Publication No. 2015 / 0158176 Summary of the Invention

[0004] According to a first aspect, there is provided a soldering apparatus that irradiates processing light to melt solder placed on a circuit board, the soldering apparatus comprising: a light irradiation device having a galvanometer mirror and irradiating the processing light via the galvanometer mirror; a detection device that detects light from the circuit board and generates at least one of image data and shape data; a robot arm to which the light irradiation device and the detection device are provided and which has a drive unit that moves the light irradiation device and the detection device; and a control device that controls the orientation of the galvanometer mirror so that the processing light from the light irradiation device, which displaces together with the detection device, is irradiated to the same position based on the at least one of the data that changes with the displacement of the detection device.

[0005] According to a second aspect, there is provided a processing apparatus that irradiates processing light onto an object, the processing apparatus comprising: a light irradiation device having a scanning unit and irradiating the processing light via the scanning unit; a detection device that detects light from the object; a moving device in which the light irradiation device and the detection device are provided and having a drive unit that moves the light irradiation device and the detection device; and a control device that controls the scanning unit based on the detection result of the detection device.

[0006] According to a third aspect, there is provided a soldering system for soldering an element to a circuit board, the soldering system comprising: a first moving device provided with a solder dispensing device that dispenses solder and having a drive unit that moves the solder dispensing device; a second moving device provided with a holding device capable of holding the element and having a drive unit that moves the holding device; a third moving device provided with a light irradiation device that irradiates processing light to melt the solder and a detection device that detects light from the circuit board and having a drive unit that moves the light irradiation device and the detection device; and a control device that (i) controls the solder dispensing device so that solder is placed on a predetermined portion of the circuit board, (ii) controls the holding device so that the element is placed on the circuit board via the placed solder, and (iii) controls the drive unit of the third moving device so as to bring the light irradiation device closer to the circuit board based on the detection result of the detection device, and controls the light irradiation device so as to melt the placed solder.

[0007] According to a fourth aspect, there is provided a soldering system for soldering an element to a circuit board, the soldering system comprising: a solder dispensing device that dispenses solder; a holding device capable of holding the element; a light irradiation device that irradiates processing light to melt the solder; a detection device that detects light from the circuit board; a moving device having a drive unit that moves the solder dispensing device, the holding device, the light irradiation device and the detection device; and a control device that (i) controls the drive unit so that the solder dispensing device, the holding device, the light irradiation device and the detection device approach the circuit board, (ii) controls the solder dispensing device so that solder is placed on a predetermined portion of the circuit board, (iii) controls the holding device so that the element is placed on the circuit board via the placed solder, and (iv) controls the light irradiation device to melt the placed solder.

[0008] According to a fifth aspect, there is provided a processing apparatus that irradiates processing light onto an object, the processing apparatus comprising: a light irradiation device that irradiates the processing light; a detection device that detects light from the object; a moving device in which the light irradiation device and the detection device are provided and which has a drive unit that moves the light irradiation device and the detection device; and a control device that controls the drive unit based on the detection result of the detection device. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing a schematic diagram of the overall structure of a soldering system according to a first embodiment. [Diagram 2] FIG. 2 is a system configuration diagram showing the configuration of a robot that constitutes a part of the soldering system according to the first embodiment. [Diagram 3] FIG. 3 is a system configuration diagram showing the configuration of a robot that constitutes another part of the soldering system according to the first embodiment. [Figure 4] FIG. 4 is a system configuration diagram showing the configuration of a robot that constitutes another part of the soldering system according to the first embodiment. [Diagram 5]FIG. 5 is a diagram illustrating a schematic configuration of one detection device according to the first embodiment. [Figure 6] FIG. 6 is a diagram illustrating a schematic configuration of another detection device according to the first embodiment. [Figure 7] FIG. 7 is a diagram showing an example of structured light projected by a projector included in another detection device according to the first embodiment. [Figure 8] FIG. 8 is a diagram showing a part of an optical path of the light irradiation device according to the first embodiment. [Figure 9] FIG. 9 is a diagram illustrating a schematic configuration of a matching processing unit included in the control device according to the first embodiment. [Figure 10] FIG. 10 is a diagram for explaining the concept of the matching process according to the first embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of a timing chart of the matching process according to the first embodiment. [Figure 12] FIG. 12 is a diagram illustrating a schematic configuration of a tracking unit included in the control device according to the first embodiment. [Figure 13] FIG. 13 is a diagram showing an example of a timing chart of the tracking process according to the first embodiment. [Figure 14] FIG. 14 is a flowchart showing the operation of the soldering system according to the first embodiment. [Figure 15] FIG. 15 is a diagram showing an example of a method of irradiating processing light. [Figure 16] FIG. 16 is a schematic diagram showing an air blower and smoke suction machine. [Figure 17] FIG. 17 is a system configuration diagram showing the configuration of a soldering system according to the second embodiment. [Figure 18] FIG. 18 is a flowchart showing the operation of the soldering system according to the second embodiment. [Figure 19] FIG. 19 is a diagram showing a part of an optical path of a light irradiation device according to a modified example. [Figure 20] FIG. 20 is a diagram illustrating a schematic configuration of a tracking unit according to a modified example. [Figure 21] FIG. 21 is a diagram illustrating a schematic overall configuration of a laser welding system according to the third embodiment. [Figure 22] FIG. 22 is a system configuration diagram showing the configuration of a robot that constitutes a part of the laser welding system according to the third embodiment. [Figure 23] FIG. 23 is a system configuration diagram showing the configuration of a robot that constitutes another part of the laser welding system according to the third embodiment. [Figure 24] FIG. 24 is a flowchart showing the operation of the laser welding system according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment will be described with reference to the drawings.

[0011] First Embodiment A first embodiment will be described with reference to Figures 1 to 14. In this embodiment, a soldering system including a robot that performs soldering is provided.

[0012] (overview) An overview of the soldering system according to the first embodiment will be described with reference to Figures 1 to 4. In Figure 1, the soldering system is a soldering system that solders an element to a circuit board T. The soldering system includes robots 1, 2, and 3.

[0013] The robot 1, which may be referred to as a processing device or a solder application device, is provided with a dispenser 40 (see Figures 2(a) and 2(b)), which may be referred to as a solder dispensing device, that dispenses solder, and has a robot arm 110, which may be referred to as a first moving means, having a drive unit 111 (see Figure 2(b)) that moves the dispenser 40.

[0014] The robot 2, which may be referred to as a processing device or an element installation device, is provided with a holding device 50 (see Figures 3(a) and 3(b)), which may be referred to as a gripping device, capable of holding an element, and has a robot arm 210, which may be referred to as a second moving means, having a drive unit 211 (see Figure 3(b)) that moves the holding device 50.

[0015] The robot 3, which may be called a processing device or a soldering device, is provided with a light irradiation device 60 (see Figures 4(a) and 4(b)) that irradiates processing light to melt the solder, and detection devices 320 and 330 (see Figures 4(a) and 4(b)) that detect light from the circuit board T, and has a robot arm 310, which may be called a third moving means, having a drive unit 311 (see Figure 4(b)) that moves the light irradiation device 60 and the detection devices 320 and 330.

[0016] The "circuit board" may be a circuit board having a three-dimensional shape made up of a circuit film on which a circuit is formed and a substrate (i.e., a 3D circuit board). That is, the circuit board may be a circuit board manufactured by the IMPC (registered trademark) (In-Mold Printed Circuit) manufacturing method. The circuit board is not limited to a circuit board manufactured by the IMPC manufacturing method, but may be, for example, a circuit board having a three-dimensional shape made up of a circuit film and a substrate manufactured by another manufacturing method. The circuit board is not limited to a circuit board made up of a circuit film and a substrate, but may be a circuit board having another three-dimensional shape. The circuit board is not limited to a circuit board having a three-dimensional shape (3D circuit board), but may be a circuit board having a planar shape made up of a circuit film on which a circuit is formed and a substrate. The circuit board does not have to be a circuit board having a three-dimensional shape (3D circuit board), but may be a circuit board on which a circuit is formed on the substrate itself. The circuit board may be a circuit board for surface mounting, or may be a circuit board for insertion mounting.

[0017] The circuit board T may include, for example, markers (e.g., two-dimensional codes or cross marks such as AR (Augmented Reality) markers) that can be used for posture control or position control of at least one of the end effectors (i.e., the dispenser 40, the holding device 50, the light irradiation device 60) of the robots 1, 2, and 3, or a detection device described below, and solder pads (lands). Here, the above-mentioned markers, solder pads, and the like are detectable (e.g., recognizable by an image) by a detection device described below for posture control or position control of at least one of the end effectors of the robots 1, 2, and 3, or a detection device described below.

[0018] An "element" is an element soldered to a circuit board T by the soldering system, and examples thereof include electronic elements and electric elements. The element may be an element for surface mounting or an element for insertion mounting (i.e., a lead element). Such an "element" may be referred to as a "component." Examples of elements include an LED (Light Emitting Diode) (e.g., a well-known element such as a chip LED), a resistor (e.g., a well-known element such as a chip resistor), a capacitor (e.g., a well-known element such as a chip capacitor), a transistor (e.g., a well-known element such as a chip transistor), a connector, etc.

[0019] Although the term "robot arm" is used, it is not limited to a robot arm (i.e., a vertical articulated robot), and various existing modes such as a SCARA robot (i.e., a horizontal articulated robot), a parallel link robot, an orthogonal robot, etc. can be applied. In addition, an existing movement mechanism may be applied in place of the robot arm 312, etc., as long as it can move the light irradiation device 60, etc. The robot arms 110, 210, and 310 may be industrial robots or collaborative robots.

[0020] The soldering system includes a control device 1000 (see Figures 2(b), 3(b) and 4(b)) that (i) controls a dispenser 40 serving as an end effector of a robot 1 so that solder is placed on a predetermined portion of a circuit board T, (ii) controls a holding device 50 serving as an end effector of a robot 2 so that an element is placed on the circuit board T via the placed solder, and (iii) controls a drive unit 311 of a robot arm 310 so as to bring a light irradiation device 60 serving as an end effector of a robot 3 close to the circuit board T based on the detection results of at least one of the detection devices 320 and 330, and controls the light irradiation device 60 so as to melt the placed solder.

[0021] In FIG. 1, the control device 1000 first controls the dispenser 40 of the robot 1 so that solder is placed (in other words, so that solder is applied) on a predetermined portion of the circuit board T transported by the belt conveyer (solder placement process). The control device 1000 then controls the holding device 50 of the robot 2 so that an element is placed on the circuit board T' on which the solder has been placed via the placed solder (element placement process). The control device 1000 then controls the drive unit 311 of the robot arm 310 so as to bring the light irradiation device 60 close to the circuit board T'' on which the element has been placed based on at least one of the detection results of the detection devices 320 and 330, and controls the light irradiation device 60 so as to melt the placed solder (soldering process). After that, the control device 1000 may inspect the soldered solder and the element based on, for example, the detection result of the detection device 330 (inspection process).

[0022] In this way, the three robots 1, 2, and 3 work together to share the work, thereby making it possible to efficiently solder elements onto the circuit board T, thereby improving the throughput of element soldering.

[0023] In Figure 1, the circuit board on which the solder is placed is shown as "circuit board T'" and the circuit board on which the elements are mounted is shown as "circuit board T''. However, in the following, to avoid complexity of description, all will be referred to as "circuit board T".

[0024] The solder may be, for example, cream solder (i.e., solder paste), solder wire, solder bar, etc. That is, the dispenser 40 may place, on the circuit board T, for example, cream solder, solder wire, solder bar, etc.

[0025] Regarding the light irradiation device 60, "so as to melt the placed solder" includes irradiating a predetermined portion of the circuit board T with processing light to melt the solder. For example, this predetermined portion includes the solder placed on the solder pad of the circuit board T. In this case, the processing light from the light irradiation device 60 is directly irradiated to the solder to melt the solder. The predetermined portion also includes a part of the solder pad provided on the circuit board T (for example, a part of the solder pad where the solder is not placed) and a part of the element arranged on the circuit board T (for example, an electrode of the element). In this case, by irradiating a part other than the solder arranged on the circuit board T with processing light, the solder is indirectly melted by heat conduction from that part to the solder. The circuit board T may be a planar board, or may be a 3D circuit board having a three-dimensional shape as described above. When the circuit board T is a 3D circuit board, the predetermined portion may be set on an inclined surface on the circuit board T. In this case, the dispenser 40 of the robot 1 may place the solder on at least a part of the predetermined portion (for example, the solder pad) of the inclined surface. Then, the light irradiation device 60 of the robot 3 may irradiate processing light onto a specific portion (for example, a portion of the solder pad where no solder is placed) so as to melt the solder placed on the specific portion of the inclined surface as the specific portion.

[0026] Each of the robots 1, 2, and 3 will be described with reference to Figures 5 to 8 in addition to Figures 1 to 4. Here, the robot 3 will be mainly described, and the description of the robots 1 and 2 common to the robot 3 will be omitted as appropriate.

[0027] (Robot 3) As described above, the robot 3 is a robot that irradiates processing light to melt the solder arranged on the circuit board T. The robot 3 includes: (i) a light irradiation device 60 having a galvanometer mirror 61 (see FIG. 8 ) and irradiating processing light via the galvanometer mirror 61; (ii) detection devices 320 and 330 that detect light from the circuit board T and generate at least one of image data and shape data; and (iii) a robot arm 310 provided with the light irradiation device 60 and the detection devices 320 and 330 and having a drive unit 311 that moves the light irradiation device 60 and the detection devices 320 and 330.

[0028] 4(a), the robot arm 310 has arms 310a and 310b and a wrist 310c. The driving unit 311 may have, for example, a motor for rotating the entire robot arm 310, a motor for moving the entire robot arm 310 back and forth, a motor for moving each of the arms 310a and 310b up and down, a motor for rotating the arm 310b and the wrist 310c, a motor for rotating the wrist 310c, and a motor for bending the wrist 310c (all of which are not shown). The robot arm 310 may have a linear joint in addition to a rotary joint.

[0029] The driving unit 311 rotates and moves the entire robot arm 310 back and forth, and moves at least one of the arms 310a and 310b up and down, thereby moving the wrist 310c to a position near the circuit board T, for example. The driving unit 311 further rotates the arm 310b and the wrist 310c, and rotates and bends the wrist 310c, thereby moving the light irradiation device 60 or changing the attitude of the light irradiation device 60 so that processing light that melts the solder arranged on the circuit board T can be irradiated to at least a part of a predetermined portion (for example, the solder arranged on the circuit board T, the solder pads provided on the circuit board T, the elements arranged on the circuit board T, etc.).

[0030] That is, the driving unit 311 operates the robot arm 310, for example, as described above, whereby the detection devices 320 and 330 and the light irradiation device 60 are moved toward the circuit board T, for example.

[0031] It is assumed that the robot arm 310 and the detection device 320 (more precisely, the cameras 21 and 22 of the detection device 320) have been calibrated by an existing method. For example, an object (e.g., a checkerboard, etc.) whose exact shape is known is placed at an exact position in the three-dimensional coordinate system (so-called world coordinate system) of the robot arm 310, the object is imaged by the detection device 320, and a well-known arithmetic process is performed to obtain a correspondence relationship between the coordinate system of the detection device 320 (so-called camera coordinate system) and the coordinate system of the robot arm (i.e., calibration is performed). Similarly, it is assumed that the robot arm 310 and the detection device 330 (more precisely, the cameras 31 and 32 of the detection device 330) have been calibrated by an existing method.

[0032] 4(a), the detection device 320 is disposed on the arm 310b of the robot arm 310, and the detection device 330 is disposed on the wrist 310c of the robot arm 310, but the arrangement of the detection devices 320 and 330 is not limited thereto. The robot 3 may be provided with only one of the detection devices 320 and 330, or may be provided with another detection device in addition to the detection devices 320 and 330 (i.e., the robot 3 may be provided with three or more detection devices). The robot 3 may be provided with at least one detection device other than the detection devices 320 and 330. In other words, as long as the light irradiation device 60 can be brought close to the circuit board T or a predetermined portion of the circuit board T (e.g., a solder pad provided on the circuit board T, an element or solder arranged on the circuit board T, etc.) by driving the driving unit 311 of the robot arm 310 so that the solder arranged on the circuit board T can be melted with processing light, the configuration (e.g., the number and specifications of cameras in the detection device, the presence or absence of a projector, etc.) and the arrangement position and number of the detection devices 320 and 330 are not limited.

[0033] Detection device 320 The detection device 320 will be described with reference to Fig. 5. In Fig. 5, the detection device 320 is configured with cameras 21 and 22, which may be referred to as imaging devices. The cameras 21 and 22 each have an optical member such as a lens, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) or a CCD (Charge Coupled Device). As an example, in the detection device 320, the cameras 21 and 22 may be configured as stereo cameras arranged at a predetermined base length from each other.

[0034] The cameras 21 and 22 are configured to be able to detect light incident on them by the imaging element. Examples of the incident light include light reflected by an object (e.g., at least a part of the circuit board T), light scattered by the object, and light transmitted through the object. In other words, the cameras 21 and 22 detect light from an object present within each of the angles of view and capture an image of the object. That is, the cameras 21 and 22 are configured to be able to detect light incident on them and generate image data (i.e., data showing a two-dimensional image) as a detection result. In other words, the cameras 21 and 22 are configured to be able to output image data showing the captured image. In other words, the cameras 21 and 22 are configured to be able to detect an object. Here, the "image data" is, for example, data in which each pixel of the imaging element of each of the cameras 21 and 22 is associated (in other words, linked) with a pixel value, such as a luminance value, of each pixel.

[0035] The detection device 320 is configured to capture an image of an object (for example, at least a part of a circuit board T) by the cameras 21 and 22 at the same time, and generate and output shape data (i.e., shape data showing a three-dimensional shape of the object) as a detection result based on two pieces of image data output from the cameras 21 and 22, respectively. The detection device 320 is configured to output the generated shape data as shape data used in, for example, a matching process or a tracking process, which will be described later. Here, as an example, the shape data is three-dimensional point cloud data (hereinafter, also simply referred to as point cloud data). In this case, the detection device 320 calculates the distance from the cameras 21 and 22 to the object by a well-known method based on the difference (i.e., parallax) between the position of the object on the image captured by the camera 21 and the position of the object on the image captured by the camera 22, the focal lengths of the cameras 21 and 22, and the distance (i.e., baseline length) between the cameras 21 and 22. For example, the point cloud data is data in which points corresponding to each pixel of the cameras 21 and 22 are associated with three-dimensional information (X coordinate, Y coordinate, Z coordinate). Note that the shape data is not limited to point cloud data, and may be data representing existing three-dimensional information such as depth image data in which the distance to an object is associated as a luminance value of each pixel. In addition, the detection device 320 is configured to be capable of outputting image data generated by at least one of the cameras 21 and 22 as image data to be used in, for example, a matching process or a tracking process described below.

[0036] The detection device 320 is configured to be able to detect at least a part of the circuit board T from a wide range in order to approach the circuit board T when the circuit board T as an object is relatively far away from, for example, the light irradiation device 60. In other words, the detection device 320 is configured to be able to capture at least a part of the circuit board T and its periphery and generate at least one of image data and shape data over a wide range. For this reason, cameras with a wider field of view than the cameras 31 and 32 of the detection device 330 described later are used for the cameras 21 and 22. For example, cameras with a larger angle of view (in other words, a shorter focal length) than the cameras 31 and 32 described later are used for the cameras 21 and 22. In other words, for example, lenses with a larger angle of view (in other words, a shorter focal length) than the lenses of the cameras 31 and 32 described later are used for the cameras 21 and 22.

[0037] In addition, the detection device 320 can detect at least a part of the circuit board T from a wide range not only when the circuit board T as the target object and, for example, the light irradiation device 60 are relatively far apart, but also when the circuit board T as the target object and, for example, the light irradiation device 60 are relatively close to each other.

[0038] The detection device 320 generates, for example, at least one of image data and shape data of at least a part of the circuit board T as the target. Here, at least a part of the circuit board T (i.e., the target) is, for example, at least a part of the circuit board T itself, a marker (for example, a two-dimensional code such as an AR marker or a cross mark) formed on the circuit board T, or a solder pad (land) formed on the circuit board T. The detection device 120 may also generate, for example, at least one of image data and shape data of a marker (for example, a two-dimensional code such as an AR marker or a cross mark) arranged on the circuit board T or an element provided on the circuit board T. The detection device 120 may also generate, for example, at least one of image data and shape data of a marker (for example, a two-dimensional code such as an AR marker or a cross mark) arranged in the vicinity of the circuit board T.

[0039] The detection device 320 may be configured with a single camera instead of the cameras 21 and 22. In this case, the single camera generates image data of the object. The detection device 320 may further include another camera in addition to the cameras 21 and 22. In this case, for example, shape data may be generated by the cameras 21 and 22 (stereo cameras), and image data may be generated by another camera. The detection device 320 may include a projector in addition to the cameras 21 and 22, as in the detection device 330 described later. In this case, the detection device 320 may be configured to generate and output at least one of image data and shape data of the object. The detection device 320 may include a single camera and a projector instead of the cameras 21 and 22. In this case, the detection device 320 may be configured to generate and output at least one of image data and shape data of the object. For a specific method of generating shape data in a mode in which the detection device 320 includes a projector, see the description of the detection device 330 described later.

[0040] Detection device 330 The detection device 330 will be described with reference to FIG. 6. In FIG. 6, the detection device 130 is configured with cameras 31 and 32, which may be called imaging devices, and a projector 33, which may be called a projection device. For example, the cameras 31 and 32 each include an optical member such as a lens and an imaging element such as a CMOC or a CCD. As an example, the cameras 31 and 32 may be configured as stereo cameras arranged at a predetermined base length from each other. The cameras 31 and 32 are configured to be able to detect light incident on them. Examples of the incident light include light reflected by an object (for example, at least a part of the circuit board T), light scattered by an object, and light transmitted through an object. In other words, the cameras 31 and 32 detect light from an object present within each angle of view and capture an image of the object. In other words, the cameras 31 and 32 detect light from an object present within each angle of view and capture an image of the object. That is, each of the cameras 31 and 32 is configured to detect light incident thereon and generate image data (i.e., data representing a two-dimensional image) as a result of the detection. In other words, the cameras 31 and 32 are configured to generate image data representing a captured image. The detection device 330 is configured to output image data generated by at least one of the cameras 31 and 32 as image data to be used in, for example, a matching process or a tracking process described below.

[0041] The projector 33 is configured to be capable of projecting structured light having a predetermined intensity distribution (in other words, a predetermined pattern) as shown in, for example, Figures 7(a) to (c) during operation of the detection device 330. Note that the projector 33 can be implemented in various existing modes, such as a DLP (Digital Light Processing) type projector.

[0042] Here, the detection device 330 is configured to project structured light from the projector 33 onto the object, and generate image data of the object onto which the structured light is projected by the cameras 31 and 32. By projecting a predetermined pattern of structured light by the projector 33, the detection device 330 can generate highly accurate shape data that is less affected by disturbances, even if the surface of the object is dark or has few feature points.

[0043] The detection device 330 is configured to simultaneously capture images of an object onto which structured light is projected from the projector 33 using the cameras 31 and 32, generate shape data (i.e., shape data showing the three-dimensional shape of the object) based on the two image data outputted from the cameras 31 and 32, and output the shape data. Here, as an example, the detection device 320 calculates the distance from the cameras 31 and 32 to the object using a known method based on the difference (i.e., parallax) between the position of the pattern by the structured light on the image captured by the camera 31 and the position of the pattern by the structured light on the image captured by the camera 32, the focal lengths of the cameras 31 and 32, and the distance (i.e., base length) between the cameras 31 and 32, and generates three-dimensional point cloud data (hereinafter also simply referred to as point cloud data). The detection device 330 is configured to be able to output the generated shape data as shape data to be used in, for example, a matching process or a tracking process described later.

[0044] The shape data is not limited to point cloud data, and may be data representing existing three-dimensional information such as depth image data in which the distance to the object is associated with the brightness value of each pixel. The shape data can be generated using various existing methods such as a phase shift method, a random dot method, and a TOF (Time-of-Flight) method.

[0045] Furthermore, the detection device 330 is configured to be able to generate image data by at least one of the cameras 31 and 32 in a state in which structured light is not projected from the projector 33 .

[0046] When the circuit board T as an object and, for example, the light irradiation device 60 are relatively close to each other, the detection device 330 is configured to be able to detect at least a part of the circuit board T with high accuracy since it approaches a part of the circuit board T (for example, solder arranged on the circuit board T, solder pads provided on the circuit board T, elements arranged on the circuit board T, etc.). In other words, it is configured to be able to capture at least a part of the circuit board T and its periphery and generate at least one of highly accurate image data and shape data. For this reason, the cameras 31 and 32 have a higher resolution than the cameras 21 and 22 provided in the detection device 320. In this case, for example, the cameras 31 and 32 have a narrower angle of view (in other words, a longer focal length) than the cameras 21 and 22. For example, the lenses of the cameras 31 and 32 have a narrower angle of view (in other words, a longer focal length) than the lenses of the cameras 21 and 22. In addition, the lenses of the cameras 31 and 32 may have a higher shooting magnification than the lenses of the cameras 21 and 22.

[0047] Since the resolution of the object captured by the cameras 31 and 32 is high, the resolution of the image data and the accuracy of the shape data generated by the detection device 230 including the cameras 31 and 32 are higher than the resolution of the image data and the accuracy of the shape data generated by the detection device 320 including the cameras 21 and 22. For this reason, the detection device 330 can detect at least a part of the circuit board T with higher accuracy than the detection device 320. Therefore, by using the image data and the shape data generated by the detection device 330 in the control device 1000, the accuracy of estimating the position and orientation by the matching process in the matching processing unit 200 described later and the accuracy of estimating the position and orientation by the tracking process in the tracking unit 300 are improved.

[0048] In addition, the detection device 330 can detect at least a part of the circuit board T with high accuracy not only when the circuit board T as the target object and, for example, the light irradiation device 60 are relatively close to each other, but also when the circuit board T as the target object and, for example, the light irradiation device 60 are relatively far apart.

[0049] As a result, the field of view of cameras 31 and 32 of detection device 330 is narrower than the field of view of cameras 21 and 22 of detection device 320. Note that detection device 320 may be referred to as a first imaging unit, and detection device 330 may be referred to as a second imaging unit.

[0050] The detection device 330 generates, for example, at least one of image data and shape data of at least a part of the circuit board T as the target. Here, at least a part of the circuit board T (i.e., the target) is, for example, at least a part of the circuit board T itself, a marker (for example, a two-dimensional code such as an AR marker or a cross mark) formed on the circuit board T, or a solder pad (land). The detection device 330 may also generate, for example, at least one of image data and shape data of a marker (for example, a two-dimensional code such as an AR marker or a cross mark) arranged on the circuit board T, or an element or solder provided on the circuit board T. The detection device 330 may also generate, for example, at least one of image data and shape data of a marker (for example, a two-dimensional code such as an AR marker or a cross mark) arranged in the vicinity of the circuit board T.

[0051] The detection device 330 may be configured with a single camera instead of the cameras 31 and 32. In this case, the single camera generates image data of the object. The detection device 330 may further include another camera in addition to the cameras 21 and 22. In this case, for example, the cameras 31 and 32 (stereo cameras) and the projector 33 may generate shape data, and the other camera may generate image data. The detection device 330 may be configured with a single camera and the projector 33 instead of the cameras 31 and 32. In this case, the detection device 330 may be configured to generate and output at least one of the image data and shape data of the object.

[0052] Light irradiation device 60 The light irradiation device 60 will be described with reference to FIG. 8. In FIG. 8, the light irradiation device 60 is configured to include a galvanometer mirror 61, which can also be referred to as a scanning unit, and an fθ lens 62. Therefore, the light irradiation device 60 can move the irradiation position of the processing light L on the object (for example, at least a part of the circuit board T) along a desired direction (in other words, scan the irradiation position of the processing light L on the object). The galvanometer mirror 61 is configured to be able to change the direction of its own mirror, and by changing the direction of its own mirror, the emission direction of the processing light L incident from a light source (not shown) is changed. The processing light L emitted from the galvanometer mirror 61 is incident on the fθ lens 62. The fθ lens 62 condenses the processing light L incident from the galvanometer mirror 61. In other words, the light irradiation device 60 can change the irradiation position of the processing light L irradiated onto the target circuit board T via the fθ lens 62 depending on the orientation of its own mirror in the galvanometer mirror 61 (in other words, the change in the emission direction of the processing light L from the galvanometer mirror 61).

[0053] For example, the galvanometer mirror 61 includes a first scanning mirror 61Y and a second scanning mirror 61X, each of which has a mirror that can oscillate or rotate around a predetermined axis, and is arranged so that the axes of oscillation or rotation of the first scanning mirror 61Y and the second scanning mirror 61X intersect (for example, perpendicularly). In this case, the processing light L incident on the first scanning mirror 61Y is reflected by the first scanning mirror 61Y and incident on the second scanning mirror 61X, and is then reflected by the second scanning mirror 61X and incident on the fθ lens 62. The fθ lens 62 condenses the processing light L incident from the second scanning mirror 61X. The emission direction of the processing light L from the second scanning mirror 61X differs depending on the orientation of the first scanning mirror 61Y around the axis and the orientation of the second scanning mirror 61X around the axis (in other words, the incident position of the processing light L on the fθ lens 62 differs), so the irradiation position of the processing light on the circuit board T changes depending on the orientation of the first scanning mirror 61Y and the second scanning mirror 61X.

[0054] Furthermore, the light irradiation device 60 can melt the solder arranged on the circuit board T by irradiating the processing light L onto the solder. Furthermore, it is not limited to directly irradiating the processing light L onto the solder, but for example, it is also possible to indirectly melt the solder by irradiating the processing light onto a solder pad (e.g., a portion of the solder pad on which no solder is arranged) provided on the circuit board T, or to indirectly melt the solder by irradiating the processing light onto a part (e.g., an electrode) of an element (component) arranged on the circuit board T.

[0055] The galvanometer mirror 61 is not limited to two scanning mirrors (first scanning mirror 61Y and second scanning mirror 61X), but may be a single scanning mirror or may be composed of three or more scanning mirrors. The light irradiation device 60 is not limited to the galvanometer mirror 61, but may be another existing device that changes the emission direction of light, such as a polygon mirror, a DMD (Digital Micromirror Device), or a spatial light modulator. The light irradiation device 60 is not limited to a configuration including an fθ lens 62, but may not include the fθ lens 62, or may include one or more other lenses instead of the fθ lens 62.

[0056] The light source (not shown) of the processing light L incident on the galvanometer mirror 61 of the light irradiation device 60 may be disposed outside the soldering system, may be included in the soldering system, may be included in the robot 3, or may be included in the light irradiation device 60. The light source (not shown) can change the intensity of the processing light L irradiated to the object. The method of changing the intensity of the processing light L irradiated to the object is not limited to the method of changing the intensity of the light emitted from the light source, and a method using an existing light intensity changing member such as an ND filter can also be applied. The light from the light source (not shown) is incident on the galvanometer mirror 61 of the light irradiation device 60 by an existing method. The light irradiation device 60 may include a focus lens. The focus lens is composed of one or more lenses, and by changing the position along the optical axis direction of at least a part of the lenses, the focusing position of the processing light L in the optical axis direction of the light irradiation device 60 (i.e., the focal position of the light irradiation device 60) can be changed. In other words, the spot size of the processing light L irradiated to the object can be changed. For example, the focus lens may be disposed on the optical path of the processing light L before it is incident on the galvanometer mirror 61. Note that the configuration of the light irradiation device 60 for changing the spot size of the processing light L on the target object is not limited to the focus lens, and an existing configuration may also be applied.

[0057] The control device 1000 may control the orientation of the galvanometer mirror 61 of the robot 3 configured as described above based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 320 and 330 so that the processing light L from the light irradiation device 60, which displaces with the displacement of at least one of the detection devices 320 and 330, is irradiated to the same position.

[0058] The control device 1000 may control the driving unit 311 to stop driving the driving unit 311. The control device 1000 may control the orientation of the galvanometer mirror 61 based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 320 and 330 after the driving of the driving unit 311 is stopped, so that the processing light L from the light irradiation device 60, which displaces together with at least one of the detection devices 320 and 330, is irradiated to the same position.

[0059] The control device 1000 may control the drive unit 311 so that the light irradiation device 60 and the detection devices 320 and 330 are moved, while controlling the orientation of the galvanometer mirror 61 so that the processing light L from the light irradiation device 60 moved by the robot arm 310 is irradiated to the same position based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 320 and 330 moved by the robot arm 310.

[0060] The control device 1000 may control the orientation of the galvanometer mirror 61 so that the irradiation position of the processing light L from the light irradiation device 60, which displaces with the displacement of at least one of the detection devices 320 and 330, is maintained at a first position and then maintained at a second position different from the first position, based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 320 and 330.

[0061] The control device 1000 may control the driving unit 311 to stop driving the driving unit 311. Based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 320 and 330 after the driving of the driving unit 311 is stopped, the control device 1000 may control the orientation of the galvanometer mirror 61 so that the irradiation position of the processing light L from the light irradiation device 60, which displaces with the displacement of at least one of the detection devices 320 and 330, is maintained at a first position and then maintained at a second position different from the first position.

[0062] The control device 1000 may control the drive unit 311 to move the light irradiation device 60 and the detection devices 320 and 330, while controlling the orientation of the galvanometer mirror 61 so that the irradiation position of the processing light L from the light irradiation device 60 moved by the robot arm 310 is maintained at a first position and then maintained at a second position different from the first position, based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 320 and 330 moved by the robot arm 310.

[0063] The control device 1000 controls the drive unit 311 of the robot arm 310 so that the light irradiation device 60 and the detection devices 320 and 330 approach the circuit board T based on at least one of the image data and shape data, and when the light irradiation device 60 and the detection devices 320 and 330 approach the circuit board T to a predetermined distance, the control device 1000 may control the orientation of the galvanometer mirror 61 so that the processing light L from the light irradiation device 60, which displaces together with at least one of the detection devices 320 and 330, is irradiated to the same position based on the at least one of the above data that changes in accordance with the displacement of at least one of the detection devices 320 and 330.

[0064] The light irradiation device 60 does not necessarily have to include a scanning unit such as the galvanometer mirror 61.

[0065] Here, the control device 1000 may be a device different from the robot 3, which can also be described as a soldering device, or may constitute a part of the robot 3 (in other words, the robot 3 may be equipped with the control device 1000). In the latter case, the control device 1000 may be provided independently by the robot 3, or may be shared by the robot 3 and at least one of the robots 1 and 2 (i.e., the control device 1000 that constitutes a part of the robot 3 may control at least one of the robots 2 and 3 in addition to the robot 3). When the robot 3 is provided with its own control device 1000, the robots 1 and 2 may each be provided with their own control device 1000 that is different from the control device 1000 provided in the robot 3.

[0066] (Robot 1) As described above, the robot 1 is a robot that places solder on a predetermined portion (e.g., a solder pad, a part of a circuit, etc.) of the circuit board T. In Fig. 2(a) and Fig. 2(b), the robot 1 includes: (i) a dispenser 40 that dispenses solder; (ii) detection devices 120 and 220 that detect light from the circuit board T and generate at least one of image data and shape data; and (iii) a robot arm 110 that is provided with the dispenser 40 and the detection devices 120 and 130 and has a drive unit 111 that moves the dispenser 40 and the detection devices 120 and 130.

[0067] The robot arm 110 has arms 110a and 110b and a wrist 110c, similar to the robot arm 310. The detectors 120 and 130 may be configured similarly to the detectors 320 and 330, respectively. The dispenser 40 is capable of changing the amount of cream solder dispensed, and the control device 1000 can control the amount of solder dispensed from the dispenser 40.

[0068] In FIG. 2(a), the detection device 120 is disposed on the arm 110b of the robot arm 110, and the detection device 130 is disposed on the wrist 110c of the robot arm 110, but the arrangement of the detection devices 120 and 130 is not limited thereto. The robot 1 may be provided with only one of the detection devices 120 and 130, or may be provided with another detection device in addition to the detection devices 120 and 130 (i.e., the robot 1 may be provided with three or more detection devices). In other words, as long as the dispenser 40 can be brought close to the circuit board T or a predetermined portion of the circuit board T (e.g., a solder pad or a part of a circuit) by driving the drive unit 111 of the robot arm 110 so that solder can be placed on a predetermined portion of the circuit board T (e.g., a solder pad or a part of a circuit), the configuration (e.g., the number and specifications of the cameras in the detection devices, the presence or absence of a projector, etc.) and the arrangement positions and number of the detection devices 120 and 130 may be any.

[0069] The detection devices 120 and 130 may have the same configuration as the above-mentioned detection devices 320 and 330, respectively. However, the detection devices 120 and 130 do not have to have the same configuration as the above-mentioned detection devices 320 and 330, respectively. The configuration and specifications of the detection device 120 can be appropriately changed within a range that does not contradict the gist or idea that can be read from the description of the above-mentioned detection device 320, etc. Similarly, the configuration and specifications of the detection device 130 can be appropriately changed within a range that does not contradict the gist or idea that can be read from the description of the above-mentioned detection device 330, etc.

[0070] The control device 1000 may control the drive unit 111 of the robot 1 configured as described above so that the solder dispensed from the dispenser 40, which displaces with the displacement of at least one of the detection devices 120 and 130, is placed at a predetermined portion of the circuit board T, based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 120 and 130.

[0071] The control device 1000 may control the driving unit 111 to stop driving the driving unit 111. The control device 1000 may control the driving unit 111 so that the solder dispensed from the dispenser 40, which displaces together with at least one of the detection devices 120 and 130, is placed at a predetermined portion of the circuit board T, based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 120 and 130 after the driving of the driving unit 111 is stopped.

[0072] The control device 1000 may control the drive unit 111 so that the solder dispensed from the dispenser 40, which displaces with the displacement of at least one of the detection devices 120 and 130, is placed at a first position on the circuit board T, and then is placed at a second position different from the first position, based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 120 and 130.

[0073] The control device 1000 may control the drive unit 111 to stop driving the drive unit 111. The control device 1000 may control the drive unit 111 based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 120 and 130 after the drive of the drive unit 111 is stopped, so that the solder dispensed from the dispenser 40, which displaces with the displacement of at least one of the detection devices 120 and 130, is placed at a first position on the circuit board T and then placed at a second position different from the first position.

[0074] The control device 1000 may control the drive unit 111 of the robot arm 110 so that the dispenser 40 and the detection devices 120 and 130 approach the circuit board T based on at least one of the image data and shape data, and when the dispenser 40 and the detection devices 120 and 130 approach a predetermined distance from the circuit board T, control the drive unit 111 so that the solder dispensed from the dispenser 40, which displaces together with at least one of the detection devices 120 and 130, is placed on a predetermined portion of the circuit board T based on the at least one of the data that changes in accordance with the displacement of at least one of the detection devices 120 and 130.

[0075] Here, the control device 1000 may be a device different from the robot 1, which can also be referred to as a solder application device, or may constitute a part of the robot 1 (in other words, the robot 1 may be equipped with the control device 1000).

[0076] (Robot 2) As described above, the robot 2 is a robot that places an element via solder arranged on the circuit board T. In Fig. 3(a) and Fig. 3(b), the robot 2 includes: (i) a holding device 50 that holds the element; (ii) detection devices 220 and 230 that detect light from the circuit board T and generate at least one of image data and shape data; and (iii) a robot arm 210 that is provided with the holding device 50 and the detection devices 220 and 230 and has a drive unit 211 that moves the holding device 50 and the detection devices 220 and 230.

[0077] The robot arm 210 has arms 210a and 210b and a wrist 210c, similar to the robot arm 310. The detection devices 220 and 230 may be configured similarly to the detection devices 220 and 230, respectively. In addition, the holding device 50 can be an existing device such as a tweezers hand or a suction device, as long as it can hold an element.

[0078] The holding force (gripping force) of the element in the holding device 50 can be changed, and the control device 1000 can control the holding force of the element in the holding device 50. For example, when the holding device 50 is a tweezers hand, the holding device 50 can control the force with which the tip of the tweezers pinches the element. The robot 2 may include a storage unit (not shown) for storing the element, and a supply device (not shown) for supplying a desired element from the storage unit to the holding device 50. Examples of the storage unit include a reel, a tray, a stick, and the like. Since various existing aspects can be applied to the storage unit and the supply device, detailed explanations thereof will be omitted. The control device 1000 may control the supply device to supply a desired element to be placed on a part (predetermined part) of the circuit board T from the storage unit to the holding device 50, and control the holding device 50 to hold the element. In this case, the robot 2 can omit the task of bringing the holding device 50 close to a separate element supplying device (so-called parts feeder) (not shown) that is provided to supply elements to be placed on a part (specific portion) of the circuit board T and holding the desired element with the holding device 50, thereby making it possible to efficiently place elements on the circuit board T.

[0079] In FIG. 3(a), the detection device 220 is disposed on the arm 210b of the robot arm 210, and the detection device 230 is disposed on the wrist 210c of the robot arm 210, but the arrangement of the detection devices 220 and 230 is not limited thereto. The robot 2 may be provided with only one of the detection devices 220 and 230, or may be provided with another detection device in addition to the detection devices 220 and 230 (i.e., the robot 2 may be provided with three or more detection devices). In other words, as long as the holding device 50 can be brought close to a predetermined portion of the circuit board T (solder pads, solder arranged on a circuit, etc.) by driving the driving unit 211 of the robot arm 210 so that an element can be arranged on the predetermined portion of the circuit board T (solder pads, solder arranged on a circuit, etc.), the configuration (for example, the number and specifications of cameras in the detection devices, the presence or absence of a projector, etc.) and the arrangement positions and number of the detection devices 120 and 130 may be any.

[0080] The detection devices 220 and 230 may have the same configuration as the above-mentioned detection devices 320 and 330, respectively. However, the detection devices 220 and 230 do not have to have the same configuration as the above-mentioned detection devices 320 and 330, respectively. The configuration and specifications of the detection device 220 can be appropriately changed within a range that does not contradict the gist or idea that can be read from the above-mentioned description of the detection device 320, etc. Similarly, the configuration and specifications of the detection device 230 can be appropriately changed within a range that does not contradict the gist or idea that can be read from the above-mentioned description of the detection device 330, etc.

[0081] The control device 1000 may control the drive unit 211 for the robot 2 configured as described above so that an element grasped (held) by the holding device 50, which displaces with the displacement of at least one of the detection devices 220 and 230, is positioned at a predetermined portion of the circuit board T, based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 220 and 230.

[0082] The control device 1000 may control the drive unit 211 to stop driving the drive unit 211. The control device 1000 may control the drive unit 211 so that the element held by the holding device 50, which displaces together with at least one of the detection devices 220 and 230, is positioned at a predetermined portion of the circuit board T, based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 220 and 230 after the drive of the drive unit 211 has stopped.

[0083] The control device 1000 may control the drive unit 211 so that, based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 220 and 230, after one element held by the holding device 50, which displaces with the displacement of at least one of the detection devices 220 and 230, is positioned at a first position on the circuit board T, another element held by the holding device 50 is positioned at a second position different from the first position.

[0084] The control device 1000 may control the drive unit 211 to stop driving the drive unit 211. The control device 1000 may control the drive unit 211 based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 220 and 230 after the drive of the drive unit 211 is stopped, so that after one element held by the holding device 50 that displaces with the displacement of at least one of the detection devices 220 and 230 is placed at a first position on the circuit board T, another element held by the holding device 50 is placed at a second position different from the first position.

[0085] The control device 1000 may control the drive unit 211 of the robot arm 210 so that the holding device 50 and the detection devices 220 and 230 approach the circuit board T based on at least one of the image data and shape data, and when the holding device 50 and the detection devices 220 and 230 approach a predetermined distance from the circuit board T, control the drive unit 211 so that the element held by the holding device 50, which displaces together with at least one of the detection devices 220 and 230, is positioned at a predetermined portion of the circuit board T based on the at least one of the data that changes in accordance with the displacement of at least one of the detection devices 220 and 230.

[0086] Here, the control device 1000 may be a device separate from the robot 2, which can also be referred to as an element installation device, or may constitute a part of the robot 2 (in other words, the robot 2 may be equipped with the control device 1000).

[0087] In FIG. 1, the control device 1000 controls the drive unit 311 so that the positional relationship between the circuit board T transported by the belt conveyor and, for example, the light irradiation device 60 of the robot 3 becomes a desired positional relationship and / or so that the posture of the light irradiation device 60 becomes a desired posture (for example, the entire robot arm 310 is rotated, moved back and forth, the arms 310a and 310b are moved up and down, the arms 310b and the wrist 310c are rotated, and the wrist 310c is rotated and bent). In other words, the control device 1000 controls at least one of the position and posture of the light irradiation device 60 by controlling the drive unit 311 so that the processing light L from the light irradiation device 60 is irradiated to a predetermined portion of the circuit board T transported by the belt conveyor. At this time, the belt conveyor is temporarily stopped. That is, the belt conveyor is temporarily stopped after transporting the circuit board T within, for example, the driveable range of the robot arm 310 of the robot 3. Then, after the processing light L from the light irradiation device 60 is irradiated onto a predetermined portion of the stopped circuit board T, the belt conveyor resumes driving in order to transport the circuit board T.

[0088] Similarly, the control device 1000 controls the drive unit 111 so that the positional relationship between the circuit board T transported by the belt conveyor and, for example, the dispenser 40 of the robot 1 becomes a desired positional relationship and / or so that the attitude of the dispenser 40 becomes a desired attitude. In other words, the control device 1000 controls at least one of the position and attitude of the dispenser 40 by controlling the drive unit 111 so that the solder discharged from the dispenser 40 is placed on a predetermined portion of the circuit board T transported by the belt conveyor. At this time, the belt conveyor is temporarily stopped. That is, the belt conveyor is temporarily stopped after transporting the circuit board T within, for example, the driveable range of the robot arm 110 of the robot 1. Then, after the solder discharged from the dispenser 40 is placed on the predetermined portion of the stopped circuit board T, the belt conveyor resumes driving to transport the circuit board T.

[0089] Similarly, the control device 1000 controls the drive unit 211 so that the positional relationship between the circuit board T transported by the belt conveyor and, for example, the holding device 50 of the robot 2 is a desired positional relationship, and / or so that the attitude of the holding device 50 is a desired attitude. In other words, the control device 1000 controls at least one of the position and attitude of the holding device 50 by controlling the drive unit 211 so that the element held by the holding device 50 is placed in a predetermined portion of the circuit board T transported by the belt conveyor. At this time, the belt conveyor is temporarily stopped. That is, the belt conveyor is temporarily stopped after transporting the circuit board T within, for example, the driveable range of the robot arm 210 of the robot 2. Then, after the element held by the holding device 50 is placed in the predetermined portion of the stopped circuit board T, the belt conveyor resumes driving to transport the circuit board T.

[0090] The belt conveyor may be constantly driven without pausing in front of each robot arm. In other words, the control device 1000 may control the driving unit 311 so that the processing light L from the light irradiation device 60 is irradiated to a predetermined portion of the circuit board T that is being transported (i.e., moving) by the belt conveyor, control the driving unit 111 so that the solder dispensed from the dispenser 40 is disposed, and control the driving unit 211 so that the element held by the holding device 50 is disposed.

[0091] The conveying device for the circuit board T is not limited to a belt conveyor, and various existing modes can be applied as long as the circuit board T can be conveyed. In addition, the circuit board T does not have to be conveyed by a belt conveyor or the like. For example, the above-mentioned robots 1, 2, and 3 may be arranged to surround a stage on which the circuit board T is placed. In this case, the control device 1000 may control the driving unit 311 so that the processing light L from the light irradiation device 60 is irradiated to a predetermined portion of the circuit board T placed on the stage, control the driving unit 111 so that the solder discharged from the dispenser 40 is placed, or control the driving unit 211 so that the element held by the holding device 50 is placed. At this time, the circuit board T may be carried into the stage and carried out from the stage by a robot other than the robots 1, 2, and 3.

[0092] At least one of the robot arms 110, 210, and 310 may be mounted on, for example, an AGV (Automatic Guided Vehicle). In this case, the control device 1000 may control at least one of the drive unit of at least one of the robot arms 110, 210, and 310, the end effector of at least one of the robot arms 110, 210, and 310, and the drive unit of the AGV, based on information on the position and posture of an object acquired by a matching process or a tracking process described below.

[0093] 9 to 13, a process performed by the control device 1000 to enable control of the driving unit 311 etc. (in other words, the robot arm 310 etc.) will be described. Here, as an example, a process using the outputs of the detection devices 320 and 330 provided in the robot 3 will be described. The control device 1000 may perform similar processes using the outputs of the detection devices 120 and 130 provided in the robot 1 and the outputs of the detection devices 220 and 230 provided in the robot 2.

[0094] (Calculation of object position and orientation by matching process) The matching process of the circuit board T as the target object performed by the control device 1000 will be described with reference to Fig. 9 to Fig. 11. It is necessary to bring the light irradiation device 60 of the robot 3 closer to the target object by the robot arm 310 so that the position and posture of the light irradiation device 60 of the robot 3 relative to a part of the target object (for example, a part of the circuit board T to be irradiated with the processing light L) transported by the belt conveyer is a desired position and posture. In order to bring the light irradiation device 60 closer to the target object by the robot arm 310, the control device 1000 can perform a matching process to calculate (estimate) the position and posture of the target object.

[0095] In FIG. 9, a control device 1000 has a robot control unit 100 and a matching processing unit 200 as a processing block logically realized therein or as a processing circuit physically realized therein.

[0096] In addition, when the light irradiation device 60, which is located relatively far from the circuit board T, approaches the circuit board T, the matching processing unit 200 determines whether to use the output of the detection device 320 (e.g., at least one of image data and shape data) or the output of the detection device 330 (e.g., at least one of image data and shape data) to control the drive unit 311.

[0097] The matching processing unit 200 is configured to include a first matching unit 201, a second matching unit 202, and a comparison unit 203.

[0098] The first matching unit 201 performs matching between the output of the detection device 320 (for example, at least one of image data and shape data) and CAD (Computer-Aided Design) data. As a result of matching, the first matching unit 201 outputs a position and orientation estimation result of the object as a result of estimating (calculating) the position and orientation of the object, and a matching rate. The position and orientation estimation result of the object may be expressed in so-called 6DoF (six degrees of freedom). In this case, the position and orientation estimation result becomes data representing, for example, the X coordinate, Y coordinate, Z coordinate, a component around the X axis (θX component), a component around the Y axis (θY component), and a component around the Z axis (θZ component) in the coordinate system (so-called world coordinate system) of the robot arm 310 of the X axis, Y axis, and Z axis. These data may be expressed in a matrix, or each value may be expressed as a table, or may be data in other well-known formats. Note that the X, Y, and Z coordinates are estimation results indicating the position of the object, and the components around the X, Y, and Z axes are estimation results indicating the orientation of the object. Note that the components around the X, Y, and Z axes can be rephrased as yaw, roll, and pitch. Note that in the first matching unit 201, calculating (estimating) the position and orientation of a feature region (described later) in the object is included in the concept of calculating (estimating) the position and orientation of the object.

[0099] As a matching process, the first matching unit 201, for example, calculates (estimates) the position and orientation of the object by comparing a feature area of ​​the object in the image data output from the detection device 320 (for example, a part of the contour of the object in the image data) with a feature area of ​​the object in the CAD data of the object (for example, CAD data corresponding to a part of the contour of the object in the image data). At this time, the first matching unit 201 first extracts the feature area of ​​the object in the image data output from the detection device 320 and the feature area of ​​the object in the CAD data of the object. The first matching unit 201 then calculates the position and orientation of the object in the coordinate system (world coordinate system) of the robot arm 310 by, for example, changing the dimensions of the feature area of ​​the object in the CAD data and rotating it to make a correspondence between the feature area of ​​the object in the image data and the feature area of ​​the object in the CAD data. More specifically, first, the first matching 201 obtains a correspondence relationship between the CAD coordinate system (so-called local coordinate system) and the coordinate system (camera coordinate system) of the detection device 320 so that the feature area of ​​the object in the CAD data matches the feature area of ​​the object in the image data. Since the position and orientation of the feature area of ​​the object in the CAD coordinate system (i.e., the feature area of ​​the object in the CAD data) are known, the position and orientation of the feature area of ​​the object in the coordinate system of the detection device 320 can be known by matching the CAD coordinate system with the coordinate system of the detection device 320. Next, the first matching 201 calculates the position and orientation of the object in the coordinate system of the robot arm 310 based on the correspondence relationship between the CAD coordinate system and the coordinate system of the detection device 320 and the correspondence relationship between the coordinate system of the detection device 320 and the coordinate system of the robot arm 310 obtained in advance by calibration.

[0100] The first matching unit 201 may use various existing methods such as Scale-Invariant Feature Transform (SIFT) and Speed-Upped Robust Feature (SURF) for the matching process of image data. The first matching unit 201 may perform matching process between a plurality of feature regions of an object in the image data and a plurality of feature regions of the object in the CAD data, and calculate the positions and orientations of the plurality of feature regions in the coordinate system of the robot arm 310. In this case, the first matching unit 201 may output the calculated positions and orientations of the plurality of feature regions of the object, or may calculate (estimate) and output the position and orientation of the center of gravity of the object based on the positions and orientations of the plurality of feature regions.

[0101] The characteristic region of the object for which the position and orientation are calculated is not limited to a part of the contour of the object, but may be any region that can be identified as being different from the surroundings on the image, such as a marker provided on the object or a pattern on the surface of the object. For example, the characteristic region of the circuit board T as the object may be, for example, a marker (e.g., a two-dimensional code or a cross mark such as an AR marker) or a solder pad or solder formed on the circuit board T as at least a part of the circuit board T, a marker (e.g., a two-dimensional code or a cross mark such as an AR marker) arranged on the circuit board T, an element arranged on the circuit board T, or a marker (e.g., a two-dimensional code or a cross mark such as an AR marker) arranged in the vicinity of the circuit board T. The data that the first matching unit 201 performs matching processing with the characteristic region in the image data output from the detection device 320 is not limited to CAD data, and other data may be used. For example, the first matching unit 201 may perform a matching process between a feature region of image data generated in advance by capturing an image of a reference object (e.g., a reference circuit board) and a feature region in image data of the circuit board T output from the detection device 320. In this case, the reference object may be, for example, a non-defective object (e.g., a non-defective circuit board).

[0102] In addition, the matching rate is a value that indicates the degree of agreement between the characteristic area of ​​an object in image data and the characteristic area of ​​the object in CAD data (here, the higher the degree of agreement between the two, the higher the matching rate).

[0103] In addition, as a matching process, the first matching unit 201 can also calculate (estimate) the position and posture of the object by comparing point cloud data of a characteristic region of the object (e.g., part of a corner of the object in the point cloud data) in the shape data (e.g., point cloud data) output from the detection device 320 with point cloud data of a characteristic region of the object in the CAD data of the object (e.g., point cloud data in the CAD data corresponding to part of a corner of the object in the shape data).

[0104] At this time, the first matching unit 201 first extracts point cloud data of a feature region of the object in the shape data (for example, point cloud data) output from the detection device 320 and point cloud data of the feature region of the object in the CAD data of the object. The first matching unit 201 then calculates the position and posture of the object in the coordinate system of the robot 310 by, for example, changing the coordinates of each point or the interval between each point in the point cloud data of the feature region of the object in the CAD data, or by rotating the point cloud, thereby making the point cloud data of the feature region of the object in the shape data correspond to the point cloud data of the feature region of the object in the CAD data. More specifically, the first matching unit 201 first obtains a correspondence relationship between the CAD coordinate system and the coordinate system of the detection device 320 so that the point cloud data of the feature region of the object in the CAD data matches the point cloud data of the feature region of the object in the shape data. Since the position and orientation of the point cloud of the feature region of the object in the CAD coordinate system are known, the position and orientation of the point cloud of the feature region of the object in the coordinate system of the detection device 320 can be known by matching the CAD coordinate system with the coordinate system of the detection device 320. Next, first matching 201 calculates the position and orientation of the object in the coordinate system of the robot arm 310 based on the correspondence between the CAD coordinate system and the coordinate system of the detection device 320 and the correspondence between the coordinate system of the detection device 320 and the coordinate system of the robot arm 310 obtained in advance by calibration.

[0105] The matching process of the shape data by the first matching unit 201 can use various existing methods such as RANSAC (Random Sample Consensus), SIFT (Scale-Invariant Feature Transform), and ICP (Iterative Closest Point). The first matching unit 201 may output the calculated position and orientation of the feature region of the object as the position and orientation of the object. The first matching unit 201 may also perform a matching process between point cloud data of a plurality of feature regions of the object in the shape data and point cloud data of a plurality of feature regions of the object in the CAD data to calculate the positions and orientations of the plurality of feature regions in the coordinate system of the robot arm 310. In this case, the first matching unit 201 may output the calculated positions and orientations of the plurality of feature regions of the object as the position and orientation of the object, or may calculate (estimate) the position and orientation of the center of gravity of the object based on the positions and orientations of the plurality of feature regions and output them as the position and orientation of the object.

[0106] The first matching unit 201 may perform matching processing using a depth image as shape data output from the detection device 320, instead of using point cloud data. The feature region of the object for which the position and orientation are calculated is not limited to a part of the corners of the object, but may be any region in which a difference in shape from the surroundings can be identified, such as an edge or unevenness of the object. For example, the feature region of the circuit board T as the object may be a marker (e.g., a code or cross mark having a three-dimensional shape) or a solder pad or solder formed on the circuit board T as at least a part of the circuit board T, a marker (e.g., a code or cross mark having a three-dimensional shape) arranged on the circuit board T, or an element arranged on the circuit board T. The data that the first matching unit 201 performs matching processing with the feature region in the image data output from the detection device 320 is not limited to CAD data, and other data may be used. For example, the first matching unit 201 may perform a matching process between point cloud data of a feature region in shape data (e.g., point cloud data) generated in advance by imaging a reference object (e.g., a reference circuit board) and point cloud data of a feature region in the shape data of a circuit board T output from the detection device 320. In this case, the reference object may be, for example, a non-defective object (e.g., a non-defective circuit board). Note that both the CAD data and the image data and shape data obtained by imaging the reference object are reference data in the matching process, and therefore may be referred to as reference data.

[0107] In addition, the matching rate is a value indicating the degree of agreement between the point cloud data of the characteristic area of ​​the object in the shape data and the point cloud data of the characteristic area of ​​the object in the CAD data (here, the higher the degree of agreement between the two, the higher the matching rate).

[0108] Similarly, the second matching unit 202 matches the output of the detection device 330 (for example, at least one of image data and shape data) with the CAD data. The second matching unit 202 outputs the position and orientation estimation result of the object and a matching rate as a result of the matching. Note that the matching process in the second matching unit 202 (i.e., estimating the position and orientation of the object and calculating the matching rate) is similar to that of the above-mentioned first matching unit 201, and therefore a description thereof will be omitted.

[0109] That is, each of the first matching units 201 and 202 calculates (estimates) the position and orientation of the object, and outputs the calculation result as a position and orientation estimation result. Here, for example, in order to bring the light irradiation device 60 of the robot 3 closer to a place on the circuit board T where the processing light L is to be irradiated, the object whose position and orientation are calculated by each of the first matching unit 201 and the second matching unit 202 may be the place itself where the processing light L is to be irradiated, for example, at least a part of the circuit board T (for example, a solder pad formed on the circuit board T), or an element or solder arranged on the circuit board T. In addition, the object may be a place whose relative position to the place where the processing light L is to be irradiated is known, for example, at least a part of the circuit board T (for example, a two-dimensional code or a cross mark such as an AR marker as a marker formed on the circuit board T), a marker arranged on the circuit board T (for example, a two-dimensional code or a cross mark such as an AR marker), or a marker arranged in the vicinity of the circuit board T (for example, a two-dimensional code or a cross mark such as an AR marker). Furthermore, the target object may be a portion other than the above-mentioned portions on the circuit board T. That is, the target object whose position and orientation are calculated by each of the first matching units 201 and 202 may be the above-mentioned characteristic region.

[0110] It is assumed that the CAD data and data of the reference object used for matching (image data and shape data) are stored in advance in, for example, a memory (not shown) of the control device 1000. The first matching unit 201 and the second matching unit 202 may calculate (estimate) both the position and orientation of the object, or may calculate (estimate) only one of the position and orientation of the object. In other words, the first matching unit 201 and the second matching unit 202 may calculate (estimate) at least one of the position and orientation of the object.

[0111] The comparison unit 203 compares the matching rate output from the first matching unit 201 (hereinafter referred to as the "first matching rate" as appropriate) with the matching rate output from the second matching unit 202 (hereinafter referred to as the "second matching rate" as appropriate).

[0112] If the first matching rate is greater than the second matching rate, in other words, if the second matching rate is less than the first matching rate (first matching rate>second matching rate), the comparison unit 203 outputs the position and orientation estimation result output from the first matching unit 201. On the other hand, if the second matching rate is equal to or greater than the first matching rate, in other words, if the first matching rate is equal to or less than the second matching rate (first matching rate≦second matching rate), the comparison unit 203 outputs the position and orientation estimation result output from the second matching unit 202.

[0113] The robot control unit 100 as a part of the control device 1000 controls, for example, the driving unit 311 of the robot 3 based on the position and orientation estimation result output from the matching processing unit 200.

[0114] The comparison unit 203 may compare the above-mentioned matching rates for all matching process results (i.e., calculation results of the position and orientation of the target object) output from each of the first matching unit 201 and the second matching unit 202 at a predetermined time interval. The comparison unit 203 may not compare the above-mentioned matching rates for all matching process results output from each of the first matching unit 201 and the second matching unit 202 at a predetermined time interval. In this case, the comparison unit 203 may compare the above-mentioned matching rates at a predetermined time point, and after the predetermined time point, output the results of the matching process output from the first matching unit 201 or the second matching unit 202 at predetermined time intervals based on the comparison result of the matching rates executed at the predetermined time point.

[0115] The control device 1000 may output at least one of the results of the matching process using image data and the results of the matching process using shape data from at least one of the first matching unit 201 and the second matching unit 202 via the comparison unit 203.

[0116] Incidentally, as described above, each of the detection devices 320 and 330 of the robot 3 can output at least one of image data and shape data. Therefore, each of the first matching unit 201 and the second matching unit 202 may perform at least one of a matching process using image data (i.e., data showing a two-dimensional image) (hereinafter, appropriately referred to as "2D matching") and a matching process using shape data (e.g., three-dimensional point cloud data) (hereinafter, appropriately referred to as "3D matching"). 2D matching can calculate the position and orientation of an object faster than 3D matching. On the other hand, 3D matching can calculate (estimate) the position and orientation of an object with higher accuracy than 2D matching. Therefore, each of the first matching unit 201 and the second matching unit 202 may perform 2D matching or 3D matching depending on the purpose.

[0117] When performing 3D matching, if all of the shape data (for example, all of the generated point cloud data) is used, the 3D matching takes a relatively long time. Therefore, each of the first matching unit 201 and the second matching unit 202 may perform the following process to shorten the time required for 3D matching.

[0118] First, each of the first matching unit 201 and the second matching unit 202 specifies the position of the object obj in the two-dimensional image shown in FIG. 10(a) from the result of the 2D matching. Next, each of the first matching unit 201 and the second matching unit 202 determines the range A (see FIG. 10(b)) in which 3D matching should be performed based on the specified position of the object obj (in other words, narrows down the range in which 3D matching should be performed). Note that the object obj whose position is specified by the 2D matching of each of the first matching unit 201 and the second matching unit 202 may be the characteristic region of the object described above. Then, each of the first matching unit 201 and the second matching unit 202 performs 3D matching using shape data corresponding to the determined range A (for example, point cloud data included in the range A). Therefore, each of the first matching unit 201 and the second matching unit 202 performs 3D matching using the minimum necessary point cloud data, and therefore the time required for extraction processing of point cloud data of the characteristic region in particular can be shortened compared to the conventional method, and therefore the 3D matching can be accelerated. Furthermore, 3D matching makes it possible to calculate (estimate) the position and orientation of an object (feature region) with high accuracy.

[0119] As an example, this process will be described with reference to the timing chart of Fig. 11. In Fig. 11, the width of the black band along the time axis indicates the length of time required for each process.

[0120] In FIG. 11, T 2-1 Using the result of the 2D matching, the area to be 3D matched is determined (see “Area crop” in Fig. 11). Then, 3-1 The th 3D matching is performed. 3-1 In parallel with the 3D matching,2-2 th and T 2-3 The 2D matching is performed. 3-1 When the 3D matching is completed, T 2-4 The result of the 2D matching is used to determine the range in which 3D matching should be performed, and then 3-2 The 3D matching is performed for the th time. In this case, the above-mentioned comparison unit 203 (see FIG. 9) sequentially compares the 3D matching results by the first matching unit 201 and the second matching unit 202, and outputs a position and orientation estimation result with a high matching rate to the robot control unit 100 at predetermined time intervals. The robot control unit 100 controls, for example, the driving unit 311 of the robot 3 based on the position and orientation estimation result output from the matching processing unit 200 at predetermined time intervals.

[0121] In addition, the first matching unit 201 and the second matching unit 202 each use T that is not used for 3D matching. 2-2 th and T 2-3 It is not necessary to perform the first 2D matching. The comparison unit 203 may compare the results of the 2D matching and output a position and orientation estimation result with a high matching rate to the robot control unit 100. The timing chart is not limited to that of Fig. 11, and each of the first matching unit 201 and the second matching unit 202 may perform 2D matching and 3D matching at a predetermined timing (for example, timing at which a tracking process described later starts, etc.), and may determine a range in which 3D matching should be performed using the result of the 2D matching as described above, and perform 3D matching.

[0122] The above-mentioned matching process is an example, and is not limited thereto. The control device 1000 may calculate (estimate) the position and orientation of the object (characteristic region) only by 3D matching (i.e., matching using shape data), or may calculate (estimate) the position and orientation of the object (characteristic region) only by 2D matching (i.e., matching using image data). When calculating (estimating) the position and orientation of the object only by 2D matching, the detection devices 320 and 330 may include only a single camera. In addition, when either one of the detection devices 320 and 330 is provided on the robot 3, the matching processing unit 200 may be configured with only either one of the first matching unit 201 and the second matching unit 202.

[0123] (Calculation of object position and orientation by tracking process) Due to the transport of the circuit board T by the belt conveyor and / or due to the movement of the light irradiation device 60 by, for example, the robot arm 310 of the robot 3, the relative position between the object and the light irradiation device 60 changes over time.

[0124] It is necessary to move the light irradiation device 60 of the robot 3 closer to a part of the circuit board T as an object, using the robot arm 310, so that the position and posture of the light irradiation device 60 of the robot 3 relative to a part of the circuit board T as an object transported by a belt conveyer (for example, a part to be irradiated with the processing light L) becomes a desired position and posture. In order to move the light irradiation device 60 closer to the object using the robot arm 310, the control device 1000 can perform a tracking process to calculate (estimate) a change in the position and posture of the object. Note that the object for which the change in position and posture is calculated (estimated) may be the part itself to be irradiated with the processing light L, as described above, or may be, for example, at least a part of the circuit board T (for example, a solder pad formed on the circuit board T) or an element or solder arranged on the circuit board T. The target object may be a location whose relative position to the location to be irradiated with the above-mentioned processing light L is known, and may be, for example, at least a part of the circuit board T (for example, a two-dimensional code or a cross mark such as an AR marker as a marker formed on the circuit board T), a marker arranged on the circuit board T (for example, a two-dimensional code or a cross mark such as an AR marker), or a marker arranged in the vicinity of the circuit board T (for example, a two-dimensional code or a cross mark such as an AR marker). The target object may be a location on the circuit board T other than the above-mentioned locations.

[0125] The tracking process performed in the control device 1000 will be described with reference to FIGS.

[0126] 12, the control device 1000 has a tracking unit 300 as a processing block logically realized therein or as a processing circuit physically realized therein. The tracking unit 300 is configured to include a matching unit 301, a 2D tracking unit 302, and a 3D tracking unit 303.

[0127] Image data and shape data are input to the tracking unit 300 at predetermined time intervals. Here, when the position and orientation estimation result output from the first matching unit 201 of the matching processing unit 200 described above is output to the robot control unit 100 (i.e., when the first matching rate>the second matching rate), image data and shape data output from the detection device 320 of the robot 3 at predetermined time intervals are input to the tracking unit 300. On the other hand, when the position and orientation estimation result output from the second matching unit 202 of the matching processing unit 200 described above is output to the robot control unit 100 (i.e., when the first matching rate≦the second matching rate), image data and shape data output from the detection device 330 of the robot 3 at predetermined time intervals are input to the tracking unit 300.

[0128] The matching unit 301 calculates (estimates) the position and orientation of an object by performing a matching process from, for example, image data and shape data input at a predetermined time interval. The matching process in the matching unit 301 is similar to that in the first matching unit 201 and the second matching unit 202 described above, and therefore a description thereof will be omitted. Note that, as described above, the matching unit 301 may narrow down the range in which 3D matching should be performed based on the result of 2D matching using the input image data, and calculate (estimate) the position and orientation of the object by performing 3D matching using shape data corresponding to that range (see FIG. 10).

[0129] Image data is input to the 2D tracking unit 302 at a predetermined time interval. The 2D tracking unit 302 calculates (estimates) the displacement of the object at a predetermined time interval based on, for example, two pieces of image data, the first image data and the second image data, input at the predetermined time interval. Meanwhile, at least the first image data input to the 2D tracking unit 302 is also input to the matching unit 301. In other words, the image data input to the tracking unit 300 is input to the 2D tracking unit 302 and the matching unit 301 at approximately the same time. The matching unit 301 calculates (estimates) the position and orientation of the object by performing a matching process as described above using the input first image data.

[0130] The 2D tracking unit 302 calculates (estimates) the position and orientation of the object at a predetermined time (i.e., the time when the second image data is generated) by adding the calculated (estimated) displacement of the object to the position and orientation of the object calculated (estimated) by the matching unit 301 at the very beginning (hereinafter, also referred to as the initial position and orientation). The 2D tracking unit 302 then calculates (estimates) the displacement of the object at a predetermined time interval based on each image data input at a predetermined time interval, and calculates (estimates) the position and orientation of the object at each time by performing a process of adding the calculated (estimated) displacement of the object to the calculated (estimated) position and orientation of the object each time (i.e., performing a tracking process). Note that the position and orientation of the object calculated (estimated) at each time by the tracking process are also values ​​expressed in 6DoF, like the position and orientation calculated by the matching process described above. More specifically, the 2D tracking unit 302 first extracts a feature region of the object in the first image data (e.g., a part of the contour of the object in the first image data) and a feature region of the object in the second image data (e.g., a part of the contour of the object in the second image data) at each input timing. The 2D tracking unit 302 then associates the feature region of the object in the first image data with the feature region of the object in the second image data, and obtains a displacement in the camera coordinate system of the feature region of the object in the second image data relative to the feature region of the object in the first image data. The 2D tracking unit 302 further calculates a displacement in the camera coordinate system of the robot arm 310 between the object at the time the first image data was generated and the object at the time the second image data was generated, based on the obtained displacement in the camera coordinate system of the feature region of the object in the second image data relative to the feature region of the object in the first image data, and a previously obtained correspondence between the camera coordinate system and the coordinate system of the robot arm 310.

[0131] Then, the 2D tracking unit 302 calculates the position and orientation of the object at a predetermined time (i.e., the time when the second image data is generated) by adding the above-mentioned displacement to the position and orientation (initial position and orientation) of the characteristic region of the object calculated by the matching unit 301. Since the time required for the 2D tracking unit 302 to calculate the position and orientation of the object is shorter than the time required for the 3D tracking unit 303 to calculate the position and orientation of the object, which will be described later, the position and orientation of the object can be tracked at high speed. Note that various existing methods can be used for extracting the characteristic region of the object in the first image data and the second image data, associating the characteristic region of the object in the first image data with the characteristic region of the object in the second image data, and calculating the displacement of the characteristic region of the object in the coordinate system of the robot arm 310.

[0132] Various existing methods can be used to extract the feature region of the object in the first image data or the second image data, to associate the feature region of the object in the first image data with the feature region of the object in the second image data, and to calculate the displacement of the feature region of the object in the second image data relative to the feature region of the object in the first image data in the global coordinate system. The 2D tracking unit 302 may extract a plurality of feature regions of the object in the first image data and a plurality of feature regions of the object in the second image data, to associate the respective feature regions with each other, and to calculate (estimate) the displacement of each feature region of the object in the second image data relative to each feature region of the object in the first image data. The 2D tracking unit 302 may use the position and orientation estimation result of the object output from the comparison unit 203 (i.e., the position and orientation estimation result with a high matching rate among the 3D matching results output from the first matching unit 201 and the second matching unit 202, respectively, as referred to in FIG. 9 and FIG. 10) as the initial position and orientation. In addition, the characteristic area of ​​the circuit board T as the target object may be, for example, a marker (e.g., a two-dimensional code such as an AR marker or a cross mark), a solder pad, or solder formed on the circuit board T as at least a part of the circuit board T, a marker (e.g., a two-dimensional code such as an AR marker or a cross mark) arranged on the circuit board T, an element arranged on the circuit board T, or a marker (e.g., a two-dimensional code such as an AR marker or a cross mark) arranged in the vicinity of the circuit board T.

[0133] Shape data is input to the 3D tracking unit 303 at a predetermined time interval. The 3D tracking unit 303 calculates (estimates) the displacement of the object at a predetermined time interval based on, for example, two pieces of shape data, the first shape data and the second shape data, input at a predetermined time interval. Meanwhile, at least the first shape data input to the 3D tracking unit 303 is also input to the matching unit 301. In other words, the shape data input to the tracking unit 300 is input to the 3D tracking unit 303 and the matching unit 301 at approximately the same time. The matching unit 301 calculates (estimates) the position and orientation of the object by performing a matching process as described above using the input first shape data.

[0134] The 3D tracking unit 303 calculates the three-dimensional displacement of the object over a short period of time. The 3D tracking unit 303 adds the calculated displacement to the initial posture and position detected by the matching unit 301 to estimate the current posture and position of the object.

[0135] The 3D tracking unit 303 calculates the position and orientation of the object at a predetermined time (i.e., the time when the second shape data is generated) by adding the calculated displacement of the object to the position and orientation of the object calculated by the matching unit 301 at the very beginning (initial position and orientation). The 3D tracking unit 303 then calculates the displacement of the object at a predetermined time interval based on each piece of shape data input at a predetermined time interval, and calculates the position and orientation of the object at each time by performing a process of adding the calculated displacement of the object to the calculated position and orientation of the object each time (i.e., performing a tracking process). More specifically, for example, when the shape data is point cloud data, the 3D tracking unit 303 first extracts point cloud data of a characteristic region of the object in the first shape data described above (for example, a part of a corner of the object in the first shape data) and point cloud data of a characteristic region of the object in the second shape data described above (for example, a part of a corner of the object in the second shape data) at each input timing. The 3D tracking unit 303 then associates the point cloud data of the feature region of the object in the first shape data with the point cloud data of the feature region of the object in the second shape data, and obtains the displacement in the camera coordinate system of the point cloud data of the feature region of the object in the second shape data relative to the point cloud data of the feature region of the object in the first shape data. The 3D tracking unit 303 further calculates the displacement in the camera coordinate system of the object at the time the first shape data was generated and the displacement in the coordinate system of the robot arm 310 of the object at the time the second shape data was generated, based on the displacement in the camera coordinate system of the point cloud data of the feature region of the object in the second shape data relative to the point cloud data of the feature region of the object in the first shape data obtained, and the correspondence between the camera coordinate system and the coordinate system of the robot arm 310 obtained in advance.

[0136] Then, the 3D tracking unit 303 calculates the position and orientation of the object at a predetermined time (i.e., the time when the second shape data is generated) by adding the above-mentioned displacement to the position and orientation (initial position and orientation) of the point cloud data of the feature region of the object calculated by the matching unit 301. Note that when the shape data is point cloud data, the tracking process by the 3D tracking unit 303 can use various existing methods such as RANSAC (Random Sample Consensus), SIFT (Scale-Invariant Feature Transform), and ICP (Iterative Closest Point).

[0137] Also, for example, when the shape data is depth image data, the 3D tracking unit 303 first extracts a feature area of ​​the object in the first shape data (for example, a part of a corner of the object in the first shape data) and a feature area of ​​the object in the second shape data (for example, a part of a corner of the object in the second shape data) at each input timing. The 3D tracking unit 303 then associates the feature area of ​​the object in the first shape data with the feature area of ​​the object in the second shape data, and calculates the displacement of the feature area of ​​the object in the second shape data relative to the feature area of ​​the object in the first shape data in the camera coordinate system. The 3D tracking unit 303 further calculates the displacement of the object in the second shape data relative to the feature area of ​​the object in the first shape data in the camera coordinate system and the displacement of the object in the robot arm 310 at the time the first shape data was generated and the time the second shape data was generated based on the displacement of the object in the second shape data relative to the feature area of ​​the object in the first shape data obtained in the camera coordinate system and the correspondence between the camera coordinate system and the coordinate system of the robot arm 310 obtained in advance. Then, the 3D tracking unit 303 calculates the position and orientation of the object at a predetermined time (i.e., the time when the second shape data is generated) by adding the above-mentioned displacement to the position and orientation (initial position and orientation) of the feature region of the object calculated by the matching unit 301. When the shape data is depth image data, the tracking process by the 3D tracking unit 303 can use various existing methods such as DSO (Direct Sparse Odometry). Since the accuracy of the position and orientation of the object calculated by the 3D tracking unit 303 is higher than the accuracy of the position and orientation of the object calculated by the above-mentioned 2D tracking unit 302, the position and orientation of the object can be tracked with high accuracy.

[0138] In addition, the position and orientation of the object estimated by the 2D tracking unit 302, the position and orientation of the object estimated by the 3D tracking unit, and the position and orientation of the object detected by the matching unit 301 may each be output in a format corresponding to the position and orientation estimation results output from the first matching unit 201 and the second matching unit 202 described above (i.e., 6DoF format).

[0139] Here, an example of the tracking process will be described with reference to the timing chart of Fig. 13. In Fig. 13, the width of the black band in the time axis direction indicates the length of time required for each process.

[0140] 13, it is assumed that the initial position and orientation (initial position and orientation) of the object has already been detected by the matching unit 301, and that the current position and orientation of the object is estimated by adding the displacement of the object at a predetermined time interval to the initial position and orientation. It is also assumed that the accuracy of the position and orientation estimation by the 3D tracking unit 303 and the accuracy of the position and orientation estimation by the matching unit 301 are higher than the accuracy of the position and orientation estimation by the 2D tracking unit 301.

[0141] For example, the position and orientation of the object at time t1 (i.e., the position and orientation estimated by the matching unit 301) is denoted by x1. For example, the displacement of the object from time t1 to time t2 obtained by the 2D tracking unit 302 is denoted by Δx 12 , the displacement of the object from time t2 to time t3 is Δx 23 In this case, the position and orientation of the object at time t3 estimated by the 2D tracking unit 302 is “x1+Δx 12 +Δx 23 " is expressed as

[0142] Displacement “Δx 12 " and "Δx 23 Since there is an estimation error in each of "x1" and "x2", each time the displacement detected by the 2D tracking unit 302 is accumulated and added to the position and orientation "x1", the error in the position and orientation of the object estimated by the 2D tracking unit 302 becomes larger.On the other hand, the time required for processing in the 2D tracking unit 302 is shorter than the time required for processing in the matching unit 301 and the 3D tracking unit 303.

[0143] Therefore, it is possible to achieve both faster tracking processing and improved estimation accuracy by correcting the position and orientation of the object estimated by the 2D tracking unit 302 using at least one of the position and orientation of the object estimated by the matching unit 301 and the position and orientation of the object estimated by the 3D tracking unit 303.

[0144] Specifically, the T of the 2D tracking unit 302 2-1 The 3D tracking unit 303 uses the image data used in the first processing and the shape data at the same time. 3-1 The position and orientation of the object estimated by the first processing and the above T 2-1 The difference between the position and orientation of the object estimated by the first processing and the position and orientation of the object estimated by the second processing is regarded as an error of the position and orientation of the object estimated by the 2D tracking unit 302. 3-1 When the process of the th process is completed, the process (for example, T 2-2 The error may be corrected in the first processing step (the first processing step) or in the processing step subsequent to the first processing step.

[0145] Similarly, the matching unit 301 i-1 In the processing of the th time, T 2-1 When the position and orientation of the object are calculated (estimated) from the image data used in the processing of the T i-1 The position and orientation calculated (estimated) by the th processing and the above T 2-1 The difference between the position and orientation of the object estimated by the first processing and the position and orientation of the object estimated by the second processing is regarded as an error of the position and orientation of the object estimated by the 2D tracking unit 302. i-1 When the process of the Tth process is completed, the process in progress in the 2D tracking unit 302 or the process to be performed next to the process (for example, 2-9 In the second processing, the error may be corrected.

[0146] In addition, the matching unit 301 i-1 In the processing of the th time, the T 3-1When the position and orientation of the object are calculated (estimated) from the shape data used in the processing of the T i-1 The position and orientation detected by the first processing and the above T 3-1 The difference between the position and orientation of the object estimated by the first processing and the position and orientation of the object estimated by the 3D tracking unit 303 is regarded as an error in the position and orientation of the object estimated by the 3D tracking unit 303. i-1 The error may be corrected in the process being performed in the 3D tracking unit 303 when the th process is completed or in the process that follows the process.

[0147] The position and orientation of the object estimated by the 2D tracking unit 302 are output to the robot control unit 100. The robot control unit 100 as a part of the control device 1000 controls, for example, the drive unit 311 of the robot 3 based on the estimated position and orientation of the object.

[0148] It is to be noted that the tracking unit 300 may receive either the image data or the shape data, but not the other. The tracking unit 300 may include either the 2D tracking unit 302 or the 3D tracking unit 303, but may not include the other. In this case, either the position and orientation of the object estimated by the 2D tracking unit 302 or the position and orientation of the object estimated by the 3D tracking unit 303 may be output to the robot control unit 100. The position and orientation of the object estimated by the 2D tracking unit 302 may not be corrected using the position and orientation of the object estimated by the 3D tracking unit 303. The position and orientation of the object estimated by the 3D tracking unit 303 may be corrected using the position and orientation of the object estimated by the 2D tracking unit 302. The tracking unit 300 may select data to be used in the tracking process, regardless of the result of the comparison of the matching rates by the matching processing unit 200. In this case, the matching unit 301 of the tracking unit 300 may compare the matching rate when the position and orientation of the target object is calculated (estimated) using, for example, image data and shape data output from the detection device 320 with the matching rate when the position and orientation of the target object is calculated (estimated) using the image data and shape data output from the detection device 330, and may select (in other words, switch) the data to be used for the tracking process based on the result of the comparison.

[0149] Alternatively, the tracking unit 300 may always perform the tracking process using the image data and shape output from the detection device 330. Alternatively, the tracking unit 300 may always perform the tracking process using the image data and shape output from the detection device 320. That is, the above-mentioned tracking process can be performed using image data and shape data output from only one of the detection devices 320 and 330. In this case, for example, the robot 3 may have only one of the detection devices 320 and 330. For example, when the robot 3 has only one of the detection devices 320 and 330, the above-mentioned tracking process may be performed using only the image data output from the one detection device, or may be performed using only the shape data output from the one detection device.

[0150] The above-mentioned tracking process is an example, and is not limited thereto. That is, the control device 1000 may perform a known tracking process instead of the above-mentioned tracking process. Also, the control device 1000 may not perform the above-mentioned tracking process. In this case, the control device 1000 may not include the above-mentioned tracking unit 300, but may include the above-mentioned matching processing unit 200. The control device 1000 may perform a matching process using at least one of image data and shape data at a predetermined time interval, and may control, for example, the driving unit 311 of the robot 3 based on the calculated (estimated) position and posture of the target object.

[0151] (Movement of each robot) The operations of the robots 1, 2, and 3 will be described with reference to the flowchart in Fig. 14. Here, the robot 3 will be mainly described, and the description of the robots 1 and 2 common to the robot 3 will be omitted as appropriate.

[0152] Here, the belt conveyor is temporarily stopped after transporting the circuit board T within, for example, the driving range of the robot arm 110 of the robot 1. Then, after the solder discharged from the dispenser 40 is placed on a predetermined portion of the stopped circuit board T, the belt conveyor resumes driving to transport the circuit board T. Also, the belt conveyor is temporarily stopped after transporting the circuit board T within, for example, the driving range of the robot arm 210 of the robot 2. Then, after the element held by the holding device 50 is placed on a predetermined portion of the stopped circuit board T, the belt conveyor resumes driving to transport the circuit board T. Also, the belt conveyor is temporarily stopped after transporting the circuit board T within, for example, the driving range of the robot arm 310 of the robot 3. Then, after the processing light L from the light irradiation device 60 is irradiated on a predetermined portion of the stopped circuit board T, the belt conveyor resumes driving to transport the circuit board T.

[0153] Robot 3 First, the control device 1000 may perform calibration of the light irradiation device 60 before performing the processes of steps S131 to S138 below.

[0154] It is assumed that the detection device 330 and the light irradiation device 60 are provided on the robot arm 310 in a positional relationship such that a part of the light irradiation device 60 (e.g., the tip) is within the field of view of each of the cameras 31 and 32 of the detection device 330.

[0155] The control device 1000 performs the above-mentioned matching process using the shape data including a part of the light irradiation device 60 output from the detection device 330 and the CAD data of the light irradiation device 60 as a calibration of the light irradiation device 60, and calculates in advance the position and orientation of the light irradiation device 60 (for example, the position and orientation of the tip of the light irradiation device 60 included in the field of view of the cameras 31 and 32 of the detection device 330). That is, the control device 1000 calculates in advance the position and orientation of the light irradiation device 60 in the coordinate system of the robot arm 310 based on at least a part of the shape data of the light irradiation device 60.

[0156] A marker may be provided in a part of the light irradiation device 60 included in the field of view of each of the cameras 31 and 32 of the detection device 330. In this case, the control device 1000 may perform the above-mentioned calibration based on, for example, shape data including the marker output from the detection device 330.

[0157] The control device 1000 may calibrate the light irradiation device 60 by executing a matching process using not only the shape data but also the image data output from the detection device 330 and the CAD data of the light irradiation device 60. As described above, the control device 1000 may use not only the CAD data but also the shape data and image data of the light irradiation device 60 acquired in advance in the matching process.

[0158] Note that the calibration of the light irradiation device 60 is not limited to using the detection device 330, and the control device 1000 may use shape data or image data output from the detection device 320. In this case, it is assumed that the detection device 320 and the light irradiation device 60 are provided on the robot arm 310 in a positional relationship such that a part of the light irradiation device 60 is within the field of view of each of the cameras 21 and 22 of the detection device 320.

[0159] In the process of steps S131 to S138 below, the position and orientation of the light irradiation device 60 with respect to the detection device 330 may change due to the light irradiation device 60 coming into contact with a predetermined object, etc. In this case, the control device 1000 can detect the change in the position and orientation of the light irradiation device 60 with respect to the detection device 330 based on a partial change in the light irradiation device 60 in the image data or shape data output from the detection device 330 (for example, a partial change in the light irradiation device 60 on the image). When a change in the position and orientation of the light irradiation device 60 with respect to the detection device 330 is detected, the control device 1000 may execute calibration.

[0160] The control device 1000 that controls the robot 3 calculates (estimates) the position and posture of the circuit board T as an example of the target object (step S131). In the process of step S131, the control device 1000 calculates the initial position and posture (i.e., initial position and posture) of the circuit board T by the matching process of the matching unit 301. Furthermore, the control device 1000 calculates the position of each solder pad formed on the circuit board T. The control device 1000 calculates the position and posture of each solder pad on the circuit board T based on the Gerber data of the circuit board T (i.e., design data of the circuit board T). More specifically, the control device 1000 calculates the position and posture of each solder pad on the circuit board T from the calculated initial position and posture of the circuit board T and the positional relationship of each solder pad on the circuit board T. Furthermore, the Gerber data of the circuit board T includes data on the order in which elements are mounted on each solder pad, and the control device 1000 specifies the order in which elements are mounted on each solder pad based on the Gerber data. The control device 1000 does not need to calculate the initial position and orientation of the circuit board T, and may calculate the position and orientation of any object that can be used in the above-mentioned matching process, such as a cross mark formed on the circuit board T or an AR marker arranged near the circuit board T or the circuit layout T. The control device 1000 does not need to specify the position and orientation of each solder pad or the mounting order based on the Gerber data, and may specify them using other design data (e.g., CAD data) of the circuit board T, or may specify them using information input by a user via an interface (not shown). The control device 1000 does not need to calculate the position and orientation of each solder pad on the circuit board T, and may calculate the position and orientation of a location to be irradiated with the processing light L or the vicinity of that location, such as the element itself, an area where the element is arranged, or an area where the solder is arranged.

[0161] Next, the control device 1000 controls the driving unit 311 so that the detection devices 320 and 330 (and further the light irradiation device 60) approach the circuit board T, and moves the robot arm 310 (step S132). In the process of step S132, the control device 1000 controls the driving unit 311 of the robot arm 310 so that an element (e.g., a chip LED having two electrodes) arranged on the solder pad to be mounted first comes into the field of view of at least one of the detection devices 320 and 330. More specifically, the control device 1000 uses information on the initial position and orientation of the circuit board T calculated (estimated) by the process of step S131 and the position and orientation of the solder pad to be mounted first, and controls the driving unit 311 to move the robot arm 310 based on the position and orientation of the circuit board T output from the 2D tracking unit 302 of the tracking unit 300 at predetermined time intervals.

[0162] Next, the control device 1000 judges whether the element placed on the solder pad to be mounted first is within the field of view of at least one of the detection devices 320 and 330 (step S133). In the process of S133, the control device 1000 judges whether the detection devices 320 and 330 are in the desired position and posture with respect to the solder pad to be mounted first, based on the information on the position and posture of the circuit board T output from the 2D tracking unit 302 at predetermined time intervals and the information on the position and posture of the solder pad to be mounted first calculated in step S131. At this time, when the detection devices 320 and 330 are in the desired position and posture with respect to the solder pad to be mounted first, the control device 1000 judges that the element placed on the solder pad is within the field of view of at least one of the detection devices 320 and 330. In addition, without being limited to the above determination method, the control device 1000 may determine whether or not the position and posture information output from the 2D tracking unit 302 at a predetermined time interval includes position and posture information of the element placed on the first solder pad, or may determine whether or not at least one of the image data and shape data generated by at least one of the detection devices 320 and 330 includes information regarding the element.

[0163] In the process of step S133, if it is determined that the element placed on the solder pad to be mounted first is not within the field of view of at least one of the detection devices 320 and 330 (step S133: No), the control device 1000 controls the drive unit 311 based on the position and attitude information of the solder pad to be mounted first calculated (estimated) in the process of step S131 and the position and attitude information of the circuit board T output at predetermined time intervals from the 2D tracking unit 302 of the tracking unit 300 to continue moving the robot arm 310. In other words, the process of step S132 is performed until it is determined that the element placed on the solder pad to be mounted first is within the field of view of at least one of the detection devices 320 and 330.

[0164] On the other hand, in the process of step S133, if it is determined that the element placed on the solder pad to be mounted first has entered the field of view of at least one of the detection devices 320 and 330 (step S133: Yes), the control device 1000 calculates (estimates) the position and orientation of the element placed on the solder pad to be mounted first (step S134). In the process of step S134, the control device 1000 calculates (estimates) the very initial position and orientation (initial position and orientation) of the element placed on the solder pad to be mounted first by the matching process of the matching unit 301. Note that the control device 1000 does not need to calculate (estimate) the initial position and orientation of the element, and may calculate (estimate) the position and orientation of any object that can be used in the above-mentioned matching process, such as a solder pad or a cross mark formed on the circuit board T, or an AR marker or solder placed on the circuit board T.

[0165] Next, the control device 1000 controls the driving unit 311 to move the robot arm 310 so that the position and posture of the light irradiation device 60 are at a desired position and posture that allows the processing light L to melt the solder arranged on the first solder pad (step S135). In the process of step S135, the control device 1000 uses the information on the initial position and posture of the element calculated (estimated) by the process of step S134 to control the driving unit 311 to move the robot arm 310 based on the position and posture of the element output from the 2D tracking unit 302 of the tracking unit 300 at predetermined time intervals. In other words, the control device 1000 controls the driving unit 311 to move the robot arm 310 so that the light irradiation device 60 (detection devices 320 and 330) approaches the element arranged on the first solder pad of the circuit board T.

[0166] Next, the control device 1000 judges whether or not the position and posture of the light irradiation device 60 are the desired position and posture that allows the processing light L to melt the solder placed on the first solder pad (step S136). In the process of step S136, the control device 1000 judges whether or not the position and posture of the light irradiation device 60 relative to the element are the desired position and posture based on, for example, information on the position and posture of the element output from the 2D tracking unit 302 at predetermined time intervals. At this time, if the position and posture of the light irradiation device 60 relative to the element are the desired position and posture, the control device 1000 judges that the position and posture of the light irradiation device 60 are the desired position and posture that allows the processing light L to melt the solder placed on the first solder pad.

[0167] In addition, in the process of step S136, if it is determined that the position and posture of the light irradiation device 60 are not the desired position and posture at which the solder placed on the first solder pad can be melted by the processing light L (step S136: No), the control device 1000 continues to move the robot arm 310 by controlling the driving unit 311 based on the position and posture of the element placed on the first solder pad outputted at predetermined time intervals from the 2D tracking unit 302 so that the light irradiation device 60 approaches the element. In other words, the process of step S135 is performed until it is determined that the position and posture are the desired position and posture at which the solder placed on the first solder pad can be melted by the processing light L.

[0168] On the other hand, if it is determined that the position and posture of the light irradiation device 60 are the desired position and posture that allows the processing light L to melt the solder placed on the first solder pad (step S136: Yes), the control device 1000 controls the light irradiation device 60 to irradiate the processing light L to the electrodes of the element placed on the solder pad to be mounted first, that is, the two electrodes of the chip LED, so as to melt the solder placed on the solder pad to be mounted first (step S137). As a result, the solder placed on the solder pad melts, and the element is soldered to the circuit board T (the solder pad to be mounted first). Here, the following two modes can be given as specific examples of the process of step S137.

[0169] That is, the first mode is a mode in which the processing light L is irradiated onto the electrodes of the chip LED when the robot arm 310 is driven by the driving unit 311, that is, when the light irradiation device 60 and the detection devices 320 and 330 are moved by the robot arm 310, as shown in, for example, Figures 15(a) and (b). In this mode, the control device 1000 may control the direction of the galvanometer mirror 61 of the light irradiation device 60 so that the irradiation position of the processing light L from the light irradiation device 60 moved by the robot arm 310 is maintained on one electrode of the chip LED (for example, so that the processing light L from the light irradiation device 60 is irradiated to the same position of one electrode of the chip LED) while controlling the driving unit 311 so that the light irradiation device 60 and the detection devices 320 and 330 are moved. For example, the control device 1000 may control the orientation of the galvanometer mirror 61 of the light irradiation device 60 so that the irradiation position of the processing light L from the light irradiation device 60 moved by the robot arm 310 is maintained at the same position of one electrode of the chip LED (in other words, so that the processing light L from the light irradiation device 60 continues to be irradiated to the same position of one electrode of the chip LED for a predetermined time) while controlling the drive unit 311 so that the light irradiation device 60 and the detection devices 320 and 330 are moved. Note that the same position is a concept that also includes extremely minute fluctuations in the irradiation position of the processing light L at the location where the processing light L should be irradiated to an extent that does not affect the melting of the solder.

[0170] For example, as shown in Figures 15(a) and (b), when a light irradiation device 60 or the like is moved from right to left on the paper by a robot arm 310, the control device 1000 may control the orientation of the galvanometer mirror 61 of the light irradiation device 60 so that the irradiation position of the processing light L from the moved light irradiation device 60 is maintained on one of the electrodes of the chip LED.

[0171] In the above first aspect, for example, the control device 1000 may control the orientation of the galvanometer mirror 61 so that the processing light L from the light irradiation device 60 moved by the robot arm 310 is irradiated onto the processing target (for example, so that the same position on the processing target is irradiated) based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 320 and 330 moved by the robot arm 310.

[0172] In the second mode, the control device 1000 controls the driving unit 311 of the robot arm 310, which has been driven in the process of step S135, to stop driving, and after the driving of the driving unit 311 is stopped, the processing light L is irradiated onto the electrode of the chip LED. In this mode, the control device 1000 controls the driving unit 311 to stop driving the driving unit 311. After the driving of the driving unit 311 is stopped, the control device 1000 may control the orientation of the galvanometer mirror 61 of the light irradiation device 60 so that the irradiation position of the processing light L from the light irradiation device 60 is maintained on one electrode of the chip LED. For example, the control device 1000 controls the driving unit 311 to stop driving the driving unit 311. The control device 1000 may control the orientation of the galvanometer mirror 61 of the light irradiation device 60 so that the irradiation position of the processing light L from the light irradiation device 60 is maintained at the same position on one of the electrodes of the chip LED after the drive unit 311 stops driving (so that the processing light L from the light irradiation device 60 continues to be irradiated to the same position on one of the electrodes of the chip LED for a predetermined period of time).

[0173] In the above second aspect, for example, the control device 1000 may control the orientation of the galvanometer mirror 61 so that the processing light L from the light irradiation device 60, which displaces together with at least one of the detection devices 320 and 330, is irradiated onto the processing target (for example, so that the same position on the processing target is irradiated) based on at least one of the image data and shape data that changes with the displacement of at least one of the detection devices 320 and 330 after the drive of the drive unit 311 is stopped.

[0174] In the first embodiment, for example, when chip LEDs are mounted on a plurality of solder pads on a circuit board T at high speed, the control device 1000 controls the driving unit 311 of the robot arm 310 and the light irradiation device 60 so that the electrode (in other words, the first position) of the chip LED arranged on the solder pad to be mounted first and the electrode (in other words, the second position) of the chip LED arranged on the solder pad to be mounted second are sequentially irradiated with the processing light L while moving the light irradiation device 60 (detection devices 320 and 330) from the solder pad to be mounted first to the solder pad to be mounted second. In this case, the control device 1000 moves the light irradiation device 60 (detection devices 320 and 330) from the solder pad to be mounted first to the solder pad to be mounted second, in parallel with this step S137, based on the position and posture of the solder pad to be mounted second calculated (estimated) in step S131.

[0175] First, the control device 1000 controls the orientation of the galvanometer mirror 61 of the light irradiation device 60 (detection devices 320 and 330) that moves (displaces) relative to the electrode of the chip LED arranged on the solder pad to be mounted first, so that the electrode of the chip LED is irradiated with the processing light L for a predetermined time from the light irradiation device 60 (detection devices 320 and 330). More specifically, the control device 1000 gradually changes the orientation of the galvanometer mirror 61 according to the position and attitude of the chip LED output from the 2D tracking unit 302 at predetermined time intervals, in accordance with the movement of the light irradiation device 60 and the detection devices 320 and 330 by the robot arm 310, so that the irradiation position of the processing light L is maintained at one electrode of the chip LED (for example, the same position of one electrode of the chip LED). At this time, the control device 1000 can recognize changes in the position and attitude of one electrode of the chip LED relative to the light irradiation device 60 based on the position and attitude of the chip LED output from the 2D tracking unit 302 at predetermined time intervals, and can control the orientation of the galvanometer mirror 61 so that the irradiation position of the processing light L is maintained at one electrode of the chip LED (for example, the same position of one electrode of the chip LED). Note that the position and attitude of the chip LED in the image represented by the image data sequentially output from at least one of the detection devices 320 and 330 and the position and attitude of the chip LED in the point cloud represented by the shape data change with time due to the relative displacement between the chip LED and the light irradiation device 60 (detection devices 320 and 330).

[0176] After the control device 1000 has finished irradiating one electrode of the chip LED with the processing light L for a predetermined time, the control device 1000 gradually changes the orientation of the galvanometer mirror 61 so that the irradiation position of the processing light L is maintained on the other electrode of the chip LED (for example, the same position of the other electrode of the chip LED) based on the position and attitude of the chip LED output from the 2D tracking unit 302 at predetermined time intervals, in accordance with the continued movement of the light irradiating device 60 and the detection devices 320 and 330 by the robot arm 310. After the control device 1000 has finished irradiating the other electrode of the chip LED with the processing light L for a predetermined time, the control device 1000 repeats the above-mentioned steps S132 to S137 to similarly irradiate the electrode of the chip LED arranged on the solder pad to be mounted second.

[0177] For example, the control device 1000 may control the orientation of the galvanometer mirror 61 so that the irradiation position of the processing light L from the light irradiation device 60, which displaces with the displacement of at least one of the detection devices 320 and 330, is maintained at a first position and then maintained at a second position different from the first position, based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 320 and 330.

[0178] For example, the control device 1000 may control the drive unit 311 to move the light irradiation device 60 and the detection devices 320 and 330, while controlling the orientation of the galvanometer mirror 61 so that the irradiation position of the processing light L from the light irradiation device 60 moved by the robot arm 310 is maintained at a first position and then maintained at a second position different from the first position, based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 320 and 330 moved by the robot arm 310.

[0179] In the second embodiment, since inertial force and elastic force act on the light irradiation device 60 and the detection devices 320 and 330 provided on the robot arm 310, for example, the control device 1000 controls the driving unit 311 of the robot arm 310, which was driven in the process of step S135, to stop driving, and after stopping the driving unit 311, the light irradiation device 60 and the detection devices 320 and 330 are displaced due to vibration or the like, and the relative positions of the light irradiation device 60 (detection devices 320 and 330) and the chip LED fluctuate more or less with time. Therefore, the control device 1000 controls the galvanometer mirror 61 of the light irradiation device 60 so that the irradiation position of the processing light L from the light irradiation device 60 is maintained at one electrode (in other words, the first position) of the chip LED for a predetermined time, even if the light irradiation device 60 (detection devices 320 and 330) is displaced due to vibration or the like. Next, the control device 1000 controls the galvanometer mirror 61 of the light irradiation device 60 so that the irradiation position of the processing light L from the light irradiation device 60 is maintained on the other electrode of the chip LED (in other words, the second position) for a predetermined time even if the light irradiation device 60 (detection devices 320 and 330) continues to be displaced due to vibration or the like.

[0180] More specifically, first, the control device 1000 changes the orientation of the galvanometer mirror 61 over time based on the position and attitude of the chip LED output from the 2D tracking unit 302 at a predetermined time interval (as a result of displacement due to vibration of the light irradiation device 60, etc.) so that the irradiation position of the processing light L is maintained at one electrode of the chip LED (for example, the same position of one electrode of the chip LED). Since the control device 1000 can recognize the change in the position and attitude of one electrode of the chip LED relative to the light irradiation device 60, it can control the orientation of the galvanometer mirror 61 so that the irradiation position of the processing light L is maintained at one electrode of the chip LED (for example, the same position of one electrode of the chip LED). Note that the position and attitude of the chip LED in the image represented by the image data sequentially output from at least one of the detection devices 320 and 330 and the position and attitude of the chip LED in the point cloud represented by the shape data change over time due to the relative displacement between the chip LED and the light irradiation device 60 (detection devices 320 and 330).

[0181] After the control device 1000 has finished irradiating one electrode of the chip LED with the processing light L for a predetermined period of time, the control device 1000 then changes the orientation of the galvanometer mirror 61 over time so that the irradiation position of the processing light L is maintained on the other electrode of the chip LED (for example, the same position of the other electrode of the chip LED) based on the position and attitude of the chip LED output from the 2D tracking unit 302 at predetermined time intervals (due to displacement due to vibration of the light irradiation device 60, etc.).

[0182] When the processing light L is irradiated onto one or the other electrode of the chip LED, the position of the chip LED relative to the solder pad may change over time due to the surface tension of the molten solder. In this case, the control device 1000 gradually changes the orientation of the galvanometer mirror so that the irradiation position of the processing light L is maintained on one or the other electrode of the chip LED based on the position and attitude of the chip LED output from the 2D tracking unit 302 at predetermined time intervals. At this time, the control device 1000 can recognize the change in the position and attitude of one or the other electrode of the chip LED relative to the light irradiation device 60 based on the position and attitude of the chip LED output from the 2D tracking unit 302 at predetermined time intervals, so that the orientation of the galvanometer mirror can be controlled so that the irradiation position of the processing light L is maintained on one electrode of the chip LED.

[0183] In addition, not only when the driving unit 311 of the driven robot arm 310 is stopped, but also when the robot arm 310 accelerates or decelerates or after acceleration or deceleration, the light irradiation device 60 and detection devices 320 and 330 provided on the robot arm 310 are displaced due to vibration, etc., and the relative position between the light irradiation device 60 (detection devices 320 and 330) and the point to which the processing light L should be irradiated (e.g., the electrode of the chip LED) changes over time. However, even in this case, the control device 1000 can recognize the temporal change in the position and attitude of the point to which the processing light L should be irradiated relative to the light irradiation device 60 based on the position and attitude of the chip LED output from the 2D tracking unit 302 at predetermined time intervals, and can control the orientation of the galvanometer mirror so that the irradiation position of the processing light L is maintained at the point to which the processing light L should be irradiated.

[0184] In step S135, the control device 1000 may control the light irradiation device 60 to change the spot size and intensity of the processing light L when irradiating the processing light L to a location to be irradiated with the processing light L (for example, an electrode of a chip LED). In the case where the intensity of the processing light L is changed by changing the intensity of the light emitted from a light source (not shown) and the light source (not shown) is disposed outside the light irradiation device 60, it is sufficient to control the external light source (not shown).

[0185] For example, the control device 1000 may control the orientation of the galvanometer mirror 61 so that the irradiation position of the processing light L from the light irradiation device 60, which displaces with the displacement of at least one of the detection devices 320 and 330, is maintained at a first position and then maintained at a second position different from the first position, based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 320 and 330.

[0186] For example, the control device 1000 may control the driving unit 311 to stop driving the driving unit 311. Based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 320 and 330 after the driving of the driving unit 311 is stopped, the control device 1000 may control the orientation of the galvanometer mirror 61 so that the irradiation position of the processing light L from the light irradiation device 60, which displaces with the displacement of at least one of the detection devices 320 and 330, is maintained at a first position and then maintained at a second position different from the first position.

[0187] In addition, when the processing light L is irradiated while the light irradiation device 60, etc. is being moved by the robot arm 310, the control device 1000 may control at least one of the posture and position of the light irradiation device 60 and the orientation of the galvanometer mirror 61, etc., based on the prediction result of predicting the operation of the robot arm 310, in addition to the result of the tracking process.

[0188] In the processing of steps S131 to S137 described above, the control device 1000 may, for example, control the drive unit 311 of the robot arm 310 so that the light irradiation device 60 and the detection devices 320 and 330 approach the circuit board T based on at least one of the image data and shape data generated by at least one of the detection devices 320 and 330, and when the light irradiation device 60 and the detection devices 320 and 330 approach the circuit board T to a predetermined distance, control the orientation of the galvanometer mirror 61 based on the at least one of the above data that changes with the displacement of the detection devices 320 and 330 so that the processing light L from the light irradiation device 60, which displaces together with at least one of the detection devices 320 and 330, is irradiated to a location where the processing light L should be irradiated as part of the target object (for example, a solder pad formed on the circuit board T, an element or solder arranged on the circuit board T) (for example, so that the processing light L is irradiated to the same position as the location where the processing light L should be irradiated).

[0189] After the process of step S137, the control device 1000 performs a quality inspection of the soldered element and the solder based on at least one of the image data and the shape data output from at least one of the detection devices 320 and 330 (step S138). Examples of the inspection items include misalignment of the element with respect to the solder pad, floating of the electrode of the element with respect to the solder pad (so-called Manhattan phenomenon in which the electrode of the element separates from the solder pad), and the like. For example, when performing a quality inspection of the misalignment of the element with respect to the solder pad, the control device 1000 recognizes the solder pad and the element in the image shown by the image data based on the image data output from at least one of the detection devices 320 and 330, and detects the misalignment of the element with respect to the solder pad. For example, the control device 1000 may determine that the element is of good quality (good quality) when at least a part of the electrode of the element overlaps the solder pad, and may determine that the element is of poor quality when at least a part of the electrode of the element does not overlap the solder pad. The control device 1000 may detect misalignment of the element with respect to the solder pad based on shape data output from at least one of the detection devices 320 and 330, or on image data and shape data, without being limited to image data. The condition for the control device 1000 to determine a quality defect related to misalignment of the element with respect to the solder pad does not have to be whether or not at least a part of the electrode of the element overlaps the solder pad. For example, the control device 1000 may determine the quality based on the area where the electrode of the element overlaps the solder pad.

[0190] The control device 1000 may display at least one of the image data, shape data, and the results of the soldering quality inspection in the process of step S138 on a display device (not shown).

[0191] The control device 1000 may perform machine learning by an existing method using data in which at least one of the image data and shape data used in the processes of steps S131 to S137 is associated with the soldering quality determined in the process of step S138 as teacher data. In this case, the control device 1000 may use the results of the machine learning to control each device of the robot 3 (for example, control of the position and attitude of the light irradiation device 60 and control of the light irradiation device 60). Here, the control of the light irradiation device 60 includes setting the conditions of the processing light L irradiated from the light irradiation device 60 (for example, at least one of the intensity of the processing light L, the spot size of the processing light L, the irradiation time of the processing light L, and the irradiation range of the processing light L). The control device 1000 may use the results of the machine learning to control at least one of the devices of the robot 1 and the devices of the robot 2.

[0192] After the process of step S138, the control device 1000 starts moving the light irradiation device 60 (detection devices 320 and 330) to the solder pad to be mounted second based on the position and attitude of the chip LED output from the 2D tracking unit 302 at predetermined time intervals in the process of step S137 or step S136 and the position and attitude of the solder pad to be mounted second calculated in step S131, and repeats the processes of steps S133 to S138 described above. When starting the movement of the light irradiation device 60 (detection devices 320 and 330) to the solder pad to be mounted second, the control device 1000 may perform the process of step S131 described above and then execute the processes of steps S133 to S138 described above. The control device 1000 repeats the above steps until mounting (i.e., soldering) of all elements (e.g., chip LEDs, etc.) arranged on each solder pad of the circuit board T is completed.

[0193] In addition, if there is only one element placed on the solder pad on the circuit board T, the control device 1000 may control the drive unit 311, etc. so that the robot arm 310, etc. assumes a predetermined initial posture after processing of step S136.

[0194] As described above, in each step (particularly the first and second modes of step S137), the control device 1000 can recognize the position and posture of the object at a predetermined time interval by the tracking process. As a result, the control device 1000 can irradiate the processing light L to a desired position of the object (in other words, a place where the processing light L should be irradiated) by controlling at least one of the driving unit 311 of the robot arm 310 and the galvanometer mirror 61, even if the relative position between the object and the light irradiation device 60 (light detection devices 320 and 330) changes (displaces) over time.

[0195] In the first and second modes of step S137, the control device 1000 may control at least one of the driving unit 311 of the robot arm 310 and the galvanometer mirror 61 so that the irradiation position of the processing light L changes over time over a wide range of the area to be irradiated with the processing light L (for example, the entire area to be irradiated with the processing light L). For example, when the control device 1000 irradiates the processing light L to an electrode of an element as the area to be irradiated with the processing light L, the control device 1000 may control the mirror orientation of the galvanometer mirror 61 so that the irradiation position of the processing light L changes over time over the entire electrode while recognizing the position and posture of the element (the electrode) at predetermined time intervals by a tracking process. In other words, the mirror orientation of the galvanometer mirror 61 may be controlled so that the processing light L is scanned over the electrode. Such control can suppress localized heat input to the electrode (in other words, localized heat input to the solder), and can prevent damage to the element due to heat, insufficient melting of the solder due to local heating, damage to the circuit board due to local heating, and the like. In this case, as in the first mode of step S137, while moving the light irradiation device 60 and the detection devices 320 and 330 by the robot arm 310, the orientation of the galvanometer mirror may be changed so that the irradiation position of the processing light L changes over time over the entire one electrode of the chip LED based on the position and attitude of the chip LED (electrode of the chip LED) output at predetermined time intervals from the 2D tracking unit 302. At this time, the control device 1000 can recognize the change over time in the position and attitude of one electrode of the chip LED relative to the light irradiation device 60 based on the position and attitude of the chip LED output at predetermined time intervals from the 2D tracking unit 302, and therefore can control the orientation of the galvanometer mirror so that the irradiation position of the processing light L changes over time over the entire one electrode of the chip LED.

[0196] The position and orientation of the chip LED in the image shown by the image data and the position and orientation of the chip LED in the point cloud or depth image shown by the shape data, which are sequentially output from at least one of the detection devices 320 and 330, change over time due to the relative displacement between the chip LED and the light irradiation device 60 (detection devices 320 and 330). After finishing the irradiation of the processing light L to one electrode of the chip LED for a predetermined time, the control device 1000 changes the orientation of the galvanometer mirror so that the irradiation position of the processing light L changes over time over the entire other electrode of the chip LED based on the position and orientation of the chip LED output from the 2D tracking unit 302 at predetermined time intervals as the robot arm 310 continues to move the light irradiation device 60 and the detection devices 320 and 330. Then, after the control device 1000 has finished irradiating the other electrode of the chip LED with the processing light L for a predetermined period of time, it repeats the above-mentioned steps S132 to S137 to similarly irradiate the processing light L to the electrode of the chip LED located on the solder pad to be mounted second.

[0197] Also, for example, the control device 1000 changes the orientation of the galvanometer mirror over time based on the position and attitude of the chip LED output from the 2D tracking unit 302 at a predetermined time interval (as a result of displacement due to vibration of the light irradiation device 60, etc.) so that the irradiation position of the processing light L changes over time over the entire one electrode of the chip LED. Since the control device 1000 can recognize the change in the position and attitude of one electrode of the chip LED relative to the light irradiation device 60, it can control the orientation of the galvanometer mirror so that the irradiation position of the processing light L changes over time over the entire one electrode of the chip LED. Note that the position and attitude of the chip LED (the electrode of the chip LED) in the image shown by the image data sequentially output from at least one of the detection devices 320 and 330 and the position and attitude of the chip LED in the point cloud shown by the shape data change over time due to the relative displacement between the chip LED and the light irradiation device 60 (detection devices 320 and 330). After the control device 1000 has finished irradiating one of the electrodes of the chip LED with the processing light L for a predetermined period of time, the control device 1000 then changes the orientation of the galvanometer mirror over time so that the irradiation position of the processing light L changes over time across the entire other electrode of the chip LED, based on the position and attitude of the chip LED output from the 2D tracking unit 302 at predetermined time intervals (due to displacement due to vibration of the light irradiation device 60, etc.).

[0198] Therefore, the control device 1000 recognizes the temporal change in the position and attitude of the point to which the processing light L should be irradiated relative to the light irradiation device 60 based on the position and attitude of the point to which the processing light L should be irradiated (e.g., an electrode of an element) output from the 2D tracking unit 302 at a predetermined time interval, and can irradiate the processing light L to the desired range (the same position or the entire area) of the point to which the processing light L should be irradiated.

[0199] As described above, the location to be irradiated with the processing light L as part of the target object is not limited to an electrode of an element, but may be a solder pad or solder. Even in this case, the control device 1000 may control the mirror orientation of the galvanometer mirror 61 so that the irradiation position of the processing light L changes over time over the entire location to be irradiated with the processing light L (solder pad or solder). In addition, when the location to be irradiated with the processing light L is wide, the control device 1000 may drive the drive unit 311 of the robot 310 while controlling the orientation of the galvanometer mirror 61 to scan the location to be irradiated with the processing light L.

[0200] Also, for example, the control device 1000 may irradiate the processing light L alternately with time to one electrode and the other electrode of an element having a plurality of electrodes (for example, a chip LED having two electrodes) as a portion to be irradiated with the processing light L, to melt the solder. In this case, the position and posture of the element (one and the other electrodes) may be recognized at predetermined time intervals by a tracking process, and the orientation of the mirror of the galvanometer mirror 61 may be controlled so that the irradiation position of the processing light L changes alternately with time between the one electrode and the other electrode. Furthermore, the control device 1000 may control the orientation of the mirror of the galvanometer mirror 61 so that the irradiation position of the processing light L changes with time over the entire one electrode during the time period when the processing light L is irradiated to the one electrode. Similarly, the control device 1000 may control the orientation of the mirror of the galvanometer mirror 61 so that the irradiation position of the processing light L changes with time over the entire other electrode during the time period when the processing light L is irradiated to the other electrode. By such control, local heat input to the electrode (in other words, local heat input to the solder) can be suppressed.

[0201] As described above, the location to be irradiated with the processing light L is not limited to an electrode, and may be a solder pad or solder. Even in this case, the control device 1000 may control the orientation of the galvanometer mirror 61 so that the irradiation position of the processing light L alternates over time between a first location (one solder pad or one solder) and a second location (the other solder pad or the other solder) to be irradiated with the processing light L. Furthermore, when the first location and the second location to be irradiated with the processing light L are separated from each other, the control device 1000 may scan the processing light L by controlling the orientation of the galvanometer mirror 61 while driving the drive unit 311 of the robot 310.

[0202] In the process of step S134 described above, the control device 1000 may perform CAD matching using the shape data output from the detection device 330 and CAD data related to the element, and measure the position and orientation of the element. At this time, the control device 1000 may perform CAD matching after removing data corresponding to the board surface of the circuit board T from the shape data. With this configuration, the time required for CAD matching (in other words, the time required for the process of step S134) can be shortened. Note that various existing aspects can be applied to the method of removing the shape data corresponding to the board surface, for example, and detailed explanations thereof will be omitted.

[0203] If the elements can be mounted by melting the solder placed on the solder pads, control device 1000 does not need to execute the processes of steps S131 and S133, or steps S135 and S133.

[0204] In addition, the control device 1000 may display at least one of the image data, shape data, information on the position and attitude of the circuit board T, information on the position and attitude of each solder pad, information on the order of mounting on each solder pad, and information on the position and attitude of the elements used in the processing of the above-mentioned steps S131 to S137 on a display device (not shown).

[0205] In addition, in the process of steps S131 to S137 described above, the control device 1000 may detect at least one of the state of the solder (at least one of information on the distance between the solder pad and the solder, the shape of the solder, the volume of the solder, and the position and posture of the solder) and the area of ​​the solder pad based on at least one of the image data and the shape data output from the detection devices 320 and 330. That is, the control device 1000 may detect information on the state of the place to be irradiated with the processing light L. Then, the control device 1000 may use the information on the state of the place to be irradiated with the processing light L detected as described above to control the conditions of the processing light L irradiated from the light irradiation device 60 (for example, at least one of the intensity of the processing light L, the spot size of the processing light L, the irradiation time of the processing light L, and the irradiation range of the processing light L). That is, the control device 1000 may determine the conditions of the processing light L based on the detected information on the state of the place to be irradiated with the processing light L. In addition to the above information, the information regarding the state of the area to be irradiated with the processing light L may also include information regarding elements, solder, and solder pads as the areas to be irradiated with the processing light L that can be detected based on at least one of the image data and shape data output from at least one of the detection devices 320 and 330.

[0206] 16, an air blower and smoke suction machine 70 may be disposed near a light irradiation device 60 serving as an end effector of a robot arm 310 (not shown). While one of the air blower and the smoke suction machine is disposed, the other of the air blower and the smoke suction machine may not be disposed.

[0207] With this configuration, it is possible to effectively prevent smoke that is generated when the solder is heated by the processing light L emitted from the light irradiation device 60 from adhering to the optical system of the light irradiation device 60.

[0208] Robot 1 The control device 1000 that controls the robot 1 performs the processes of steps S111 and S112 that correspond to the processes of steps S131 and S132 described above, respectively. However, the control device 1000 performs the processes of steps S111 and S112 using the output of at least one of the detection devices 120 and 130 provided in the robot 1.

[0209] The control device 1000 may perform calibration of the dispenser 40 before performing the processes of steps S111 to S117 below.

[0210] It is assumed that the detection device 130 and the dispenser 40 are provided on the robot arm 110 in a positional relationship such that a part of the dispenser 40 (for example, a tip portion) is within the field of view of each of the cameras 31 and 32 of the detection device 130 having the same configuration as the detection device 330. For convenience of explanation, the cameras 31 and 32 of the detection device 130 will be referred to as the cameras 31 and 32 of the detection device 130 as an example in which the detection device 130 has the same cameras 31 and 32 as the detection device 330.

[0211] The control device 1000 performs the above-mentioned matching process using the shape data including the dispenser 40 output from the detection device 230 and the CAD data of the dispenser 40 as a calibration of the dispenser 40, and calculates in advance the position and orientation of the dispenser 40 (for example, the position and orientation of the tip of the dispenser 40 included in the field of view of the cameras 31 and 32 of the detection device 130). That is, the control device 1000 calculates in advance the position and orientation of the dispenser 40 in the coordinate system of the robot arm 110 based on at least a part of the shape data of the dispenser 40.

[0212] A marker may be provided on a part of the dispenser 40 included in the field of view of each of the cameras 31 and 32 of the detection device 130. In this case, the control device 1000 may perform the above-mentioned calibration based on, for example, shape data including the marker output from the detection device 130.

[0213] The control device 1000 may calibrate the dispenser 40 by executing a matching process using not only the shape data but also the image data output from the detection device 130 and the CAD data of the dispenser 40. As described above, the control device 1000 may use not only the CAD data but also the shape data and image data of the dispenser 40 acquired in advance in the matching process.

[0214] In addition, the calibration of the dispenser 40 is not limited to using the detection device 130, and the control device 1000 may use shape data or image data output from the detection device 120. In this case, it is assumed that the detection device 120 and the dispenser 40 are provided on the robot arm 110 in a positional relationship such that a part of the dispenser 40 is within the field of view of each of the cameras 21 and 22 of the detection device 120 having a configuration similar to that of the detection device 320.

[0215] In the process of steps S111 to S117 below, the dispenser 40 may come into contact with a predetermined object, for example, and the position and orientation of the dispenser 40 relative to the detection device 130 may change. In this case, the control device 1000 can detect a change in the position and orientation of the dispenser 40 relative to the detection device 130 based on a partial change in the dispenser 40 in the image data or shape data output from the detection device 130 (for example, a partial change in the dispenser 40 on the image). When a change in the position and orientation of the dispenser 40 relative to the detection device 130 is detected, the control device 1000 may execute calibration.

[0216] The control device 1000 that controls the robot 1 calculates the position and orientation of the circuit board T as an example of an object (step S111). In the process of step S111, similar to step S131 described above, the control device 1000 calculates the initial position and orientation (i.e., initial position and orientation) of the circuit board T through matching processing by the matching unit 301. Furthermore, the control device 1000 calculates the position of each solder pad formed on the circuit board T. For example, the control device 1000 calculates the position and orientation of each solder pad on the circuit board T based on the Gerber data of the circuit board T (i.e., design data of the circuit board T). The control device 1000 specifies the order of mounting (here, arranging solder) on each solder pad based on the Gerber data.

[0217] Next, the control device 1000 controls the driving unit 111 to move the robot arm 110 so that the dispenser 40 (detection devices 120 and 130) approaches the circuit board T (step 112). In the process of step S112, similar to step S132 described above, the control device 1000 controls the driving unit 111 of the robot arm 110 so that the solder pad to be mounted first is within the field of view of at least one of the detection devices 120 and 130.

[0218] Next, the control device 1000 judges whether the solder pad to be mounted first is within the field of view of at least one of the detection devices 120 and 130 (step S113). In the process of S113, similar to the above-mentioned step S133, the control device 1000 judges whether the detection devices 120 and 130 are in the desired position and posture with respect to the solder pad to be mounted first, based on the information on the position and posture of the circuit board T output from the 2D tracking unit 302 at predetermined time intervals and the information on the position and posture of the solder pad to be mounted first calculated in step S111. At this time, when the detection devices 120 and 130 are in the desired position and posture with respect to the solder pad, the control device 1000 judges that the solder pad is within the field of view of at least one of the detection devices 120 and 130.

[0219] If, in the processing of step S113, it is determined that the solder pad is not within the field of view of at least one of the detection devices 120 and 130 (S113: No), the control device 1000 controls the drive unit 111 to continue moving the robot arm 110 based on the position and posture information of the solder pad calculated in the processing of step S111 and the position and posture information of the circuit board T output from the 2D tracking unit 302 at predetermined time intervals.

[0220] On the other hand, if it is determined in the process of step S113 that the solder pad has entered the field of view of at least one of detection devices 120 and 130 (step S113: Yes), control device 1000 calculates the position and posture of the solder pad to be mounted first (step S114). As in the above-mentioned step S134, in the process of step S114, control device 1000 calculates the initial position and posture (initial position and posture) of the solder pad through matching process of matching unit 301. Note that in the process of the above-mentioned step S111, the position and posture of the solder pad are calculated based on Gerber data, but since the Gerber data is design data, there is an error between the position and posture of the solder pad on the actual circuit board T. Therefore, control device 1000 executes the process of this step S114.

[0221] Next, the control device 1000 controls the driving unit 111 so that the position and posture of the dispenser 40 are the desired position and posture that allows the dispenser 40 to dispense solder onto the solder pad to be mounted first, and moves the robot arm 110 (step S115). In the process of step S115, similar to the above-mentioned step S135, the control device 1000 uses the information on the initial position and posture of the solder pad calculated by the process of step S114 to control the driving unit 111 to move the robot arm 110 based on the position and posture of the solder pad output from the 2D tracking unit 302 at predetermined time intervals. In other words, the control device 1000 controls the driving unit 111 to move the robot arm 110 so that the dispenser 40 (detection devices 120 and 130) approaches the solder pad to be mounted first on the circuit board T.

[0222] Next, the control device 1000 judges whether or not the position and attitude of the dispenser 40 are the desired position and attitude that allows the dispenser 40 to dispense solder onto the solder pad to be mounted first (step S116). In the process of step S116, similar to the above-mentioned step S136, the control device 1000 judges whether or not the position and attitude of the dispenser 40 with respect to the solder pad are the desired position and attitude based on, for example, information on the position and attitude of the solder pad to be mounted first output from the 2D tracking unit 302 at predetermined time intervals.

[0223] In the process of step S116, if it is determined that the position and posture of the dispenser 40 are not the desired position and posture that allows the solder to be dispensed onto the solder pad (step S116: No), the control device 1000 controls the drive unit 111 to continue moving the robot arm 110 based on the position and posture of the solder pad output from the 2D tracking unit 302 at predetermined time intervals so that the dispenser 40 approaches the solder pad. On the other hand, if it is determined that the position and posture of the dispenser 40 are the desired position and posture that allows the solder to be dispensed onto the solder pad (step S116: Yes), the control device 1000 controls the dispenser 40 so that the solder is disposed on at least a part of the solder pad (step S117). Specifically, for example, the control device 1000 controls the dispenser 40 to dispense the solder. Here, the control device 1000 may estimate the area of ​​the solder pad based on the position and orientation of the solder pad output from the 2D tracking unit 302 at predetermined time intervals, and control the amount of solder dispensed from the dispenser 40 according to the area of ​​the solder pad. In this case, the control device 1000 may control the dispenser 40 so that the amount of solder dispensed increases as the estimated area of ​​the solder pad increases. This makes it possible to place an appropriate amount of solder on the solder pad.

[0224] At this time, the relative position between the dispenser 40 and the processing target may vary due to, for example, vibration of the robot arm 110. For this reason, the position of the solder pad as the target in the image shown by, for example, image data sequentially output from the detection device 130 may also shift over time due to the variation in the relative position.

[0225] Therefore, in order to suppress or eliminate the influence of the fluctuation in the relative position on the processing of step S117, the control device 1000 controls at least one of the attitude and the position of the dispenser 40 based on the result of the tracking processing described above.

[0226] For example, the control device 1000 may control the drive unit 111 based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 120 and 130 so that the solder dispensed from the dispenser 40, which displaces with the displacement of at least one of the detection devices 120 and 130, is placed on a solder pad as a target object.

[0227] For example, the control device 1000 may control the driving unit 111 to stop driving the driving unit 111. The control device 1000 may control the driving unit 111 so that the solder dispensed from the dispenser 40, which displaces together with at least one of the detection devices 120 and 130, is placed on a solder pad as a target, based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 120 and 130 after the driving of the driving unit 111 is stopped.

[0228] In the processing of steps S111 to S117 described above, the control device 1000 controls the drive unit 111 of the robot arm 110 so that the dispenser 40 and the detection devices 120 and 130 approach the circuit board T based on at least one of the image data and shape data generated by at least one of the detection devices 120 and 130, and when the dispenser 40 and the detection devices 120 and 130 approach the circuit board T to a predetermined distance, the control unit 111 may control the drive unit 111 so that the solder dispensed from the dispenser 40, which displaces together with at least one of the detection devices 120 and 130, is placed on a solder pad as the target object based on the at least one of the above data that changes in accordance with the displacement of at least one of the detection devices 120 and 130.

[0229] After the process of step S117, the control device 1000 starts moving the dispenser 40 to the solder pad to be mounted second based on the position and posture of the solder pad to be mounted first outputted from the 2D tracking unit 302 at a predetermined time interval in the process of step S117 or step S116 and the position and posture of the solder pad to be mounted second calculated in step S111, and repeats the processes of steps S113 to S117 described above. Note that when starting the movement of the dispenser 40 to the solder pad to be mounted second, the control device 1000 may perform the process of step S111 described above and then execute the processes of steps S113 to S117 described above. The control device 1000 repeats the above steps until the placement of solder on each solder pad of the circuit board T is completed.

[0230] If the circuit board T has only one solder pad, the control device 1000 may control the driving unit 111, etc. so that the robot arm 110, etc. assumes a predetermined initial posture after the process of step S117.

[0231] In addition, the control device 1000 may display at least one of the image data, shape data, information on the position and attitude of the circuit board T, information on the position and attitude of each solder pad, and information on the order of mounting on each solder pad, used in the processing of the above-mentioned steps S111 to S117 on a display device (not shown).

[0232] If the solder can be placed on the solder pads, the control device 1000 does not need to execute the processes of steps S111 and S113 or steps S115 and S116.

[0233] In the processes of steps S115 to S117 described above, the control device 1000 may detect (calculate) the area of ​​the solder pad based on at least one of the image data and the shape data output from the detection devices 120 and 130.

[0234] After the process of step S117, the control device 1000 may detect the state of the solder placed on the solder pad based on at least one of the image data and shape data output from at least one of the detection devices 120 and 130, or based on matching processing. The detected solder state may include the distance between the solder pad and the solder, the solder shape, the solder volume, the position and posture of the solder, etc. Then, the control device 1000 may judge the quality of the solder based on the detected solder state.

[0235] For example, when detecting the distance between the solder pad and the solder, the control device 1000 recognizes the solder pad and the solder in the image shown by the image data based on the image data output from at least one of the detection devices 120 and 130, and detects the distance between the solder pad and the solder. For example, the control device 1000 may determine whether the placement position of the solder is good or bad based on the detected distance between the solder pad and the solder. For example, the control device 1000 may determine that the placement position of the solder is bad when the detected distance between the solder pad and the solder is equal to or greater than a predetermined threshold (for example, a state in which the solder is not placed on the solder pad). Also, for example, when detecting the shape of the solder, the control device 1000 recognizes the solder in the point cloud shown by the shape data based on the shape data output from at least one of the detection devices 120 and 130, and detects the shape of the solder. For example, the control device 1000 may determine whether the shape of the solder is good or bad based on the detected shape of the solder. For example, the control device 1000 may determine that the solder shape is defective when the difference between the detected solder shape and the desired shape is equal to or greater than a threshold value. For example, when detecting the volume of the solder, the control device 1000 may estimate the volume of the solder by an existing method based on the shape of the solder detected by the above-mentioned method. For example, the control device 1000 may determine whether the volume of the solder is good or bad based on the estimated volume of the solder. For example, the control device 1000 may determine that the volume of the solder is defective when the detected volume of the solder is outside a threshold value (for example, when the volume of the solder is too large or too small). For example, when detecting the position and posture of the solder, the control device 1000 recognizes the solder in the point cloud indicated by the shape data based on the shape data output from at least one of the detection devices 120 and 130, and detects the position and posture of the solder. For example, the control device 1000 may determine whether the shape of the solder is good or bad based on the detected position and posture of the solder. For example, when at least one of the detected solder position and posture is outside a threshold, the control device 1000 may determine that the solder position and posture are defective.

[0236] The control device 1000 may perform machine learning by an existing method using data in which at least one of the image data and shape data used in the processing of steps S111 to S117 is associated with the solder quality detected as described above as teacher data. In this case, the control device 1000 may use the results of the machine learning to control each device of the robot 1 (for example, control of the position and attitude of the dispenser 40 and control of the discharge of the dispenser 40). The control device 1000 may use the results of the machine learning to control at least one of the devices of the robot 2 and the devices of the robot 3.

[0237] Furthermore, the control device 1000 may use the information on the area of ​​the solder pad and the state of the solder detected as described above (at least one of information on the distance between the solder pad and the solder, the shape of the solder, the volume of the solder, and the position and attitude of the solder) to control the position and attitude of the holding device 50 in the robot arm 210 of the robot 2. In this case, the element held by the holding device 50 can be efficiently placed on the solder.

[0238] Furthermore, the control device 1000 may use the information on the area of ​​the solder pad and the state of the solder (at least one of information on the distance between the solder pad and the solder, the shape of the solder, the volume of the solder, and the position and attitude of the solder) detected as described above for at least one of the control of the position and attitude of the light irradiation device 60 by the robot arm 310 of the robot 3 and the control of the galvanometer mirror 61. In this case, the light irradiation device 60 can efficiently irradiate the processing light L to the places on the circuit board T where the processing light L should be irradiated (for example, the arranged elements, the arranged solder, the solder pads, etc.).

[0239] The control device 1000 may use at least one of the information on the area of ​​the solder pad and the information on the state of the solder (at least one of the information on the distance between the solder pad and the solder, the shape of the solder, the volume of the solder, and the position and posture of the solder) detected as described above to control the conditions of the processing light L irradiated from the light irradiation device 60 by the robot arm 310 of the robot 3 (for example, at least one of the intensity of the processing light L, the spot size of the processing light L, the irradiation time of the processing light L, and the irradiation range of the processing light L). Note that the irradiation range of the processing light L includes, for example, at least a part of the element, the solder, the solder pad, etc. as the part to be irradiated with the processing light L. Here, at least one of the information on the area of ​​the solder pad and the information on the state of the solder (at least one of the information on the distance between the solder pad and the solder, the shape of the solder, the volume of the solder, and the position and posture of the solder) can be rephrased as information on the state of the part to be irradiated with the processing light L. For example, the control device 1000 may determine the conditions of the processing light L based on the detected information on the state of the part to be irradiated with the processing light L. As an example, the control device 1000 may determine the spot size of the processing light L based on the area of ​​the solder pad. Note that, in addition to the above information, the information on the state of the location to be irradiated with the processing light L may include information on the element, the solder, and the solder pad as the location to be irradiated with the processing light L that can be detected based on at least one of the image data and the shape data output from at least one of the detection devices 120 and 130 or a matching process.

[0240] Furthermore, the control device 1000 may use at least one of the information on the position and posture of the circuit board T calculated in the processing of steps S111 to S117 described above, the information on the position and posture of each solder pad, and the information on the order of mounting for each solder pad, for at least one of the control of the position and posture of the holding device 50 by the robot arm 210 of the robot 2 and the control of the holding force of the holding device 50. In this case, the element held by the holding device 50 can be efficiently placed on the solder. For example, when the above information is used for the position and posture control of the holding device 50 by the robot arm 210 of the robot 2, the processing of step S121 described later by the control device 1000 can be omitted, so that the element gripped by the holding device 50 can be efficiently placed on the solder pad (solder).

[0241] Furthermore, the control device 1000 may use at least one of the information on the position and posture of the circuit board T calculated in the processing of steps S111 to S117 described above, the information on the position and posture of each solder pad, and the information on the order of mounting for each solder pad, for at least one of the control of the position and posture of the light irradiation device 60 by the robot arm 310 of the robot 3 and the control of the galvanometer mirror 61. In this case, the light irradiation device 60 can efficiently irradiate the processing light L to the portion on the circuit board T to be irradiated with the processing light L. For example, when the above information is used for the position and posture control of the light irradiation device 60 by the robot arm 310 of the robot 3, the processing of step S131 described later by the control device 1000 can be omitted, so that the light irradiation device 60 can efficiently melt the solder using the processing light L.

[0242] The control device 1000 may display at least one of the image data, shape data, and the detection result of the solder state used in the above-mentioned solder state detection process on a display device (not shown).

[0243] For example, the control device 1000 may control the drive unit 111 based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 120 and 130 so that the solder dispensed from the dispenser 40, which displaces with the displacement of at least one of the detection devices 120 and 130, is placed at a first position on the circuit board T, and then is placed at a second position different from the first position.

[0244] For example, the control device 1000 may control the drive unit 111 to stop driving the drive unit 111. The control device 1000 may control the drive unit 111 so that the solder dispensed from the dispenser 40, which displaces with the displacement of at least one of the detection devices 120 and 130, is placed at a first position on the circuit board T and then placed at a second position different from the first position, based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 120 and 130 after the drive of the drive unit 111 is stopped.

[0245] Robot 2 The control device 1000 that controls the robot 2 performs the processes of steps S122 and S123 that correspond to the processes of steps S131 and S132 described above, respectively. However, the control device 1000 performs the processes of steps S121 and S123 using the output of at least one of the detection devices 220 and 230 provided in the robot 2.

[0246] For example, the holding device 50 includes a tweezers hand capable of opening and closing the tips of the tweezers. The holding device 50 may include a suction device capable of suction-holding the element.

[0247] The control device 1000 may perform calibration of the holding device 50 before performing the processes of steps S121 to S129 below.

[0248] It is assumed that the detection device 230 and the holding device 50 are provided on the robot arm 210 in a positional relationship such that the tip of the holding device 50 (i.e., in the case of a tweezers hand, the tip of the tweezers that comes into contact with the element when holding the element) is within the field of view of each of the cameras 31 and 32 of the detection device 230 having a configuration similar to that of the detection device 330. For ease of explanation, the cameras 31 and 32 of the detection device 230 will be referred to as the cameras 31 and 32 of the detection device 230, as an example in which the detection device 230 has the same cameras 31 and 32 as the detection device 330.

[0249] The control device 1000 performs the above-mentioned matching process as a calibration of the holding device 50, using the shape data output from the detection device 230 when the holding device 50 is not gripping an element and the CAD data of the holding device 50, and calculates in advance the position and orientation of the holding device 50 (for example, the position and orientation of the tip of the tweezers hand included in the field of view of the cameras 31 and 32 of the detection device 230). That is, the control device 1000 calculates in advance the position and orientation of the holding device 50 in the coordinate system of the robot arm 210 based on the shape data of at least a part of the holding device 50 (for example, the tip of the tweezers hand).

[0250] Note that a marker may be provided on a part of the holding device 50 included in the field of view of each of the cameras 31 and 32 of the detection device 230. In this case, the control device 1000 may perform the above-mentioned calibration based on, for example, shape data including the marker output from the detection device 230.

[0251] The control device 1000 may calibrate the holding device 50 by executing a matching process using not only the shape data but also the image data output from the detection device 230 and the CAD data of the holding device 50. In the matching process, the control device 1000 may use not only the CAD data but also previously acquired shape data or image data of the holding device 50, as described above.

[0252] Note that the calibration of the holding device 50 is not limited to using the detection device 230, and the control device 1000 may use shape data or image data output from the detection device 220. In this case, it is assumed that the detection device 220 and the holding device 50 are provided on the robot arm 210 in a positional relationship such that the tip of the holding device 50 is within the field of view of each of the cameras 21 and 22 of the detection device 220 having the same configuration as the detection device 320.

[0253] In the processes of steps S121 to S129 below, the position and orientation of the holding device 50 relative to the detection device 230 may change due to the holding device 50 coming into contact with a predetermined object, etc. In this case, the control device 1000 can detect the change in the position and orientation of the holding device 50 relative to the detection device 230 based on a partial change of the holding device 50 in the image data or shape data output from the detection device 230 (for example, a partial change of the holding device 50 on the image). When a change in the position and orientation of the holding device 50 relative to the detection device 230 is detected, the control device 1000 may execute calibration.

[0254] The control device 1000 that controls the robot 2 holds the element (step S121). The control device 1000 controls the drive unit 211 of the robot arm 210 and the holding device 50 so that the holding device 50 of the robot 2 approaches an element supplying device (so-called a parts feeder) not shown so as to be able to hold the desired element. For example, the control device 1000 executes at least one of the above-mentioned matching process and tracking process, calculates the position and orientation of the desired element placed on the element supplying device not shown, and then moves the holding device 50 closer to the desired element placed on the element supplying device not shown so as to be able to hold the element, and causes the holding device 50 to hold the desired element. Here, the control device 1000 may determine the holding force (gripping force) of the element in the holding device 50 according to the size of the element calculated in at least one of the matching process and tracking process. This makes it possible to prevent damage to the element caused by the holding device 50 holding the element and the element from falling off from the holding device 50.

[0255] After the process of step S121, the control device 1000 calculates the position and orientation of the circuit board T as an example of the target object (step S122). In the process of step S122, similar to the above-mentioned steps S111 and S131, the control device 1000 calculates the initial position and orientation (i.e., initial position and orientation) of the circuit board T through the matching process of the matching unit 301. Furthermore, the control device 1000 calculates the position of each solder pad formed on the circuit board T. The control device 1000 calculates the position of each solder pad on the circuit board T based on the Gerber data of the circuit board T (i.e., design data of the circuit board T). The control device 1000 specifies the order of mounting (here, arranging elements) on each solder pad based on the Gerber data.

[0256] Next, the control device 1000 controls the driving unit 211 to move the robot arm 210 so that the holding device 50 (detection devices 220 and 230) approaches the circuit board T (step 123). As in the above-mentioned steps S112 and S132, in the process of step S123, the control device 1000 controls the driving unit 211 of the robot arm 210 so that the solder pad to be first mounted is within the field of view of at least one of the detection devices 220 and 230.

[0257] Next, the control device 1000 judges whether or not the marker provided near the solder pad to be mounted first has entered the field of view of at least one of the detection devices 220 and 230 (step S124). In the process of step S124, similar to the above-mentioned steps S113 and S133, the control device 1000 judges whether or not the detection devices 220 and 230 are in the desired position and posture with respect to the solder pad to be mounted first, based on the information on the position and posture of the circuit board T output from the 2D tracking unit 302 at predetermined time intervals and the information on the position and posture of the solder pad to be mounted first calculated in step S121. At this time, when the detection devices 220 and 230 are in the desired position and posture with respect to the solder pad to be mounted first, the control device 1000 judges that the marker provided near the solder pad has entered the field of view of at least one of the detection devices 220 and 230.

[0258] If, in the processing of step S124, it is determined that the marker located near the first solder pad to be mounted is not within the field of view of at least one of the detection devices 220 and 230 (S124: No), the control device 1000 continues to move the robot arm 210 by controlling the drive unit 211 based on the position and posture information of the first solder pad to be mounted calculated by the processing of step S122 and the position and posture information of the circuit board T outputted from the 2D tracking unit 302 at predetermined time intervals.

[0259] On the other hand, in the process of step S124, if it is determined that the marker provided in the vicinity of the solder pad to be mounted first has entered the field of view of at least one of detection devices 220 and 230 (step S124: Yes), control device 1000 calculates the position and orientation of the marker provided in the vicinity of the solder pad to be mounted first (step S125). As in the above-mentioned steps S114 and S134, in the process of step S125, control device 1000 calculates the initial position and orientation (initial position and orientation) of the marker provided in the vicinity of the solder pad to be mounted first by the matching process of matching unit 301.

[0260] Next, the control device 1000 controls the driving unit 211 to move the robot arm 210 so that the position and posture of the holding device 50 are the desired position and posture that allows an element to be placed on the solder placed on the solder pad to be mounted first (step S126). In the process of step S126, similar to the above-mentioned steps S115 and S135, the control device 1000 uses the information on the initial position and posture of the marker placed near the solder pad calculated by the process of step S125 to control the driving unit 211 to move the robot arm 210 based on the information on the position and posture of the marker placed near the solder pad output from the 2D tracking unit 302 at predetermined time intervals and the positional relationship between the solder pad and the marker. Note that the positional relationship between the solder pad and the marker is known.

[0261] In other words, the control device 1000 controls the driving unit 211 to move the robot arm 210 so that the holding device 50 (detection devices 220 and 230) approaches the solder pad (solder placed on the solder pad) that is to be mounted first on the circuit board T. The control device 1000 may also control the driving unit 211 to move the robot arm 210 based on information on the distance between the solder pad and the solder detected after the solder is placed on the solder pad by the dispenser 40 of the robot 1. In this case, the holding device 50 can be more accurately (more efficiently) moved closer to the solder pad (solder placed on the solder pad).

[0262] Next, the control device 1000 judges whether or not the position and attitude of the holding device 50 are the desired position and attitude that allows an element to be placed on the solder placed on the solder pad to be mounted first (step S127). In the process of step S127, similar to the above-mentioned steps S116 and S136, the control device 1000 judges whether or not the position and attitude of the holding device 50 with respect to the solder pad (solder placed on the solder pad) are the desired position and attitude based on, for example, information on the position and attitude of a marker placed near the solder pad to be mounted first output from the 2D tracking unit 302 at predetermined time intervals and the positional relationship between the solder pad and the marker.

[0263] If, in the processing of step S127, it is determined that the position and posture of the holding device 50 are not the desired position and posture for placing an element on the solder placed on the solder pad to be mounted first (step S127: No), the control device 1000 continues to move the robot arm 210 by controlling the drive unit 211 based on the position and posture of the solder pad (the solder) output from the 2D tracking unit 302 at predetermined time intervals so that the holding device 50 approaches the solder pad (the solder).

[0264] On the other hand, if it is determined that the position and posture of the holding device 50 are the desired position and posture for placing an element on the solder pad (the solder) (step S127: Yes), the control device 1000 calculates the position and posture of the element held by the holding device 50 (step S128).

[0265] First, as a premise, as described above, the tip of the holding device 50 (i.e., in the case of a tweezers hand, the tip of the tweezers that contacts the element when holding the element) is included within the field of view of each of the cameras 31 and 32 of the detection device 230. Furthermore, the detection device 230 is provided at a desired position of the robot arm 210 so that at least a part of the element held by the holding device 50 is also included within the field of view of each of the cameras 31 and 32.

[0266] The control device 1000 performs the above-mentioned CAD matching process using, for example, the shape data output from the detection device 230 while the holding device 50 is holding an element (i.e., data including shape data of the tip of the holding device 50 and at least a part of the element) and the CAD data of the element, and calculates the position and orientation of the element. The position and orientation of the element held by the holding device 50 changes every time the holding device 50 holds it, even if the elements are of the same type (i.e., the same shape). Therefore, by executing the process of this step S125, the control device 1000 can recognize the position and orientation of the element, and therefore, in step S129 described below, the element can be placed with high accuracy on the solder pad (solder) to be mounted first.

[0267] The control device 1000 may recognize shape data of the tip of the holding device 50 by CAD matching processing or the like from among the shape data output from the detection device 230 while the holding device 50 holds the element, and after performing processing to remove the shape data of the tip of the holding device 50 from the shape data output from the detection device 230, perform CAD matching processing using the CAD data of the element to calculate the position and orientation of the element. In this case, it is possible to prevent the shape data of the tip of the holding device 50 from becoming noise in the processing to calculate the position and orientation of the element, thereby preventing a decrease in the calculation accuracy of the position and orientation of the element.

[0268] The control device 1000 may, for example, remove the shape data of the tip of the holding device 50 calculated in the matching process in the calibration of the holding device 50 described above from the shape data output from the detection device 230 in a state where the holding device 50 holds the element, and then perform a CAD matching process using the CAD data of the element to calculate the position and orientation of the element. In this case as well, it is possible to prevent a decrease in the calculation accuracy of the position and orientation of the element.

[0269] In addition, for the purpose of calibrating the holding device 50 described above, the detection device 230 is provided on the robot arm 210 so that the tip of the holding device 50 is included within the field of view of each of the cameras 31 and 32 of the detection device 230. However, if calibration of the holding device 50 is not performed, the detection device 230 may be provided on the robot arm 210 so that the holding device 50 is not included within the field of view of each of the cameras 31 and 32 of the detection device 230, and at least a portion of the element held by the holding device 50 is included.

[0270] After the process of step S128, the control device 1000 controls the holding device 50 so that the element is placed on the solder pad (solder) to be mounted first (step S129). Based on the position and attitude of the element held by the holding device 50 calculated in the process of step S128 described above and the position and attitude of the solder pad (solder) output from the 2D tracking unit 302 at predetermined time intervals, the control device 1000 controls the driving unit 211 of the robot arm 210 so that the position and attitude of the element held by the holding device 50 become the desired position and attitude that allows the element to be placed on the solder pad (solder). Then, the control device 1000 controls the holding device 50 to release the holding of the element, and places the element on the solder pad (solder).

[0271] At this time, similar to the process of step S117 described above, the relative position between the holding device 50 and the solder pad (solder) as the target may vary due to, for example, vibration of the robot arm 210. Therefore, in order to suppress or eliminate the effect of the variation in the relative position on the process of step S129, the control device 1000 controls at least one of the attitude and position of the holding device 50 based on the result of the tracking process described above.

[0272] For example, the control device 1000 may control the drive unit 211 based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 220 and 230 so that an element held by the holding device 50, which displaces with the displacement of at least one of the detection devices 220 and 230, is placed on a solder pad (solder).

[0273] For example, the control device 1000 may control the driving unit 211 to stop driving the driving unit 211. The control device 1000 may control the driving unit 211 so that an element held by the holding device 50, which displaces together with at least one of the detection devices 220 and 230, is placed on a solder pad (solder) based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 220 and 230 after the driving of the driving unit 211 is stopped.

[0274] In the processing of steps S121 to S129 described above, the control device 1000 controls the drive unit 211 of the robot arm 210 so that the holding device 50 and the detection devices 220 and 230 approach the circuit board T based on at least one of the image data and shape data generated by at least one of the detection devices 220 and 230, and when the holding device 50 and the detection devices 220 and 230 approach the circuit board T to a predetermined distance, the control unit 211 may control the drive unit 211 so that the element held by the holding device 50, which displaces together with at least one of the detection devices 220 and 230, is placed on the solder pad (solder) as the target object based on at least one of the above data that changes with the displacement of at least one of the detection devices 220 and 230.

[0275] After the process of step S129, the control device 1000 drives the drive unit 211 of the robot arm 210 so that the holding device 50 approaches an element supplying device (not shown), and picks up an element to be placed on the second solder pad (solder) to be mounted. The control device 1000 then repeats the processes of steps S122 to S129 described above. The control device 1000 repeats picking up elements and the above-mentioned steps until placement of elements on the solder of each solder pad of the circuit board T is completed.

[0276] In addition, if there is only one solder pad (one element to be placed on the solder) on the circuit board T, the control device 1000 may control the drive unit 211, etc. so that the robot arm 210, etc. assumes a predetermined initial posture after processing of step S129.

[0277] The control device 1000 may execute the process of step S128 described above before the process of step S127. The control device 1000 may not execute the process of step S128 described above. In this case, the control device 1000 may control the holding device 50 so that the element is placed on the solder pad (solder) to be mounted first following the process of step S127 described above (more precisely, step S127: Yes).

[0278] In addition, the control device 1000 may display at least one of the image data, shape data, information on the position and attitude of the circuit board T, information on the position and attitude of the marker, information on the position and attitude of each solder pad, and information on the order of mounting on each solder pad, used in the processing of the above-mentioned steps S122 to S129 on a display device (not shown).

[0279] If an element can be placed on the solder on the solder pad, control device 1000 does not need to execute the processes of steps S122 and S124 or steps S126 and S127.

[0280] After the process of step S129, the control device 1000 may detect the state of the element placed on the solder of the solder pad based on at least one of the image data and shape data output from at least one of the detection devices 220 and 230 and matching processing. The state of the element to be detected may include the position and posture of the element, the distance between the element and the solder, the distance between the element and the solder pad, etc. Then, the control device 1000 may judge the quality of the placed element based on the detected state of the element.

[0281] For example, when detecting the position and orientation of an element, the control device 1000 calculates the position and orientation of the element based on the above-mentioned matching process. For example, the control device 1000 may determine whether the arrangement position of the element is good or bad based on the detected position and orientation of the element. For example, the control device 1000 may determine that the arrangement position of the solder is defective if at least one of the detected position and orientation of the element is outside a predetermined threshold.

[0282] For example, when detecting the distance between an element and a solder, the control device 1000 recognizes the element and the solder in the point cloud indicated by the shape data based on the shape data output from at least one of the detection devices 220 and 230, and calculates the distance between the element and the solder. For example, the control device 1000 may determine whether the arrangement position of the element is good or bad based on the detected distance between the element and the solder. For example, the control device 1000 may determine that the arrangement position of the solder is bad when the detected distance between the element and the solder is equal to or greater than a predetermined threshold (e.g., the element is not arranged on the solder).

[0283] For example, when detecting the distance between an element and a solder pad, the control device 1000 recognizes the element and the solder pad of the image shown by the image data based on the image data output from at least one of the detection devices 220 and 230, and calculates the distance between the element and the solder pad. For example, the control device 1000 may determine whether the placement position of the element is good or bad based on the detected distance between the element and the solder pad. For example, the control device 1000 may determine that the placement position of the element is bad when the detected distance between the element and the solder pad is equal to or greater than a predetermined threshold (e.g., the element is not placed on the solder pad).

[0284] The control device 1000 may perform machine learning by an existing method using data in which at least one of the image data and shape data used in the processes of steps S122 to S128 is associated with the quality of the element determined as described above as training data. In this case, the control device 1000 may use the results of the machine learning for controlling each device of the robot 2 (for example, controlling the position and attitude of the holding device 50 and controlling the holding of the holding device 50). The control device 1000 may use the results of the machine learning for at least one of controlling each device of the robot 1 and each device of the robot 3.

[0285] The control device 1000 may use at least one of the information detected as described above, the position and posture of the element, the distance between the element and the solder, and the distance between the element and the solder pad, for at least one of the control of the position and posture of the light irradiation device 60 by the robot arm 310 of the robot 3, the control of the galvanometer mirror 61 of the light irradiation device 60, and the change of the intensity and the spot size of the processing light L. In this case, the light irradiation device 60 can efficiently irradiate the processing light L to the portion on the circuit board T where the processing light L should be irradiated (for example, the arranged element, the arranged solder, the solder pad, etc.). The control device 1000 may use at least one of the information detected as described above, the position and posture of the element, the distance between the element and the solder, and the distance between the element and the solder pad, for the control of the position and posture of the dispenser 40 by the robot arm 110 of the robot 1.

[0286] In addition, in the process of steps S122 to S129 described above, the control device 1000 may detect at least one of the state of the solder (at least one of information on the distance between the solder pad and the solder, the shape of the solder, the volume of the solder, and the position and posture of the solder) and the area of ​​the solder pad based on at least one of the image data and the shape data output from the detection devices 120 and 130. That is, the control device 1000 may detect information on the state of the place to be irradiated with the processing light L. Then, the control device 1000 may use the information on the state of the place to be irradiated with the processing light L detected as described above to control the conditions of the processing light L irradiated from the light irradiation device 60 by the robot arm 310 of the robot 3 (for example, at least one of the intensity of the processing light L, the spot size of the processing light L, the irradiation time of the processing light L, and the irradiation range of the processing light L). That is, the control device 1000 may determine the conditions of the processing light L based on the detected information on the state of the place to be irradiated with the processing light L. In addition to the above information, the information regarding the state of the area to be irradiated with the processing light L may also include information regarding elements, solder, and solder pads as the areas to be irradiated with the processing light L that can be detected based on at least one of the image data and shape data output from at least one of the detection devices 220 and 230 or matching processing.

[0287] The control device 1000 may use at least one of the information on the position and posture of the circuit board T calculated in the processing of steps S122 to S129 described above, the information on the mounting order for each solder pad, the information on the position and posture of the marker, and the information on the position and posture of the solder pad (solder) for at least one of the control of the position and posture of the light irradiation device 60 by the robot arm 310 of the robot 3, the control of the galvanometer mirror 61, and the change of the intensity and spot size of the processing light L. In this case, the light irradiation device 60 can efficiently irradiate the processing light L to the portion on the circuit board T where the processing light L should be irradiated. The control device 1000 may use at least one of the information on the position and posture of the circuit board T and the information on the position and posture of the solder pad (solder) detected as described above for the control of the position and posture of the dispenser 40 by the robot arm 110 of the robot 1.

[0288] The control device 1000 may display at least one of the image data, shape data, and detection results of the element state used in the above-mentioned element state detection process on a display device (not shown).

[0289] For example, the control device 1000 may control the drive unit 211 so that, based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 220 and 230, after one element held by the holding device 50, which displaces with the displacement of at least one of the detection devices 220 and 230, is positioned at a first position on the circuit board T, another element held by the holding device 50 is positioned at a second position different from the first position.

[0290] For example, the control device 1000 may control the drive unit 211 to stop driving the drive unit 211. Based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 220 and 230 after the drive of the drive unit 211 is stopped, the control device 1000 may control the drive unit 211 so that after one element held by the holding device 50, which displaces with the displacement of at least one of the detection devices 220 and 230, is placed at a first position on the circuit board T, another element held by the holding device 50 is placed at a second position different from the first position.

[0291] The above-mentioned processes of steps S111 to S117, steps S121 to S129, and steps S131 to S138 are repeatedly performed in parallel.

[0292] The control device 1000 may perform machine learning by an existing method using data that associates at least one of the image data and shape data used in at least one of the processes of steps S131 to S138, steps S111 to S117, and steps S121 to S129 with at least one of the soldering quality determined in step S138, the solder quality determined as described above, and the element quality determined as described above as teacher data. In this case, the control device 1000 may use the results of the machine learning for at least one of the control of each device of robot 1, the control of each device of robot 2, and the control of each device of robots 3 and 4.

[0293] (Technical effect) LDS (Lase Direct Structuring) and SMT (Surface Mount Technology) are known as technologies for mounting elements on the surface of a circuit board having a three-dimensional shape (i.e., a 3D circuit board). However, LDS requires the use of a special heat-resistant molding material (resin). In addition, the 3D circuit board must be placed inside a mounting machine (chip mounter) or a reflow oven. For this reason, LDS and other techniques can only be applied to relatively high-priced and relatively small products.

[0294] For this reason, for relatively large products, a method is often adopted in which a daughter board is connected to a mother board with a cable. With this method, for example, the weight of the cable makes the product relatively heavy, and manual labor is required to assemble the wiring components. On the other hand, in the field of automobiles, for example, there is a demand for technology to mount elements on relatively large 3D circuit boards due to space and weight constraints.

[0295] For relatively large 3D circuit boards, it is desirable to use relatively inexpensive board materials in order to keep costs down. Since relatively inexpensive board materials have relatively low heat resistance, laser soldering, which allows pinpoint heat input, is one method for mounting elements. However, there is a technical problem in that the 3D circuit board will be thermally damaged if the irradiation position of processing light L such as laser light cannot be precisely controlled.

[0296] In contrast, in the above-described robot 3, even if the positional relationship between the light irradiation device 60 and the like and the target object (for example, a solder pad, an element, or a solder) is displaced, the orientation and the like of the galvanometer mirror 61 are controlled based on at least one of the image data and the shape data output from the detection device 330 so that the irradiation position of the processing light L remains the same. In other words, according to the robot 3, the irradiation position of the processing light L can be precisely controlled.

[0297] Soldering by the robot 3 can reduce thermal damage to the board, which increases the options for board materials. In other words, relatively inexpensive board materials can be used in soldering by the robot 3.

[0298] According to the robot 3, while moving the light irradiation device 60 and the like with the robot arm 310 (i.e., while the light irradiation device 60 and the like are moving), the irradiation position of the processing light L can be changed by the galvanometer mirror 61, so that the processing light L can be irradiated to a desired position. Therefore, one or more elements can be efficiently mounted on a relatively large substrate.

[0299] According to the robot 3, after the light irradiation device 60 and the like are moved by the robot arm 310 (in other words, when the robot arm 310 is not driven by the drive unit 311), the irradiation position of the processing light L can be adjusted by the galvanometer mirror 61 to irradiate the processing light L at the desired position. Therefore, the processing light L can be irradiated without waiting for the convergence of the vibration of the light irradiation device 60 and the like moved by the robot arm 310. Furthermore, even if an error occurs in the movement of the light irradiation device 60 and the like by the robot arm 310, the error can be corrected by controlling the orientation of the galvanometer mirror 61, and the processing light L can be irradiated at the desired position.

[0300] According to the robot 3, since it has the detection devices 320 and 330, it is possible to carry out a quality inspection of the soldering after the soldering (after the process of step S137 described above) (see the process of step S138 described above). In other words, according to the robot 3, since it has the detection devices 320 and 330, it is possible to carry out a quality inspection of the soldering immediately after the soldering. In other words, according to the robot 3, it is possible to carry out a quality inspection of the soldering efficiently.

[0301] In the robot 1, the position and posture of the target object (e.g., a solder pad) can be recognized at a predetermined time interval by a tracking process by the control device 1000. As a result, by controlling the drive unit 111 of the robot arm 110, it is possible to place solder at a desired position on the target object (in other words, a place where the solder should be placed) even if the relative position between the target object and the dispenser 40 (detection devices 120 and 130) changes (displaces) over time.

[0302] In the robot 2, the position and posture of the target object (for example, solder placed on a solder pad) can be recognized at predetermined time intervals by the tracking process performed by the control device 1000. As a result, by controlling the driving unit 211 of the robot arm 210, it is possible to place an element at a desired position on the target object (in other words, the location where the element should be placed) even if the relative position between the target object and the holding device 50 (detection devices 220 and 230) changes (displaces) over time.

[0303] <Second embodiment> The second embodiment will be described with reference to Figures 17 and 18. In this embodiment, as in the above-described first embodiment, a soldering system including a robot that performs soldering is given. In the second embodiment, descriptions that overlap with the first embodiment will be omitted, and common parts in the drawings will be indicated by the same reference numerals, and only the fundamental differences will be described with reference to Figures 17 and 18.

[0304] (overview) An overview of the soldering system according to the second embodiment will be described with reference to Fig. 17(a) and Fig. 17(b). In Fig. 17, the soldering system is a system for soldering an element to a circuit board T. The soldering system includes a robot 4. The robot 4, which may be called a processing device, a solder application device, an element installation device, or a soldering device, includes a dispenser 40 for discharging solder, a holding device 50 capable of holding an element, a light irradiation device 60 for irradiating processing light L for melting the solder, a storage section (not shown) for storing the element, a supply device (not shown) for supplying the desired element from the storage section to the holding device 50, and detection devices 420 and 430 for detecting light from the circuit board T. The robot 4 includes a robot arm 410 having a drive section 411 for moving the dispenser 40, the holding device 50, the light irradiation device 60, and the detection devices 420 and 430.

[0305] Here, the detection devices 420 and 430 correspond to the above-described detection devices 320 and 330, respectively. The detection device 420 may have a similar configuration to the above-described detection device 320. The detection device 430 may have a similar configuration to the above-described detection device 330.

[0306] The soldering system includes a control device 1000 that (i) controls a drive unit 411 so that the dispenser 40, the holding device 50, the light irradiation device 60, and the detection devices 420 and 430 approach the circuit board T, (ii) controls the dispenser 40 so that solder is placed on a predetermined portion of the circuit board T, (iii) controls the holding device 50 so that an element is placed on the circuit board T via the placed solder, and (iv) controls the light irradiation device 60 so as to melt the placed solder.

[0307] In FIG. 17(a), the control device 1000 controls the dispenser 40 and the like so that solder is placed on a predetermined portion of the circuit board T. The control device 1000 then controls the holding device 50 and the like so that an element is placed via the placed solder. The control device 1000 then controls the light irradiation device 60 so that the solder is melted. After that, the control device 1000 may perform a quality inspection on the soldering, for example, based on the detection result of the detection device 430. In other words, the robot 4 performs the work shared by the robots 1, 2, and 3 according to the first embodiment described above. With this configuration, it is possible to improve productivity while suppressing the initial investment for introducing the robot.

[0308] (Robot 4) In FIG. 17(a) and FIG. 17(b), the robot 4 includes a dispenser 40, a holding device 50, a light irradiation device 60, a storage unit (not shown) for storing elements, and a supply device (not shown) for supplying a desired element from the storage unit to the holding device 50. The robot 4 further includes (i) detection devices 220 and 230 for detecting light from the circuit board T and generating at least one of image data and shape data, and (ii) a robot arm 410 provided with the dispenser 40, the holding device 50, the light irradiation device 60, and the detection devices 420 and 430, and having a drive unit 411 for moving the holding device 50 and the detection devices 420 and 430. Examples of the storage unit include a reel, a tray, and a stick. Since various existing aspects can be applied to the storage unit and the supply device, detailed explanations thereof will be omitted.

[0309] Similar to the robot arm 310 described above, the robot arm 410 has arms 410a and 410b and a wrist 410c.

[0310] 17(a), the detection device 420 is disposed on the arm 410b of the robot arm 410, and the detection device 430 is disposed on the wrist 410c of the robot arm 410, but the arrangement of the detection devices 420 and 430 is not limited thereto. The robot 4 may be provided with only one of the detection devices 420 and 430, or may be provided with another detection device in addition to the detection devices 420 and 430 (i.e., the robot 4 may be provided with three or more detection devices). The robot 4 may be provided with at least one detection device other than the detection devices 320 and 330. In other words, as long as the dispenser 40, the holding device 50, and the light irradiation device 60 can be brought close to the circuit board T or a predetermined portion of the circuit board T (e.g., a solder pad provided on the circuit board T, an element or solder arranged on the circuit board T, etc.) by driving the driving unit 411 of the robot arm 410 so that solder can be placed on the circuit board T, an element can be placed on the placed solder, and the placed solder can be melted with processing light, the configuration (e.g., the number and specifications of cameras in the detection device, the presence or absence of a projector, etc.) and the placement positions and number of the detection devices 420 and 430 are not limited.

[0311] The control device 1000 may, for the robot 4 configured as described above, based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 420 and 430, (i) control the drive unit 411 so that the solder dispensed from the dispenser 40, which displaces with the displacement of at least one of the detection devices 420 and 430, is positioned at a predetermined portion of the circuit board T, (ii) control the drive unit 411 so that the element grasped (held) by the holding device 50, which displaces with the displacement of at least one of the detection devices 420 and 430, is positioned at a predetermined portion of the circuit board T, or (iii) control the orientation of the galvanometer mirror 61 so that the processing light L from the light irradiation device 60, which displaces with the displacement of at least one of the detection devices 420 and 430, is irradiated to the same position.

[0312] Here, the control device 1000 may be a device separate from the robot 4, or may configure a part of the robot 4 (in other words, the robot 4 may be equipped with the control device 1000).

[0313] The robot arm 410 may be mounted on, for example, an AGV (Automatic Guided Vehicle). In this case, the control device 1000 may control at least one of the driving unit of the robot arm 410, the end effector of the robot arm 410, and the driving unit of the AGV, based on information on the position and posture of the target object acquired by the above-mentioned matching process and tracking process described below.

[0314] (Robot movement) First, the control device 1000 may perform calibration of the dispenser 40, the holding device 50, and the light irradiation device 60 before performing the processes of the following steps.

[0315] It is assumed that the detection device 430, the dispenser 40, the holding device 50, and the light irradiation device 60 are provided on the robot arm 410 in a positional relationship such that part of the dispenser 40, part of the holding device 50, and part of the light irradiation device 60 are within the field of view of each of the cameras 31 and 32 of the detection device 430.

[0316] The positions and orientations of the dispenser 40, the holding device 50, and the light irradiation device 60 are calculated in advance by a process similar to the above-mentioned calibration process.

[0317] Next, the operation of the robot 4 will be described with reference to the flowchart of Fig. 18. However, the control device 1000 performs each process described in the flowchart of Fig. 18 using the output of at least one of the detection devices 420 and 430 provided in the robot 4.

[0318] The control device 1000 that controls the robot 1 calculates the position and orientation of the circuit board T as an example of an object (step S111). In the process of step S111, the control device 1000 calculates the initial position and orientation (i.e., initial position and orientation) of the circuit board T through matching processing by the matching unit 301. Furthermore, the control device 1000 calculates the position of each solder pad formed on the circuit board T. For example, the control device 1000 calculates the position and orientation of each solder pad on the circuit board T based on the Gerber data of the circuit board T (i.e., design data of the circuit board T). The control device 1000 specifies the order of mounting (here, arranging solder) on each solder pad based on the Gerber data.

[0319] Next, the control device 1000 controls the driving unit 411 to move the robot arm 410 so that the dispenser 40 (the detection devices 420 and 430) approaches the circuit board T (step S112). In the process of step S112, the control device 1000 controls the driving unit 411 of the robot arm 410 so that the solder pad to be first mounted is within the field of view of at least one of the detection devices 420 and 430.

[0320] Next, the control device 1000 judges whether the solder pad to be mounted first is within the field of view of at least one of the detection devices 420 and 430 (step S113). In the process of step S113, the control device 1000 judges whether the detection devices 420 and 430 are in the desired position and posture with respect to the solder pad to be mounted first, based on the information on the position and posture of the circuit board T output from the 2D tracking unit 302 at predetermined time intervals and the information on the position and posture of the solder pad to be mounted first calculated in step S111. At this time, when the detection devices 420 and 430 are in the desired position and posture with respect to the solder pad, the control device 1000 judges that the solder pad is within the field of view of at least one of the detection devices 420 and 430.

[0321] If, in the processing of step S113, it is determined that the solder pad is not within the field of view of at least one of the detection devices 420 and 430 (step S113: No), the control device 1000 continues to move the robot arm 410 by controlling the drive unit 411 based on the position and posture information of the solder pad calculated by the processing of step S111 and the position and posture information of the circuit board T output from the 2D tracking unit 302 at predetermined time intervals.

[0322] On the other hand, in the process of step S113, if it is determined that the solder pad is within the field of view of at least one of detection devices 420 and 430 (step S113: Yes), control device 1000 calculates the position and orientation of the solder pad to be mounted first (step S114). In the process of step S114, control device 1000 calculates the initial position and orientation (initial position and orientation) of the solder pad through matching process of matching unit 301.

[0323] Next, the control device 1000 controls the driving unit 411 so that the position and posture of the dispenser 40 are the desired position and posture that allows the solder to be discharged to the solder pad to be mounted first, and moves the robot arm 410 (step S115). In the process of step S115, the control device 1000 uses the information on the initial position and posture of the solder pad calculated by the process of step S114 to control the driving unit 411 to move the robot arm 410 based on the position and posture of the solder pad output from the 2D tracking unit 302 at predetermined time intervals. In other words, the control device 1000 controls the driving unit 411 to move the robot arm 410 so that the dispenser 40 (detection devices 420 and 430) approaches the solder pad to be mounted first on the circuit board T.

[0324] Next, the control device 1000 judges whether or not the position and attitude of the dispenser 40 are the desired position and attitude that allows the solder to be dispensed onto the solder pad to be mounted first (step S116). In the process of step S116, the control device 1000 judges whether or not the position and attitude of the dispenser 40 with respect to the solder pad are the desired position and attitude based on, for example, the information on the position and attitude of the solder pad to be mounted first output from the 2D tracking unit 302 at predetermined time intervals.

[0325] In the process of step S116, if it is determined that the position and posture of the dispenser 40 are not the desired position and posture that allows the solder to be dispensed onto the solder pad (step S116: No), the control device 1000 continues to move the robot arm 410 by controlling the driving unit 411 based on the position and posture of the solder pad output from the 2D tracking unit 302 at predetermined time intervals so that the dispenser 40 approaches the solder pad. On the other hand, if it is determined that the position and posture of the dispenser 40 are the desired position and posture that allows the solder to be dispensed onto the solder pad (step S116: Yes), the control device 1000 controls the dispenser 40 so that the solder is disposed on at least a part of the solder pad (step S117). Specifically, for example, the control device 1000 controls the dispenser 40 so as to dispense the solder. Here, the control device 1000 may estimate the area of ​​the solder pad based on the position and orientation of the solder pad output from the 2D tracking unit 302 at predetermined time intervals, and control the amount of solder dispensed from the dispenser 40 according to the area of ​​the solder pad. In this case, the control device 1000 may control the dispenser 40 so that the amount of solder dispensed increases as the estimated area of ​​the solder pad increases. This makes it possible to place an appropriate amount of solder on the solder pad.

[0326] At this time, the relative position between the dispenser 40 and the processing target may vary due to, for example, vibration of the robot arm 410. For this reason, the position of the processing target in the image shown by, for example, image data, which is sequentially output from the detection device 430, may also be displaced over time due to the variation in the relative position.

[0327] Therefore, in order to suppress or eliminate the influence of the fluctuation in the relative position on the processing of step S117, the control device 1000 controls at least one of the attitude and the position of the dispenser 40 based on the result of the tracking processing described above.

[0328] In addition, the control device 1000 may display at least one of the image data, shape data, information on the position and attitude of the circuit board T, information on the position and attitude of each solder pad, and information on the order of mounting on each solder pad, used in the processing of the above-mentioned steps S111 to S117 on a display device (not shown).

[0329] If the solder can be placed on the solder pads, the control device 1000 does not need to execute the processes of steps S111 and S113 or steps S115 and S116.

[0330] After the process of step S117, the control device 1000 may detect the state of the solder placed on the solder pad based on at least one of the image data and shape data output from at least one of the detection devices 420 and 430, or based on matching processing. The detected solder state may include the distance between the solder pad and the solder, the solder shape, the solder volume, the position and posture of the solder, etc. Then, the control device 1000 may judge the quality of the solder based on the detected solder state.

[0331] In parallel with the processing of steps S111 to S117 described above, or at least prior to the processing of step S128 described below, the control device 1000 controls a supply device (not shown) provided on the robot arm 410 to supply a desired element from a storage section (not shown) to the holding device 50, and controls the holding device 50 to hold the supplied element (step S141).

[0332] After the process of step S117 (step S141), the control device 1000 controls the driving unit 211 to move the robot arm 210 so that the position and posture of the holding device 50 are the desired position and posture that allows an element to be placed on the solder placed on the solder pad to be mounted first (step S126). In the process of step S126, the control device 1000 controls the driving unit 411 to move the robot arm 410 based on the information on the position and posture of the marker placed near the solder pad output from the 2D tracking unit 302 at predetermined time intervals using the information on the initial position and posture of the marker placed near the solder pad calculated by the process of step S114, for example, and the positional relationship between the solder pad and the marker.

[0333] Next, the control device 1000 judges whether or not the position and attitude of the holding device 50 are the desired position and attitude that allows an element to be placed on the solder placed on the solder pad to be mounted first (step S127). In the process of step S127, the control device 1000 judges whether or not the position and attitude of the holding device 50 with respect to the solder pad (solder placed on the solder pad) are the desired position and attitude based on, for example, information on the position and attitude of a marker placed near the solder pad to be mounted first output from the 2D tracking unit 302 at predetermined time intervals and the positional relationship between the solder pad and the marker.

[0334] If, in the processing of step S127, it is determined that the position and posture of the holding device 50 are not the desired position and posture for placing an element on the solder placed on the solder pad to be mounted first (step S127: No), the control device 1000 continues to move the robot arm 410 by controlling the drive unit 411 based on the position and posture of the solder pad (the solder) output from the 2D tracking unit 302 at predetermined time intervals so that the holding device 50 approaches the solder pad (the solder).

[0335] On the other hand, if it is determined that the position and posture of the holding device 50 are the desired position and posture for placing an element on the solder pad (the solder) (step S127: Yes), the control device 1000 calculates the position and posture of the element held by the holding device 50 (step S128).

[0336] After the process of step S128, the control device 1000 controls the holding device 50 so that the element is placed on the solder pad (solder) to be mounted first (step S129). Based on the position and attitude of the element held by the holding device 50 calculated in the process of step S128 described above and the position and attitude of the solder pad (solder) output from the 2D tracking unit 302 at predetermined time intervals, the control device 1000 controls the driving unit 211 of the robot arm 210 so that the position and attitude of the element held by the holding device 50 become the desired position and attitude that allows the element to be placed on the solder pad (solder). Then, the control device 1000 controls the holding device 50 to release the holding of the element, and places the element on the solder pad (solder).

[0337] At this time, similarly to the process of step S117 described above, the relative position between the holding device 50 and the processing target may vary due to, for example, vibration of the robot arm 410. Therefore, in order to suppress or eliminate the influence of the variation in the relative position on the process of step S129, the control device 1000 controls at least one of the attitude and position of the holding device 50 based on the result of the tracking process described above.

[0338] In addition, the control device 1000 may display at least one of the image data, shape data, information on the position and attitude of the circuit board T, information on the position and attitude of the marker, information on the position and attitude of each solder pad, and information on the order of mounting on each solder pad, used in the processing of the above-mentioned steps S126 to S129 on a display device (not shown).

[0339] If an element can be placed on the solder on the solder pad, control device 1000 does not need to execute the processes of steps S126 and S127 described above.

[0340] After the process of step S129, the control device 1000 may detect the state of the element placed on the solder of the solder pad based on at least one of the image data and shape data output from at least one of the detection devices 220 and 230 and matching processing. The state of the element to be detected may include the position and posture of the element, the distance between the element and the solder, the distance between the element and the solder pad, etc. Then, the control device 1000 may judge the quality of the placed element based on the detected state of the element.

[0341] The control device 1000 may display at least one of the image data, shape data, and detection results of the element state used in the above-mentioned element state detection process on a display device (not shown).

[0342] Next, the control device 1000 controls the driving unit 311 to move the robot arm 310 so that the position and posture of the light irradiation device 60 are at a desired position and posture that allows the processing light L to melt the solder arranged on the first solder pad (step S135). In the process of step S135, the control device 1000 uses the information on the initial position and posture of the element calculated (estimated) by the process of step S134 to control the driving unit 311 to move the robot arm 310 based on the position and posture of the element output from the 2D tracking unit 302 of the tracking unit 300 at predetermined time intervals. In other words, the control device 1000 controls the driving unit 311 to move the robot arm 310 so that the light irradiation device 60 (detection devices 320 and 330) approaches the element arranged on the first solder pad of the circuit board T.

[0343] Next, the control device 1000 judges whether or not the position and posture of the light irradiation device 60 are the desired position and posture that allows the processing light L to melt the solder placed on the first solder pad (step S136). In the process of step S136, the control device 1000 judges whether or not the position and posture of the light irradiation device 60 relative to the element are the desired position and posture based on, for example, information on the position and posture of the element output from the 2D tracking unit 302 at predetermined time intervals. At this time, if the position and posture of the light irradiation device 60 relative to the element are the desired position and posture, the control device 1000 judges that the position and posture of the light irradiation device 60 are the desired position and posture that allows the processing light L to melt the solder placed on the first solder pad.

[0344] In the process of step S136, if it is determined that the position and posture of the light irradiation device 60 are not the desired position and posture at which the solder placed on the first solder pad can be melted by the processing light L (step S136: No), the control device 1000 continues to move the robot arm 410 by controlling the driving unit 411 based on the position and posture of the element placed on the first solder pad outputted at predetermined time intervals from the 2D tracking unit 302 so that the light irradiation device 60 approaches the element. In other words, the process of step S135 is performed until it is determined that the position and posture are the desired position and posture at which the solder placed on the first solder pad can be melted by the processing light L.

[0345] On the other hand, if it is determined that the position and posture of the light irradiation device 60 are the desired position and posture that allow the processing light L to melt the solder placed on the first solder pad (step S136: Yes), the control device 1000 controls the light irradiation device 60 to irradiate the processing light L to the electrodes of the element placed on the solder pad to be mounted first, that is, the two electrodes of the chip LED, so as to melt the solder placed on the solder pad to be mounted first (step S137). As a result, the solder placed on the solder pad melts, and the element is soldered to the circuit board T (the solder pad to be mounted first).

[0346] After the process of step S137, the control device 1000 performs a quality inspection of the soldered element and the solder based on at least one of the image data and shape data output from at least one of the detection devices 320 and 330 (step S138). Items to be inspected include, for example, misalignment of the element with respect to the solder pad, floating of the electrode of the element with respect to the solder pad (the so-called Manhattan phenomenon in which the electrode of the element separates from the solder pad), and the like.

[0347] The control device 1000 may display at least one of the image data, shape data, and the results of the soldering quality inspection in the process of step S138 on a display device (not shown).

[0348] If the elements can be mounted by melting the solder placed on the solder pads, control device 1000 does not need to execute the processes of steps S135 and S136 described above. The control device 1000 may execute the process of step S128 described above before the process of step S127. The control device 1000 may not execute the process of step S128 described above. In this case, the control device 1000 may control the holding device 50 so that the element is placed on the solder pad (solder) to be mounted first after the process of step S127 described above.

[0349] In addition, the control device 1000 may display at least one of the image data, shape data, information on the position and attitude of the circuit board T, information on the position and attitude of each solder pad, information on the order of mounting on each solder pad, and information on the position and attitude of the elements used in the processing of the above-mentioned steps S135 to S137 on a display device (not shown).

[0350] After the process of step S138, the control device 1000 starts moving the dispenser 40 to the solder pad to be mounted second based on the position and posture of the solder pad to be mounted first output from the 2D tracking unit 302 at a predetermined time interval and the position and posture of the solder pad to be mounted second calculated in step S111, and repeats the process of step S113 and after. When starting the movement of the dispenser 40 to the solder pad to be mounted second, the control device 1000 may perform the process of step S111 described above before executing the process of step S113 and after. The control device 1000 repeats the above steps until the placement of solder on each solder pad of the circuit board T is completed.

[0351] (Technical effect) As described above, the robot 4 performs the solder placement process, the element mounting process, the soldering process, and the inspection process with one robot arm. Therefore, the processes from step S114 in FIG. 17 onward are performed after the driving unit 411 stops driving the robot arm 410.

[0352] However, immediately after the driving of the robot arm 410 stops, vibrations may occur at the tip of the robot arm 410 (for example, the light irradiation device 60 as an end effector). Vibrations may also occur due to the operation of the wrist 410c of the robot arm 410, such as solder placement. If it were necessary to wait until the vibrations subside every time vibrations occur before starting processing such as solder placement, productivity would decrease significantly.

[0353] In contrast, in the robot 4, for example, at least one of the posture and position of the light irradiation device 60 and the like, and the orientation of the galvanometer mirror 61 of the light irradiation device 60 are controlled based on the results of a tracking process similar to the tracking process according to the first embodiment described above. Therefore, even if vibration occurs, the robot 4 can appropriately place solder on the processing target, appropriately place elements, and appropriately irradiate the processing light L based on the results of the tracking process. In other words, the robot 4 can start processing such as solder placement without waiting for the vibration to converge.

[0354] <Modification> Modifications of the above-mentioned first and second embodiments will be described.

[0355] (Modification of the detection device 320) Here, a modified example of the detection device 320 will be described, but similar modified examples can also be adopted for the detection devices 120, 220, and 420.

[0356] (1) The detection device 320 may be equipped with a single camera instead of the cameras 21 and 22. In this case, the detection device 320 generates and outputs only image data. Even in this case, the control device 1000 can appropriately bring, for example, the light irradiation device 60 closer to the circuit board T based on the result of the tracking process performed by the 2D tracking unit 302 (see FIG. 12) based on the image data.

[0357] (2) The detection device 320 may include a projector in addition to the cameras 21 and 22. In this case, the detection device 320 may be configured to generate and output at least one of image data and shape data of the object.

[0358] (3) The detection device 320 may include a single camera instead of the cameras 21 and 22, and a projector. In this case, the detection device 320 may be configured to project structured light, for example as shown in FIG. 7, from the projector onto the object, and generate image data of the object onto which the structured light is projected by the single camera. The detection device 320 may be configured to generate shape data in addition to the image data. Various existing methods, such as a phase shift method, a random dot method, and a TOF method, can be applied to the method of generating the shape data.

[0359] (Modification of the detection device 330) Here, a modified example of the detection device 330 will be described, but similar modified examples can also be adopted for the detection devices 130, 230, and 430.

[0360] (1) The detection device 330 may be equipped with a single camera instead of the cameras 31 and 32 and the projector 33. In this case, the detection device 330 generates and outputs only image data. Even in this case, the control device 1000 can appropriately irradiate the processing target with, for example, the processing light L based on the result of the tracking process performed by the 2D tracking unit 302 (see FIG. 12) based on the image data.

[0361] (2) The detection device 330 does not need to include the projector 33. In this case, the detection device 330 may capture images of an object using the cameras 31 and 32 at the same time, generate shape data as a detection result based on two pieces of image data output from the cameras 31 and 32, and output the shape data.

[0362] (3) The detection device 330 may include a single camera instead of the cameras 31 and 32, and a projector 33. In this case, the detection device 330 may be configured to project structured light, for example as shown in FIG. 7, from the projector 33 onto the object, and generate image data of the object onto which the structured light is projected by the single camera. The detection device 330 may be configured to generate shape data in addition to the image data. Various existing methods, such as a phase shift method, a random dot method, and a TOF method, can be applied to the method of generating the shape data.

[0363] (Modification of the light irradiation device 60) (1) In the above-mentioned robots 3 and 4, the light irradiation device 60 and the detection devices 320, 330, 420, and 430 are provided individually on the robot arm 310 or 410. That is, the optical path of the processing light L in the light irradiation device 60 and the optical paths of the detection devices 320, 330, 420, and 430 (more precisely, the optical paths of the cameras of the detection devices 320, 330, 420, and 430) are different from each other. However, this configuration is not limited to this, and as shown in FIG. 19, a part of the optical path of the processing light L in the light irradiation device 60 and a part of the optical path of the detection device 320, 330, 420, or 430 (more precisely, the optical path of the camera of the detection device 320, 330, 420, or 430) may be a common optical path. In other words, the final optical element 63 of the light irradiation device 60 may constitute a part of the optical system of the camera equipped in the detection device 330, for example. That is, the light irradiation device 60 may be a so-called coaxial laser processing head. The final optical element 63 may include a galvanometer mirror 61 and an fθ lens 62.

[0364] (2) The light irradiation device 60 may have a mechanism capable of changing the optical path, such as a MEMS (Micro Electro Mechanical System) mirror, a polygon mirror, or a DMD, instead of the galvanometer mirror 61. In this case, the mechanism capable of changing the optical path may function as a scanning unit capable of scanning the surface of the object with the processing light L.

[0365] (3) The light irradiation device 60 does not need to include a scanning unit such as the galvanometer mirror 61.

[0366] (Modification of Matching Process and Tracking Process) First Variation The control device 1000 may include, for example, a tracking unit 300′ shown in Fig. 20 instead of the above-described matching processing unit 200 and tracking unit 300. That is, the matching unit 301 of the above-described tracking unit 300 may have a processing block or processing circuit similar to that of the above-described matching processing unit 200.

[0367] If the comparison unit 203 of the matching processing unit 200 of the tracking unit 300' determines that the first matching rate is greater than the second matching rate (i.e., the first matching rate > the second matching rate), of the image data and shape data output from the detection device 320 at a predetermined time interval, the image data is input to the 2D tracking unit 302, and of the image data and shape data, the shape data is input to the 3D tracking unit 303.

[0368] Furthermore, the comparison unit 203 outputs the position and orientation of the object calculated by the first matching unit 201 through 2D matching using image data output from the detection device 320 at predetermined time intervals (i.e., position and orientation estimation result) to the 2D tracking unit 302. The comparison unit 203 outputs the position and orientation of the object calculated by the first matching unit 201 through 3D matching using shape data output from the detection device 320 at predetermined time intervals (i.e., position and orientation estimation result) to the 3D tracking unit 303.

[0369] If the comparison unit 203 determines that the second matching rate is greater than or equal to the first matching rate (i.e., if the first matching rate is less than or equal to the second matching rate), of the image data and shape data output from the detection device 330 at a predetermined time interval, the image data is input to the 2D tracking unit 302, and of the image data and shape data, the shape data is input to the 3D tracking unit 303.

[0370] Furthermore, the comparison unit 203 outputs to the 2D tracking unit 302 the position and orientation of the object calculated by the second matching unit 202 through 2D matching using image data output from the detection device 330 at predetermined time intervals (i.e., the position and orientation estimation result). The comparison unit 203 outputs to the 3D tracking unit 303 the position and orientation of the object calculated by the second matching unit 202 through 3D matching using shape data output from the detection device 330 at predetermined time intervals (i.e., the position and orientation estimation result).

[0371] As described above, each of the first matching unit 201 and the second matching unit 202 may narrow down the range in which 3D matching should be performed based on the result of 2D matching, and perform 3D matching using shape data corresponding to the narrowed down range (see FIG. 10). With this method, each of the first matching unit 201 and the second matching unit 202 can perform 3D matching at high speed. In this case, the comparison unit 203 (see FIG. 9) may sequentially compare the results of 3D matching by each of the first matching unit 201 and the second matching unit 202, and output a position and orientation estimation result with a high matching rate to the 2D tracking unit 302 and the 3D tracking unit 303 at predetermined time intervals.

[0372] The control device 1000 may perform matching processing in the first matching unit 201 or the second matching unit 202 at predetermined time intervals according to the comparison result of the above-mentioned matching rates in the comparison unit 203, and may correct the results of the tracking processing (calculation results of the position and orientation of the object) of each of the 2D tracking unit 302 and the 3D tracking unit 303 using the results of the matching processing generated in the first matching unit 201 or the second matching unit 202 based on the results of the matching processing (calculation results of the position and orientation of the object). That is, the control device 1000 may correct the results of the tracking processing of each of the 2D tracking unit 302 and the 3D tracking unit 303 based on the results of the matching processing at predetermined time intervals, similar to the timing chart of FIG. 13.

[0373] The control device 1000 may perform matching processing in the first matching unit 201 or the second matching unit 202 at predetermined time intervals according to the comparison result of the above-mentioned matching rates in the comparison unit 203, and may correct the result of the tracking processing of the 2D tracking unit 302 or the 3D tracking unit based on the result of the matching processing using the result of the matching processing generated in the first matching unit 201 or the second matching unit 202. In this case, the control device 1000 may output the result of the matching processing to at least one of the 2D tracking unit 302 and the 3D tracking unit 303, which corrects the result of the tracking processing, from the first matching unit 201 or the second matching unit 202 via the comparison unit 203 based on the comparison result of the above-mentioned matching rates.

[0374] It should be noted that the control device 1000 does not need to correct the results of the tracking processes of the 2D tracking unit 302 and the 3D tracking unit 303 based on the results of the matching processes generated by the first matching unit 201 or the second matching unit 202 .

[0375] The control device 1000 may compare the above-mentioned matching rates in the comparison unit 203 for all matching processing results output at predetermined time intervals from the first matching unit 201 and the second matching unit 202. In this case, the control device 1000 may switch the matching processing results output from the comparison unit 203 to the 2D tracking unit 302 and the 3D tracking unit 303 between the matching processing results from the first matching unit 201 and the matching processing results from the second matching unit 202, based on the comparison results of the matching rates generated in the comparison unit 203 at predetermined time intervals.

[0376] The control device 1000 may not compare the above-mentioned matching rates in the comparison unit 203 for all the results of the matching process output at a predetermined time interval from each of the first matching unit 201 and the second matching unit 202. For example, the control device 1000 may compare the above-mentioned matching rates in the comparison unit 203 based on the results of the matching process output from each of the first matching unit 201 and the second matching unit 202 at the timing of starting the tracking process of the target object. The control device 1000 may compare the above-mentioned matching rates at a predetermined time point in the comparison unit 203, and after the predetermined time point, output the results of the matching process output from the first matching unit 201 or the second matching unit 202 at a predetermined time interval to at least one of the 2D tracking unit 302 and the 3D tracking unit 303 based on the comparison result of the matching rate performed at the predetermined time point.

[0377] In addition, the control device 1000 only needs to output at least one of the results of the 2D matching processing and the results of the 3D matching processing from at least one of the first matching unit 201 and the second matching unit 202 (via the comparison unit 203) to at least one of the 2D tracking unit 302 and the 3D tracking unit 303.

[0378] Second Variation Here, the detector 330 is used as the detector, but the same can be said for the detectors 130, 230 and 430.

[0379] In the above-mentioned tracking process, the result of the tracking process by the 2D tracking unit 302 (hereinafter referred to as "2D tracking process" as appropriate) is corrected based on the result of the tracking process by the 3D tracking unit 303 (hereinafter referred to as "3D tracking process" as appropriate), and the result of the 2D tracking process is output to the robot control unit 100 (see Figure 12).

[0380] However, the tracking process method is not limited to the above-mentioned method (see FIG. 12).

[0381] For example, the control device 1000 may select (switch) the tracking result to be output to the robot control unit 100 from among the results of the 2D tracking process and the results of the tracking process by the 3D tracking unit 303 (hereinafter referred to as "3D tracking process" as appropriate) based on a predetermined determination condition. In other words, the control device 1000 selects the tracking process results to be output to the robot control unit 100 from the results of the 2D tracking process and the results of the 3D tracking process generated at predetermined time intervals, sequentially based on a predetermined determination condition, and outputs them to the robot control unit 100. Here, the predetermined determination condition includes, for example, the number of feature regions of the object extracted by the 2D tracking process, the change over time in the position and orientation of the object calculated by the 2D tracking process, the difference between the position and orientation calculated by the 2D tracking process and the position and orientation calculated by the 3D tracking process, and the like.

[0382] For example, the control device 1000 detects the number of feature regions extracted by the 2D tracking process, and if the detected number of feature regions is less than a predetermined number, the result of the 3D tracking process is output to the robot control unit 100. This is because if the number of feature regions is small, the accuracy of estimating the position and orientation of the target object by the 2D tracking process decreases, so the control device 1000 outputs the result of the 3D tracking process, which has a higher accuracy of estimating the position and orientation of the target object than the 2D tracking process, to the robot control unit 100.

[0383] Also, for example, the control device 1000 calculates the change over time of the position and posture of the object calculated by the 2D tracking process. In the control device 1000, the change over time of the calculated position and posture of the object is separated into the change over time on each axis of the coordinate system of the robot arm (a coordinate system defined by the X-axis, the Y-axis, and the Z-axis), and if the displacement over time on the X-axis, the change over time on the Y-axis, and the change over time on the Z-axis are all greater than a predetermined threshold, the result of the 3D tracking process is output to the robot control unit 100. This is because, when the position and posture of the object change significantly three-dimensionally, the estimation accuracy of the position and posture of the object by the 2D tracking process decreases, and the control device 1000 outputs the result of the 3D tracking process, which has a higher estimation accuracy of the position and posture of the object than the 2D tracking process, to the robot control unit 100.

[0384] Also, for example, in the control device 1000, the difference between the position and orientation calculated by the 2D tracking process and the position and orientation calculated by the 3D tracking process is calculated, and if the difference is greater than a predetermined threshold, the result of the 3D tracking process is output to the robot control unit 100. This is because, if the difference between the position and orientation calculated by the 2D tracking process and the position and orientation calculated by the 3D tracking process is large, it is considered that the result of the 2D tracking process, in which the estimation accuracy of the target object's position and orientation by the 3D tracking process is low, contains many errors.

[0385] In addition, the control device 1000 may select the result to be output to the robot control unit 100 based on at least one of the predetermined judgment conditions, namely, the number of feature regions of the object extracted by the 2D tracking process, the change over time in the position and posture of the object calculated by the 2D tracking process, and the difference between the position and posture calculated by the 2D tracking process and the position and posture calculated by the 3D tracking process. In addition, when the result to be output to the robot control unit 100 is selected based on a plurality of judgment conditions among the predetermined judgment conditions, for example, when the control device 1000 judges that the selection result based on at least one of the plurality of judgment conditions is the result for which the result of the 3D tracking process is output to the robot control unit 100, the result of the 3D tracking process may be output to the robot control unit 100 regardless of the selection result based on the other judgment conditions. As described above, the control device 1000 selects the results of the tracking processing to be output to the robot control unit 100 based on predetermined judgment conditions, and can switch between processing that prioritizes processing speed (2D tracking) and processing that prioritizes estimation accuracy (3D tracking) depending on the type of object and changes in its position and posture, thereby performing appropriate tracking processing.

[0386] Note that the processes other than the process of selecting the result of the 2D tracking process and the result of the 3D tracking process based on a predetermined determination condition by the control device 1000 are similar to the processes described above, and therefore will not be described again.

[0387] The second modified example described above can also be applied to the tracking unit 300′ shown in FIG. 20. In other words, the tracking unit 300′ can also adopt the same aspect as the second modified example described above. When the second modified example described above is applied to the tracking unit 300′ shown in FIG. 20, as described above, each of the first matching unit 201 and the second matching unit 202 may narrow down the range in which 3D matching should be performed based on the result of 2D matching, and perform 3D matching using shape data corresponding to the narrowed down range (see FIG. 10). With this method, each of the first matching unit 201 and the second matching unit 202 can perform 3D matching at high speed.

[0388] Third variant In the first and second embodiments described above, for example, which of the output of the detection device 320 and the output of the detection device 330 is input to the tracking unit 300 is determined according to a comparison result of the matching rates by the matching processing unit 200 (see FIG. 9). However, the control device 1000 does not have to have the matching processing unit 200. In this case, the control device 1000 may perform the following tracking process.

[0389] As described above, the detection device 320 of the robot 3 is configured to be able to detect at least a portion of the circuit board T from a wide range when the circuit board T and the light irradiation device 60 are relatively far apart, and the detection device 330 is configured to be able to detect a portion of the circuit board T with high accuracy when the circuit board T and the light irradiation device 60 become relatively close to each other, so that the light irradiation device 60 moves closer to the portion to irradiate the processing light L onto the portion of the circuit board T (e.g., the specified portion described above).

[0390] For example, the control device 1000 may control the drive unit 311 so that the light irradiation device 60 and the detection devices 320 and 330 approach the circuit board T based on at least one of image data and shape data generated by the detection device 320, also referred to as a first imaging unit, detecting light from the circuit board T, and when the light irradiation device 60 and the detection devices 320 and 330 approach the circuit board T, for example, to a predetermined distance, the control unit 1000 may control the galvanometer mirror 61 so that the processing light L from the light irradiation device 60, which displaces together with the detection device 330, is irradiated to the same position on the circuit board T based on at least one of image data and shape data generated by the detection device 330, also referred to as a second imaging unit, detecting light from the circuit board T and which changes as the detection device 330 displaces.

[0391] In other words, when the circuit board T and the light irradiation device 60 are relatively far apart, the control device 1000 controls the drive unit 311 of the robot arm 310 so that the light irradiation device 60 approaches the circuit board T based on the output of the detection device 320 (e.g., at least one of image data and shape data), and when the circuit board T and the light irradiation device 60 are relatively close, the control device 1000 controls the drive unit 311 so that the light irradiation device 60 is in a desired position and posture based on the output of the detection device 330 (e.g., at least one of image data and shape data), and further controls the orientation of the galvanometer mirror 61 so that the processing light L irradiated from the light irradiation device 60 is irradiated to the same position on the circuit board T.

[0392] For example, the desired position and posture of the robot 3 are the relative position and posture of the light irradiation device 60 with respect to the circuit board T that enable the processing light L irradiated from the light irradiation device 60 to appropriately melt the solder on the circuit board T. Note that the control device 1000 may control the drive unit 311 based on the output of the detection device 330 so that the light irradiation device 60 approaches the circuit board T even closer.

[0393] For example, when the light irradiation device 60 of the robot 3 and the like are relatively far away from the circuit board T, the control device 1000 performs CAD matching (corresponding to the above-mentioned 2D matching) of the object using image data output from the detection device 320, and identifies the position of the object in the image shown by the image data. Next, based on the identified position of the object, the control device 1000 determines a range (e.g., corresponding to range A in FIG. 10(b)) in which CAD matching (the above-mentioned 3D matching) of the object should be performed using the shape data output from the detection device 320. Then, the control device 1000 performs CAD matching of the object using the shape data corresponding to the determined range.

[0394] At this time, the control device 1000 calculates a matching rate related to CAD matching of the object using image data, and a matching rate related to CAD matching of the object using shape data.

[0395] In parallel with the above-mentioned matching, the control device 1000 performs tracking processing similar to the tracking processing performed in the tracking unit 300 described above, using the image data and shape data output from the detection device 320. For this reason, the control device 1000 controls the drive unit 311 based on the image data and shape data output from the detection device 320 so that the light irradiation device 60 and the like approach the circuit board T.

[0396] As the light irradiation device 60 etc. approaches the circuit board T, the matching rate of the target object using the image data and shape data output from the detection device 320 decreases. This is because, for example, the image of the circuit board T captured by the cameras 21 and 22 of the detection device 320 becomes blurred, or the entire circuit board T does not fit within the field of view of the cameras 21 and 22, reducing the number of feature regions during matching.

[0397] When the matching rate of the CAD matching of the object using the image data and shape data output from the detection device 320 becomes equal to or lower than the threshold value, the control device 1000 performs CAD matching of the object using the image data and shape data output from the detection device 330 instead of the image data and shape data output from the detection device 320. In addition, the control device 1000 performs a tracking process similar to the tracking process performed in the tracking unit 300 described above, using the image data and shape data output from the detection device 330 instead of the image data and shape data output from the detection device 320. As a result, the control device 1000 controls the drive unit 311 so that the light irradiation device 60 and the like are brought closer to the circuit board T based on the image data and shape data output from the detection device 330 instead of the image data and shape data output from the detection device 320.

[0398] The control device 1000 may not use the result of CAD matching of the object using image data to determine the range in which CAD matching of the object using shape data should be performed. The control device 1000 may perform the above matching and tracking using only one of image data and shape data. When the control device 1000 performs the above matching and tracking using only shape data, the detection devices 320 and 330 may not generate image data (in other words, may generate only shape data). When the control device 1000 performs the above matching and tracking using only image data, the detection devices 320 and 330 may not generate shape data (in other words, may generate only image data). In this case, the detection devices 320 and 330 may be equipped with only a single camera.

[0399] The control device 1000 may not calculate the matching rate. In this case, the control device 1000 may perform tracking using the image data and shape data output from the detection device 330 instead of the image data and shape data output from the detection device 320 when the degree of blur of the image of the object captured by the cameras 21 and 22 of the detection device 320 becomes equal to or greater than a threshold. Alternatively, the control device 1000 may perform tracking using the image data and shape data output from the detection device 330 instead of the image data and shape data output from the detection device 320 when the number of feature points in matching using the image data and shape data output from the detection device 320 becomes equal to or less than a threshold.

[0400] <Third embodiment> A third embodiment will be described with reference to Fig. 21 to Fig. 24. In this embodiment, a welding system including a robot that performs laser welding is given. In the third embodiment, the description overlapping with the first embodiment will be omitted, and the same reference numerals will be used to indicate the common parts in the drawings, and only the fundamental differences will be described with reference to Fig. 21 to Fig. 24.

[0401] A laser welding system according to the third embodiment will be described with reference to Fig. 21 to Fig. 24. In Fig. 21, the laser welding system is a system that welds, for example, parts T1 and T2. The laser welding system includes a robot 5 and a robot 6.

[0402] Here, "welding" is a concept including, for example, fusion welding, brazing, build-up welding, etc. In the case of fusion welding, for example, parts T1 and T2 are metal members (i.e., base material). In the case of brazing, for example, a metal (i.e., brazing material / furnace material) having a lower melting point than parts T1 and T2 is placed between parts T1 and T2 (not shown). In the case of build-up welding, a metal for build-up may be supplied, for example, from a robot (not shown) different from robots 5 and 6.

[0403] Examples of the irradiation position of the laser light (in other words, the welding point) include the boundary between parts T1 and T2 that occurs when an end of part T1 and an end of part T2 are butted together, the boundary between parts T1 and T2 that occurs when one of parts T1 and T2 is placed upright on the other of parts T1 and T2 (see FIG. 20), the boundary that occurs when at least a part of one of parts T1 and T2 is overlapped with at least a part of the other of parts T1 and T2, etc. Note that at these boundaries, parts T1 and T2 do not have to be in contact (in other words, there may be a gap between parts T1 and T2).

[0404] The robot 5, which may also be referred to as a processing device, has a holding device 50 capable of holding, for example, a part T2, and detection devices 520 and 530 for detecting, for example, light from the part T2, and has a robot arm 510 having a drive unit 511 for moving the holding device 50 and the detection devices 520 and 530.

[0405] The robot 6, which may be called a processing device, is a robot that irradiates a target (here, a welding point) with laser light as processing light. The robot 6 has a light irradiation device 60 that irradiates laser light as processing light, detection devices 620 and 630 that detect light from the target, and a robot arm 610 that is provided with the light irradiation device 60 and the detection devices 620 and 630 and has a drive unit 611 that moves the light irradiation device 60 and the detection devices 620 and 630.

[0406] Here, the detection devices 520 and 630 correspond to the above-mentioned detection device 320. The detection devices 520 and 620 may have the same configuration as the detection device 320. Moreover, the detection devices 530 and 630 correspond to the above-mentioned detection device 330. The detection devices 530 and 630 may have the same configuration as the detection device 330. Note that the light irradiation device 60 does not need to include a scanning unit such as the galvanometer mirror 61.

[0407] In FIG. 21, the detection device 520 is disposed on the arm of the robot arm 510, the detection device 530 is disposed on the wrist of the robot arm 510, the detection device 620 is disposed on the arm of the robot arm 610, and the detection device 630 is disposed on the wrist of the robot arm 610. However, the arrangement of the detection devices 520, 530, 620, and 630 is not limited to this. Furthermore, the robot 5 may include other detection devices in addition to the detection devices 520 and 530 (i.e., the robot 5 may include three or more detection devices). The robot 5 may include only one of the detection devices 520 and 530. Similarly, the robot 6 may include other detection devices in addition to the detection devices 620 and 630 (i.e., the robot 6 may include three or more detection devices). The robot 6 may include only one of the detection devices 620 and 630.

[0408] The laser welding system includes a control device 1000 (see Figures 22 and 23) that (i) controls a holding device 50 serving as an end effector of a robot 5 so as to hold, for example, part T2 at a predetermined position on part T1, and (ii) controls a drive unit 611 of a robot arm 610 so as to bring a light irradiation device 60 serving as an end effector of a robot 6 close to parts T1 and T2 based on the detection results of at least one of detection devices 620 and 630, and controls the light irradiation device 60 so as to weld parts T1 and T2.

[0409] The control device 1000 may control the drive unit 511 of the robot 5 based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 520 and 530, so that, for example, part T2 held by the holding device 50 that displaces with the displacement of at least one of the detection devices 520 and 530 maintains a predetermined position on part T1.

[0410] The control device 1000 may control the orientation of the galvanometer mirror 61 of the robot 6 based on at least one of image data and shape data that changes with the displacement of at least one of the detection devices 620 and 630 so that the target (here, the welding point) is irradiated with laser light as processing light from the light irradiation device 60, which displaces with the displacement of at least one of the detection devices 620 and 630.

[0411] The control device 1000 may be a device different from the robot 5, or may constitute a part of the robot 5 (in other words, the robot 5 may be equipped with the control device 1000). Similarly, the control device 1000 may be a device different from the robot 6, or may constitute a part of the robot 6 (in other words, the robot 6 may be equipped with the control device 1000).

[0412] At least one of the robot arms 510 and 610 may be mounted on, for example, an AGV (Automatic Guided Vehicle). In this case, the control device 1000 may control at least one of the driving units of at least one of the robot arms 510 and 610, the end effector of at least one of the robot arms 510 and 610, and the driving unit of the AGV, based on information on the position and posture of the target object acquired by the above-mentioned matching process and tracking process described below.

[0413] (Movement of each robot) The operation of each of the robots 5 and 6 will be described with reference to the flowchart in Fig. 24. In the following description, the part T2 shown in Fig. 21 is assumed to have a flat plate shape extending toward the depth direction of the page. The laser light as the processing light from the light irradiation device 60 is assumed to be irradiated onto at least a part of the boundary between the parts T1 and T2 as the target objects.

[0414] Robot 5 For example, the holding device 50 includes a gripper whose tips can be opened and closed. Note that the control device 1000 may perform calibration of the holding device 50 before performing the processes of steps S151 to S155 below.

[0415] It is assumed that the detection device 530 and the holding device 50 are provided on the robot arm 510 in a positional relationship such that the tip of the holding device 50 (i.e., in the case of a gripper, the tip of the gripper that comes into contact with the part T2 when holding the part T2) is within the field of view of each of the cameras 31 and 32 of the detection device 530 having the same configuration as the detection device 330. For ease of explanation, the cameras 31 and 32 will be referred to as the cameras 31 and 32 of the detection device 530 as an example in which the detection device 530 has the same cameras as the detection device 530.

[0416] The control device 1000 performs the above-mentioned matching process to calibrate the holding device 50 using the shape data output from the detection device 530 when the holding device 50 is not holding the component T2 and the CAD data of the holding device 50, and calculates in advance the position and orientation of the holding device 50 (for example, the position and orientation of the tip of the gripper included in the field of view of the cameras 31 and 32 of the detection device 530). That is, the control device 1000 calculates in advance the position and orientation of the holding device 50 in the coordinate system of the robot arm 510 based on the shape data of at least a part of the holding device 50 (for example, the tip of the gripper).

[0417] Note that the calibration of the holding device 50 is not limited to using the detection device 530, and the control device 1000 may use shape data or image data output from the detection device 520. In this case, it is assumed that the detection device 520 and the holding device 50 are provided on the robot arm 510 in a positional relationship such that the tip of the holding device 50 is within the field of view of each of the cameras 21 and 22 of the detection device 520 having the same configuration as the detection device 320.

[0418] In the processes of steps S151 to S155 below, the position and orientation of the holding device 50 with respect to the detection device 530 may change due to the holding device 50 coming into contact with a predetermined object, etc. In this case, the control device 1000 can detect the change in the position and orientation of the holding device 50 with respect to the detection device 530 based on a partial change of the holding device 50 in the image data or shape data output from the detection device 530 (for example, a partial change of the holding device 50 on the image). When a change in the position and orientation of the holding device 50 with respect to the detection device 530 is detected, the control device 1000 may execute calibration.

[0419] Before performing the process of step S152 described later, the control device 1000 controls the drive unit 511 of the robot arm 510 and the holding device 50 so that the holding device 50 of the robot 5 appr...

Claims

1. A soldering device that irradiates processing light to melt solder arranged on a circuit board, a light irradiation device having a galvanometer mirror and irradiating the processing light via the galvanometer mirror; a detection device that detects light from the circuit board and generates at least one of image data and shape data; a robot arm provided with the light irradiation device and the detection device, the robot arm having a drive unit for moving the light irradiation device and the detection device; a control device that controls an orientation of the galvanometer mirror based on the at least one of the data that changes with the displacement of the detection device so that the processing light from the light irradiation device that displaces together with the detection device is irradiated at the same position; and A soldering apparatus comprising:

2. The control device controls the drive unit to stop driving the drive unit, The control device controls the orientation of the galvanometer mirror based on the at least one data that changes with the displacement of the detection device after the drive of the drive unit is stopped, so that the processing light from the light irradiation device that displaces together with the detection device is irradiated to the same position.

2. The soldering apparatus according to claim 1 .

3. 3. The soldering apparatus according to claim 1, wherein the control device controls the drive unit so that the light irradiation device and the detection device are moved, and controls the orientation of the galvanometer mirror so that the processing light from the light irradiation device moved by the robot arm is irradiated to the same position based on data of at least one of the data that changes with the displacement of the detection device moved by the robot arm.

4. The soldering apparatus according to any one of claims 1 to 3, characterized in that the control device controls the orientation of the galvanometer mirror based on at least one of the data that changes with the displacement of the detection device so that the irradiation position of the processing light from the light irradiation device, which displaces together with the detection device, is maintained at a first position and then at a second position different from the first position.

5. The control device controls the drive unit to stop driving the drive unit, The control device controls an orientation of the galvanometer mirror based on the at least one data that changes in accordance with the displacement of the detection device after the drive of the drive unit is stopped, so that an irradiation position of the processing light from the light irradiation device, which displaces together with the detection device, is maintained at the first position and then at the second position.

5. The soldering apparatus according to claim 4, wherein the soldering apparatus comprises:

6. The soldering apparatus of claim 4 or 5, characterized in that the control device controls the drive unit to move the light irradiation device and the detection device, while controlling the orientation of the galvanometer mirror so that the irradiation position of the processing light from the light irradiation device moved by the robot arm is maintained at the first position and then at the second position based on at least one of the data that changes with the displacement of the detection device moved by the robot arm.

7. The soldering apparatus according to any one of claims 1 to 6, characterized in that the control device controls the drive unit of the robot arm so that the light irradiation device and the detection device approach the circuit board based on at least one of the data, and when the light irradiation device and the detection device approach the circuit board to a predetermined distance, controls the orientation of the galvanometer mirror based on the at least one of the data that changes in accordance with the displacement of the detection device so that the processing light from the light irradiation device, which displaces together with the detection device, is irradiated to the same position.

8. The detection device has a first imaging unit and a second imaging unit having a narrower field of view than the first imaging unit, The control device controls the drive unit so that the light irradiation device and the detection device approach the circuit board based on the at least one of the data generated by the first imaging unit detecting the light from the circuit board, and when the light irradiation device and the detection device approach the circuit board to a predetermined distance, controls an orientation of the galvanometer mirror so that the processing light from the light irradiation device, which displaces together with the second imaging unit, is irradiated to the same position based on the at least one of the data generated by the second imaging unit detecting the light from the circuit board and changing with the displacement of the second imaging unit when the light irradiation device and the detection device approach to the circuit board to a predetermined distance.

8. A soldering apparatus according to claim 1, wherein the soldering apparatus comprises a first electrode and a second electrode.

9. 9. The soldering apparatus according to claim 1, wherein the circuit board comprises a circuit film on which a circuit is formed and a substrate.

10. 10. The soldering apparatus according to claim 1, wherein the circuit board has a three-dimensional shape.

11. 11. The soldering apparatus according to claim 1, wherein the control device performs an inspection related to soldering based on at least one of image data and shape data generated by the detection device.

12. 12. The soldering apparatus according to claim 11, wherein the control device judges whether the quality of the soldering is good or bad as the inspection related to the soldering.

13. 12. The soldering apparatus according to claim 11, wherein the control device displays the result of the inspection regarding the soldering on a display device.

14. 14. The soldering apparatus according to claim 11, wherein the control device displays at least one of image data and shape data used in the inspection relating to the soldering on a display device.

15. A processing device that irradiates a processing light onto an object, A light irradiation device that irradiates the processing light; A detection device for detecting light from the object; a moving device provided with the light irradiation device and the detection device, the moving device having a drive unit that moves the light irradiation device and the detection device; (i) a control device that controls the drive unit based on a detection result of the detection device, and (ii) performs an inspection related to processing by irradiation of the processing light based on the detection result of the detection device; A processing apparatus comprising:

16. 16. The processing device according to claim 15, wherein the control device judges whether or not quality of the processing is good as an inspection related to the processing performed by irradiating the processing light.

17. 17. The processing apparatus according to claim 15, wherein the control device displays a result of an inspection relating to processing by irradiation of the processing light on a display device.

18. 18. The processing apparatus according to claim 15, wherein the processing includes soldering an element to a circuit board.

19. 18. The processing apparatus according to claim 15, wherein the processing includes welding of metal members together.

20. A soldering device that irradiates processing light to melt solder arranged on a circuit board, a light irradiation device having a galvanometer mirror and irradiating the processing light via the galvanometer mirror; a detection device that detects light from the circuit board and generates at least one of image data and shape data; a robot arm provided with the light irradiation device and the detection device, the robot arm having a drive unit that moves the light irradiation device and the detection device; a control device that controls an orientation of the galvanometer mirror based on the at least one of the data that changes with the displacement of the detection device so that an irradiation position of the processing light from the light irradiation device that displaces together with the detection device changes over time in a region to be irradiated with the processing light; A soldering apparatus comprising:

21. A soldering system for soldering an element to a circuit board, comprising: a first robot arm including a detection device that detects light from the circuit board and generates at least one of image data and shape data, and a solder discharging device that discharges solder, the first robot arm having a drive unit that moves the detection device and the solder discharging device; a second robot arm provided with a holding device capable of holding the element and having a drive unit for moving the holding device; a third robot arm provided with a light irradiation device capable of irradiating processing light for melting solder, the third robot arm having a drive unit for moving the light irradiation device; (i) based on the data generated by the detection device, controlling the drive unit of the first robot arm to bring the solder dispensing device closer to the circuit board, and controlling the solder dispensing device to place solder on the circuit board; (ii) based on the data generated by the detection device, detecting a state of a location to be irradiated with processing light; (iii) controlling the holding device to place the element on the circuit board via the placed solder; and (iv) determining conditions for the processing light based on the detected state of the location to be irradiated with the processing light, and controlling the light irradiation device to melt the placed solder. A soldering system comprising:

22. 22. The soldering system according to claim 21, wherein the conditions of the processing light include at least one of the following conditions: intensity of the processing light, spot size of the processing light, irradiation time of the processing light, and irradiation range of the processing light.

23. A soldering system for soldering an element to a circuit board, comprising: a first robot arm provided with a solder discharging device that discharges solder and having a drive unit that moves the solder discharging device; a second robot arm including a detection device that detects light from the circuit board and generates at least one of image data and shape data, and a holding device that can hold the element, the second robot arm having a drive unit that moves the detection device and the holding device; a third robot arm provided with a light irradiation device capable of irradiating processing light for melting solder, the third robot arm having a drive unit for moving the light irradiation device; (i) controlling the solder dispenser to deposit solder on the circuit board; and (ii) a control device that controls the drive unit of the second robot arm to bring the holding device closer to the circuit board based on the data generated by the detection device, (iii) detects the state of the area to be irradiated with processing light based on the data generated by the detection device, (iv) controls the holding device so that the element is placed on the circuit board via the placed solder, and (v) determines conditions for the processing light based on the detected state of the area to be irradiated with the processing light, and controls the light irradiation device to melt the placed solder; A soldering system comprising:

24. 24. The soldering system according to claim 23, wherein the processing conditions of the processing light include at least one of the following conditions: intensity of the processing light, spot diameter of the processing light, and irradiation position of the processing light on the object to be irradiated with the processing light.

25. A processing device that irradiates a processing light onto an object, a light irradiation device having a scanning unit and irradiating the processing light via the scanning unit; A detection device for detecting light from the object; a moving device provided with the light irradiation device and the detection device, the moving device having a drive unit that moves the light irradiation device and the detection device; a control device that controls the scanning unit based on a detection result of the detection device; A processing apparatus comprising:

26. The processing apparatus according to claim 25, characterized in that the control device controls the scanning unit based on the detection result that changes with the displacement of the detection device so that the processing light from the light irradiation device, which displaces together with the detection device, is irradiated to the same position.

27. The processing apparatus according to claim 25 or 26, characterized in that the control device controls the drive unit to stop driving of the drive unit, and controls the scanning unit so that the processing light from the light irradiation device, which displaces together with the detection device, is irradiated to the same position based on the detection result that changes with the displacement of the detection device after the drive unit stops driving.

28. The processing apparatus according to any one of claims 25 to 27, characterized in that the control device controls the drive unit so that the light irradiation device and the detection device are moved, and controls the scanning unit so that the processing light from the light irradiation device moved by the moving device is irradiated to the same position based on the detection result that changes with the displacement of the detection device moved by the moving device.

29. The processing apparatus according to any one of claims 25 to 28, characterized in that the control device controls the scanning unit based on the detection result that changes with the displacement of the detection device so that the irradiation position of the processing light from the light irradiation device, which displaces together with the detection device, is maintained at a first position and then at a second position different from the first position.

30. The processing apparatus according to claim 29, characterized in that the control device controls the driving unit to stop driving of the driving unit, and controls the scanning unit based on the detection result that changes in accordance with the displacement of the detection device after the driving of the driving unit is stopped, so that the irradiation position of the processing light from the light irradiation device, which displaces together with the detection device, is maintained at the first position and then at the second position.

31. The processing apparatus according to claim 29, characterized in that the control device drives the drive unit to move the light irradiation device and the detection device, while controlling the scanning unit based on the detection result that changes with the displacement of the detection device moved by the moving device, so that the irradiation position of the processing light from the light irradiation device moved by the moving device is maintained at the first position and then at the second position.

32. 26. The processing apparatus according to claim 25, wherein the control device controls the scanning unit so that an irradiation position of the processing light is displaced over time on the object based on a detection result of the detection device.

33. The processing apparatus according to claim 32, characterized in that the control device controls the driving unit to stop driving of the driving unit, and controls the scanning unit so that the irradiation position of the processing light from the light irradiation device, which displaces together with the detection device, displaces over time on the object based on the detection result that changes in accordance with the displacement of the detection device after the driving of the driving unit has stopped.

34. The processing apparatus according to claim 32 or 33, characterized in that the control device controls the drive unit so that the light irradiation device and the detection device are moved, and controls the scanning unit so that the irradiation position of the processing light from the light irradiation device moved by the moving device is displaced over time on the object based on the detection result that changes with the displacement of the detection device moved by the moving device.

35. The processing apparatus according to any one of claims 25 to 34, characterized in that the control device calculates at least one of a position and an orientation of at least a part of the object based on the detection result, and controls the scanning unit based on the position and / or the orientation.

36. At least a portion of the object includes a marker provided on the object; The control device calculates at least one of a position and an orientation of the marker based on the detection result, and controls the scanning unit based on the position and / or the orientation.

36. The processing device according to claim 25,

37. 37. The processing apparatus according to claim 25, wherein the detection device includes at least one imaging device.

38. 37. The processing apparatus according to claim 25, wherein the detection device includes a stereo camera consisting of two image pickup devices.

39. 39. The processing apparatus according to claim 25, wherein the detection device includes a plurality of imaging devices each having a different field of view.

40. 40. The processing apparatus according to claim 37, wherein the detection device captures an image of the object with the imaging device, and generates at least one of image data and shape data as the detection result.

41. the detection device includes a projection device that projects structured light having a predetermined intensity distribution; The detection device captures an image of the object onto which the structured light is projected by the imaging device, and generates the shape data.

41. The processing device according to any one of claims 37 to 40.

42. The processing apparatus according to any one of claims 37 to 41, characterized in that at least a portion of the optical path of the processing light in the light irradiation device and at least a portion of the optical path of the light from the object in the imaging device are a common optical path.

43. 42. The processing apparatus according to claim 37, wherein an optical path of the processing light in the light irradiation device and an optical path of the light from the object in the imaging device are different from each other.

44. the object includes a circuit board; The processing light melts the solder arranged on the circuit board.

44. The processing device according to any one of claims 25 to 43.

45. The processing apparatus according to claim 43, characterized in that the control device calculates at least one of a position and an attitude of at least one of a portion of the circuit board and an element arranged on the circuit board based on the detection result, and controls the scanning unit based on at least one of the position and the attitude.

46. 46. ​​The processing device according to claim 44 or 45, wherein at least a portion of the circuit board includes at least one of a marker provided on the circuit board and a solder pad provided on the circuit board.

47. 47. The processing device according to claim 44, wherein the circuit board is made up of a circuit film on which a circuit is formed and a substrate.

48. 48. The processing device according to claim 44, wherein the circuit board has a three-dimensional shape.

49. 49. The processing apparatus according to claim 44, wherein the control device controls the scanning unit based on a detection result of the detection device so as to melt the solder arranged on the inclined surface on the circuit board.

50. The object includes a metal member to be used for welding, The processing light is irradiated onto the metal member.

44. The processing device according to any one of claims 25 to 43.

51. The processing apparatus according to claim 50, characterized in that the control device calculates at least one of a position and an attitude of at least a part of the metal component based on the detection result, and controls the scanning unit based on the position and / or the attitude.

52. The metal member includes a first base material and a second base material to be joined, The control device calculates at least one of a position and an orientation of a boundary between the first base material and the second base material based on the detection result, and controls the scanning unit based on the position and / or the orientation.

52. The processing device according to claim 50 or 51.

53. 53. The processing apparatus according to claim 25, wherein the control device controls the drive unit based on the detection result.

54. The processing apparatus according to any one of claims 25 to 53, characterized in that the control device controls the drive unit so that the light irradiation device and the detection device approach the object based on the detection result, and then controls the scanning unit so that the processing light is irradiated to the object based on the detection result.

55. The control device controls the driving unit to stop driving the driving unit when the light irradiation device and the detection device approach the target object to a predetermined distance based on the detection result, and controls the scanning unit so that the irradiation position of the processing light from the light irradiation device, which displaces together with the detection device, is maintained at a predetermined position on the target object based on the detection result that changes in accordance with the displacement of the detection device after the drive of the drive unit is stopped.

56. The detection device has a first imaging unit and a second imaging unit having a narrower field of view than the first imaging unit, The control device controls the drive unit so that the light irradiation device and the detection device approach the object based on a detection result obtained by the first imaging unit detecting the light from the object, and controls the scanning unit so that the processing light is irradiated onto the object based on a detection result obtained by the second imaging unit detecting the light from the object.

56. The processing device according to any one of claims 25 to 55.

57. 57. The processing apparatus according to claim 56, wherein at least one of the first imaging section and the second imaging section includes a stereo camera constituted by two imaging devices.

58. 58. The processing apparatus according to claim 25, wherein the detection device has a projection device that projects structured light having a predetermined intensity distribution.

59. 59. The processing apparatus according to claim 25, wherein the detection device generates at least one of image data and shape data as the detection result.

60. the scanning unit includes a galvanometer mirror, The control device controls the orientation of the galvanometer mirror.

60. The processing device according to any one of claims 25 to 59.

61. The light irradiation device has an fθ lens, The light irradiation device irradiates the processing light from the galvanometer mirror through the fθ lens.

61. The processing device according to claim 60.

62. 62. The processing apparatus according to claim 25, wherein the moving means is a robot arm.

63. A soldering system for soldering an element to a circuit board, comprising: a first moving device provided with a solder discharging device that discharges solder and having a drive unit that moves the solder discharging device; a second moving device provided with a holding device capable of holding the element and having a drive unit that moves the holding device; a third moving device including a light irradiating device that irradiates processing light for melting solder and a detection device that detects light from the circuit board, the third moving device having a drive unit that moves the light irradiating device and the detection device; (i) controlling the solder dispensing device so as to place solder on a predetermined portion of the circuit board, (ii) controlling the holding device so as to place the element on the circuit board via the placed solder, and (iii) controlling the drive section of the third moving device so as to bring the light irradiation device closer to the circuit board based on a detection result of the detection device, and controlling the light irradiation device so as to melt the placed solder; A soldering system comprising:

64. the first moving device is provided with a detection device that detects light from the circuit board; The control device controls the drive unit of the first moving device so that the solder dispensing device approaches the circuit board based on a detection result of the detection device of the first moving device.

64. The soldering system of claim 63.

65. 65. The soldering system according to claim 64, wherein the control device detects the state of a location to be irradiated with processing light based on a detection result of the detection device provided on the first moving device.

66. The soldering system of claim 65, wherein the control device determines conditions for the processing light based on the detected state of the location to be irradiated with the processing light, and controls the light irradiation device to melt the placed solder.

67. 67. The soldering system according to claim 66, wherein the processing conditions of the processing light include at least one of the intensity of the processing light, the spot diameter of the processing light, the irradiation time of the processing light, and the irradiation range of the processing light.

68. the detection device and the solder application device of the first moving device are provided on the first moving device in a positional relationship such that at least a part of the solder dispensing device is included in a field of view of the detection device of the first moving device; The control device estimates at least one of a position and an attitude of at least a part of the solder dispensing device based on a detection result of the detection device of the first moving device.

64. The soldering system of claim 63.

69. the second moving device is provided with a detection device that detects light from the circuit board; The control device controls the drive unit of the second moving device so that the holding device approaches the circuit board based on a detection result of the detection device of the second moving device.

69. A soldering system according to any one of claims 63 to 68.

70. 70. The soldering system of claim 69, wherein the control device detects the state of a location to be irradiated with processing light based on a detection result of the detection device provided on the second moving device.

71. The soldering system of claim 70, wherein the control device determines conditions for the processing light based on the detected state of the location to be irradiated with the processing light, and controls the light irradiation device so as to melt the placed solder.

72. 72. The soldering system according to claim 71, wherein the processing conditions of the processing light include at least one of the intensity of the processing light, the spot diameter of the processing light, the irradiation time of the processing light, and the irradiation range of the processing light.

73. the detection device and the holding device of the second moving device are provided on the second moving device in a positional relationship such that at least a part of the holding device is included in a field of view of the detection device of the second moving device, The control device estimates at least one of a position and an orientation of at least a part of the holding device based on a detection result of the detection device of the second mobile device.

70. The soldering system of claim 69.

74. the detection device and the holding device of the second moving device are provided on the second moving device in a positional relationship such that at least a part of the element held by the holding device is included in a field of view of the detection device of the second moving device, In a state in which the element is held by the holding device, the control device estimates at least one of a position and an orientation of at least a part of the holding device based on a detection result of the detection device of the second movable device.

74. A soldering system according to claim 69 or 73.

75. the detection device and the light irradiation device of the third movable device are provided on the third movable device in a positional relationship such that at least a part of the light irradiation device is included in a field of view of the detection device of the third movable device, The control device estimates at least one of a position and an orientation of at least a part of the light irradiation device based on a detection result of the detection device of the third mobile device.

64. The soldering system of claim 63.

76. The light irradiation device has a scanning unit, The light irradiation device irradiates the processing light via the scanning unit, The control device controls the scanning unit based on a detection result of the detection device of the third moving device.

76. A soldering system according to any one of claims 63 to 75.

77. The soldering system described in claim 76, characterized in that the control device controls the scanning unit based on the detection result that changes with the displacement of the detection device of the third moving device so that the processing light from the light irradiation device, which displaces together with the detection device, is irradiated to the same position.

78. A soldering system for soldering an element to a circuit board, comprising: a moving device including a solder dispensing device that dispenses solder, a holding device capable of holding the element, a light irradiating device that irradiates processing light that melts the solder, and a detection device that detects light from the circuit board, the moving device having a drive unit that moves the solder dispensing device, the holding device, the light irradiating device, and the detection device; (i) a control device that controls the drive unit so that the solder dispensing device, the holding device, the light irradiating device, and the detection device approach the circuit board, (ii) controls the solder dispensing device so that solder is placed on a predetermined portion of the circuit board, (iii) controls the holding device so that the element is placed on the circuit board via the placed solder, and (iv) controls the light irradiating device so as to melt the placed solder; A soldering system comprising:

79. The light irradiation device has a scanning unit, The light irradiation device irradiates the processing light via the scanning unit, The control device controls the scanning unit based on a detection result of the detection device.

79. The soldering system of claim 78.

80. The soldering system of claim 79, characterized in that the control device controls the scanning unit based on the detection result that changes with the displacement of the detection device so that the processing light from the light irradiation device, which displaces together with the detection device, is irradiated to the same position.

81. 81. The soldering system of claim 78, wherein the moving means is provided with a storage section for storing elements of different types, and a supply device for supplying a predetermined element from the storage section to the holding device.

82. A processing device that irradiates a processing light onto an object, A light irradiation device that irradiates the processing light; A detection device for detecting light from the object; a moving device provided with the light irradiation device and the detection device, the moving device having a drive unit that moves the light irradiation device and the detection device; a control device that controls the drive unit based on a detection result of the detection device; A processing apparatus comprising:

83. the object includes a circuit board; The processing light melts the solder arranged on the circuit board.

83. The processing device according to claim 82.

84. The processing apparatus according to claim 82 or 83, characterized in that at least a portion of the optical path of the processing light in the light irradiation device and at least a portion of the optical path of the light from the object in the detection device are a common optical path.

85. The processing apparatus according to any one of claims 82 to 84, characterized in that the control device controls the drive unit so that the light irradiation device and the detection device approach the object based on the detection result, and controls the light irradiation device so that irradiation of the processing light to the object is started when the light irradiation device and the detection device approach the object to a predetermined distance.

86. The detection device has a first imaging unit and a second imaging unit having a narrower field of view than the first imaging unit, The control device controls the drive unit so that the light irradiation device and the detection device approach the object based on a detection result obtained by the first imaging unit detecting the light from the object, controls the drive unit so that the light irradiation device and the detection device further approach the object based on a detection result obtained by the second imaging unit detecting the light from the object, and controls the light irradiation device so that irradiation of the processing light to the object is started when the light irradiation device and the detection device approach the object to a predetermined distance.

86. The processing device according to any one of claims 82 to 85.

87. 87. The processing apparatus according to claim 86, wherein at least one of the first imaging unit and the second imaging unit includes a stereo camera constituted by two imaging devices.

88. 88. The processing apparatus according to any one of claims 82 to 87, wherein the detection device has a projection device that projects structured light having a predetermined intensity distribution.

89. 89. The processing apparatus according to claim 82, wherein the detection device generates at least one of image data and shape data as the detection result.

Images