Assembling apparatus and assembling method

Abstract

To provide an assembling device capable of enhancing the degree of freedom of equipment for conveying or supporting components.SOLUTION: The assembly apparatus includes a first light source that projects a first line laser onto the first component and the second component, a second light source that projects a second line laser onto the first component and the second component from a direction different from an irradiation direction of the first line laser, a camera that images the first component and the second component, and an image processing device. The image processing device specifies the relative position and orientation of the first component with respect to the second component based on the projection positions of the first line laser and the second line laser on the first component in the image output from the camera and the projection positions of the first line laser and the second line laser on the second component in the image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an assembly apparatus and an assembly method. [Background technology]

[0002] Japanese Patent Application Laid-Open Publication No. 2022-136802 (Patent Document 1) discloses an electronic device assembly device that attaches a cable to a connector. The electronic device assembly device irradiates the cable and connector with slit light from an oblique direction that includes a component toward the cable attachment direction, and captures an image that includes the cable and connector. The electronic device assembly device detects the relative positional relationship between the cable and the connector based on height information obtained from the slit light and planar position information obtained from the image. The electronic device assembly device attaches the attachment portion of the cable to the connector based on the detected relative positional relationship. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-136802 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology disclosed in Patent Document 1 detects only the relative positional relationship between the cable and the connector, and therefore assumes that the postures of the cable and the connector are within a reference range. Therefore, the equipment (including jigs) that transport or support the component (cable or connector) is designed so that the posture of the component is within the reference range.

[0005] However, recent changes in manufacturing methods and materials have made parts more susceptible to unintended tilt. For example, because the boards on which connectors are mounted are manufactured using a layered manufacturing process, individual differences in the boards can cause unintended tilt in the connectors. Furthermore, when connectors are supplied by an automated guided vehicle or an autonomous transport robot, the orientation of the supplied connector can change depending on the individual differences in the automated guided vehicle or autonomous transport robot and its condition at the time of supply. The condition at the time of supply includes, for example, the parking position and orientation of the automated guided vehicle or autonomous transport robot. Furthermore, when a flexible cable is used as the cable, unintended tilt in the cable can occur due to the influence of its own weight, etc. Therefore, it is necessary to use equipment designed to further suppress unintended tilt in parts. This limits the flexibility of the equipment that can be used.

[0006] The present disclosure has been made in view of the above circumstances, and its purpose is to provide an assembly apparatus and an assembly method that can increase the degree of freedom of equipment that transports or supports parts. [Means for solving the problem]

[0007] An assembling device according to one aspect of the present disclosure assembles a first part to a second part. The assembling device includes a first light source that projects a first line laser onto the first part and a second part, a second light source that projects a second line laser onto the first part and the second part from a direction different from that of the first line laser, a camera that captures images of the first part and the second part, and an identification unit. The identification unit identifies the relative position and orientation of the first part with respect to the second part based on the projected positions of the first line laser and the second line laser on the first part in an image output from the camera and the projected positions of the first line laser and the second line laser on the second part in the image.

[0008] According to this disclosure, not only the relative positional relationship between a first part and a second part but also the relative orientation relationship between the first part and the second part can be identified. Therefore, the relative position and orientation can be used to correct assembly operations and inspect assembly conditions. This increases the tolerance for deviations in the orientation of the first part or the second part. As a result, the flexibility of equipment for transporting or supporting the first part or the second part is increased.

[0009] In the above disclosure, the assembly apparatus further includes a robot that holds the first component, and a controller that controls the robot so that the relative position and posture approach a target state.

[0010] According to this disclosure, the assembly apparatus can improve the assembly accuracy of the first part relative to the second part based on the relative position and orientation of the first part relative to the second part.

[0011] In the above disclosure, the robot has an end effector that holds a first part. The first light source, the second light source, and the camera are installed on the robot so as to move integrally with the end effector. According to this disclosure, the assembly device can be made smaller.

[0012] In the above disclosure, the assembly apparatus further includes an inspection unit that inspects whether the relative position and posture are within a reference range.

[0013] According to this disclosure, the user can understand the assembly state of the first part relative to the second part based on the inspection result.

[0014] In the above disclosure, the assembly device further includes an illumination device that irradiates illumination light onto the field of view of the camera.

[0015] According to this disclosure, a user can make the first line laser and the second line laser stand out in an image by adjusting the color and intensity of the illumination light from the illumination device.

[0016] An assembly method according to one aspect of the present disclosure assembles a first part to a second part. The assembly method includes: a first light source projecting a first line laser onto the first part and a second part; a second light source projecting a second line laser onto the first part and the second part from a direction different from that of the first line laser; and a camera capturing images of the first part and the second part. The assembly method further includes a processor determining a relative position and orientation of the first part with respect to the second part based on the projected positions of the first line laser and the second line laser on the first part in an image output from the camera and the projected positions of the first line laser and the second line laser on the second part in the image. This disclosure also increases the flexibility of equipment that transports or supports parts. [Effects of the Invention]

[0017] The assembly apparatus and assembly method according to the present disclosure can increase the degree of freedom of equipment for transporting or supporting parts. [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 2 is a side view of an example of an assembly apparatus according to an embodiment. [Figure 2] FIG. 1 is a front view of an example of an assembly apparatus according to an embodiment. [Figure 3] FIG. 1 is a schematic diagram illustrating an example of a hardware configuration of an image processing device. [Figure 4] 3 is a flowchart showing a main process flow of the assembly device according to the present embodiment. [Figure 5] FIG. 10 is a diagram showing an example of an image including a cable and a connector. [Figure 6] FIG. 10 is a diagram showing the positional relationship between a cable, a connector, and two line lasers. [Figure 7] FIG. 10 is a diagram illustrating a method for calculating coordinate values ​​x, y, and Rz. [Figure 8] FIG. 10 is a diagram illustrating a method for calculating a coordinate value z. [Figure 9] FIG. 10 is a diagram illustrating a method for calculating a coordinate value Rx. [Figure 10]FIG. 10 is a diagram illustrating a method for calculating a coordinate value Ry. [Figure 11] FIG. 10 is a diagram illustrating an example of the control of the robot performed in step S5. [Figure 12] FIG. 10 is a side view of an example of an assembly device according to a first modified example. [Figure 13] FIG. 10 is a front view of an example of an assembly device according to a first modified example. DETAILED DESCRIPTION OF THE INVENTION

[0019] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail with reference to the accompanying drawings, in which the same or corresponding parts in the drawings are designated by the same reference numerals and the description thereof will not be repeated.

[0020] §1 Application Examples An application example of the present invention will be described with reference to Figures 1 and 2. Figure 1 is a side view of an example of an assembling apparatus according to an embodiment. Figure 2 is a front view of the example of an assembling apparatus according to an embodiment.

[0021] The assembly device 1 shown in FIGS. 1 and 2 assembles a cable 2 to a connector 3. The cable 2 is an example of a "first component" in the present disclosure. The cable 2 is, for example, a flexible cable. Flexible cables include an FPC (Flexible Printed Circuit) or an FFC (Flexible Flat Cable). The connector 3 is an example of a "second component" in the present disclosure. The connector 3 is mounted on a substrate 4.

[0022] The assembly apparatus 1 includes light sources 10 and 12, a camera 14, a robot 16, a lighting device 18, an image processing device 20, a controller 30, and a frame 40.

[0023] 1 and 2, the Z axis is parallel to the optical axis 14b of the camera 14. In the example shown in FIGS. 1 and 2, the optical axis 14b of the camera 14 is parallel to the vertical direction. Therefore, the Z axis is parallel to the vertical direction. The negative direction of the Z axis is the vertical direction, pointing downward. The positive direction of the Z axis is the opposite direction to the vertical direction, pointing upward. The X axis and Y axis are perpendicular to the Z axis and parallel to the horizontal plane. The X axis and Y axis are perpendicular to each other.

[0024] The light source 10 projects a line laser 10a onto the cable 2 and the connector 3. The light source 12 projects a line laser 12a onto the cable 2 and the connector 3 from a direction different from the projection direction of the line laser 10a. The light sources 10 and 12 are arranged at a predetermined interval along the Y-axis direction so as to sandwich the camera 14. The light sources 10 and 12 irradiate the line lasers 10a and 12a by using a laser line generator lens to diverge collimated light in a fan shape along the X-axis direction. The color (wavelength) of the line lasers 10a and 12a is appropriately selected so as to be distinguishable from the color of the cable 2 and the connector 3. For example, the line lasers 10a and 12a have a red color. The light source 10 is an example of a "first light source" of the present disclosure. The light source 12 is an example of a "second light source" of the present disclosure. The line laser 10a is an example of a "first line laser" of the present disclosure. The line laser 12a is an example of a "second line laser" of the present disclosure.

[0025] The line lasers 10a and 12a are projected onto the upper surface 2a of the cable 2 and the upper surface 3a of the connector 3. If the upper surface 2a is flat, the line laser 10a projected onto the upper surface 2a is linear. Similarly, if the upper surface 3a is flat, the line laser 10a projected onto the upper surface 3a is linear.

[0026] Camera 14 is disposed at a fixed position so that the connector 3 and the surrounding area of ​​the connector 3 are included within its field of view. When the tip portion of cable 2 is assembled to connector 3, it will be positioned around connector 3. Therefore, camera 14 photographs cable 2 and connector 3 and outputs image 50 including cable 2 and connector 3. As shown in the figure, camera 14 is disposed above connector 3. Therefore, image 50 includes the top surface 2a of cable 2 and the top surface 3a of connector 3.

[0027] The lighting device 18 irradiates the field of view of the camera 14 with illumination light. As described above, the field of view of the camera 14 includes the connector 3 and the area around the connector 3. Therefore, the lighting device 18 irradiates the cable 2 and the connector 3 with illumination light. The color (wavelength) of the illumination light irradiated from the lighting device 18 is appropriately selected so as to avoid optical interference with the line lasers 10a, 12a. For example, if the line lasers 10a, 12a are red, the lighting device 18 irradiates white illumination light.

[0028] The illumination device 18 is, for example, a ring light or a bar light. In the example shown in Figures 1 and 2, a ring light is used as the illumination device 18. In this case, the illumination device 18 is preferably arranged so as to surround the camera 14.

[0029] The lighting device 18 is installed to make it easier to see the projection position of the line laser 10a in the image 50. If the projection positions of the line lasers 10a and 12a are visible in the image 50 acquired under an environment with only natural light, the lighting device 18 may be omitted.

[0030] The frame 40 supports the light sources 10 and 12 and the camera 14 at fixed positions. The frame 40 includes a lower frame 40a, an upper frame 40b, and a connecting frame 40c. The lower frame 40a is flat and arranged parallel to a horizontal plane. The board 4 on which the connector 3 is mounted is placed on the lower frame 40a. The upper frame 40b is arranged above the lower frame 40a. The upper frame 40b is, for example, rod-shaped extending in the Y-axis direction and supports the light sources 10 and 12 and the camera 14. The connecting frame 40c connects the lower frame 40a and the upper frame 40b. Alternatively, the board 4 on which the connector 3 is mounted may be transported to within the field of view of the camera 14 by, for example, an automated guided vehicle or an autonomous transport robot.

[0031] The robot 16 has an end effector 16a for holding the cable 2, and moves the end effector 16a so that the cable 2 is assembled to the connector 3. As the robot 16, for example, a vertical articulated robot, a horizontal articulated robot, a parallel link robot, or the like can be adopted.

[0032] The controller 30 controls the operation of the robot 16. Specifically, the controller 30 generates a command so that the relative position and posture of the cable 2 with respect to the connector 3 becomes a target state, and outputs the generated command to the robot 16.

[0033] The image processing device 20 performs image processing on the image 50 output from the camera 14. Specifically, the image processing device 20 identifies the relative position and orientation of the cable 2 with respect to the connector 3 based on the projection positions of the line lasers 10a, 12a on the cable 2 in the image 50 and the projection positions of the line lasers 10a, 12a on the connector 3 in the image 50. The method for identifying the relative position and orientation will be described in detail later.

[0034] According to this embodiment, not only the relative positional relationship between the connector 3 and the cable 2 but also the relative posture relationship between the connector 3 and the cable 2 can be identified. Therefore, it is possible to correct the operation of the robot 16 according to the relative position and posture. Alternatively, it is possible to inspect whether the cable 2 has been properly attached to the connector 3 according to the relative position and posture. This increases the tolerance for deviation in the posture of the connector 3 or the cable 2. As a result, the flexibility of the equipment for transporting or supporting the connector 3 or the cable 2 is increased.

[0035] For example, the user can employ a known automated guided vehicle or autonomous transport robot as a transport device that transports the connector 3 below the camera 14. Alternatively, the user can select an end effector 16a that holds the cable 2 from among various types of end effectors.

[0036] The relative position and orientation of the cable 2 with respect to the connector 3 can also be determined using a known three-dimensional vision system. However, three-dimensional vision systems are generally expensive. In contrast, the light sources 10, 12 and camera 14 used in this embodiment are inexpensive compared to three-dimensional vision systems. Therefore, the assembly apparatus 1 according to this embodiment can determine the relative position and orientation of the cable 2 with respect to the connector 3 at low cost.

[0037] Furthermore, conventionally, in order to improve the accuracy of assembling the cable 2 to the connector 3, a teaching operation is performed to define in detail the operation of the robot 16 that holds the cable 2. Generally, the teaching operation is time-consuming. In particular, when there are multiple types of cables 2 and connectors 3, a teaching operation is required for each type. However, according to this embodiment, the relative position and orientation of the cable 2 with respect to the connector 3 is used to correct the operation of the robot 16, thereby improving the accuracy of assembling the cable 2 to the connector 3. Therefore, the time and effort required for the teaching operation to define in detail the operation of the robot 16 that holds the cable 2 is reduced.

[0038] Alternatively, in the past, installing a force sensor on the robot 16 holding the cable 2 has been considered to improve the assembly accuracy of the cable 2 relative to the connector 3. However, force sensors are effective in improving assembly accuracy when the deviation between the position of the cable 2 relative to the connector 3 and the reference position is relatively small, but cannot improve the assembly accuracy of the cable 2 relative to the connector 3 when the deviation is large. In contrast, the assembly apparatus 1 according to the present embodiment can detect even large deviations from the reference position and correct the deviation by projecting line lasers 10a and 12a onto the connector 3 and the cable 2 within the field of view of the camera 14. Furthermore, force sensors have low accuracy in detecting forces on soft materials such as flat cables. Therefore, when using force sensors, the materials of the components to be assembled are limited. In contrast, the assembly apparatus 1 according to the present embodiment can determine the relative position and orientation between the components, regardless of the materials of the components to be assembled.

[0039] §2 Specific examples <Hardware configuration of image processing device> The image processing device 20 is typically a computer having a general-purpose architecture, and executes a pre-installed program (instruction code). Such a program is typically distributed in a state stored on various recording media, or is installed in the image processing device 20 via a network, etc.

[0040] When using such a general-purpose computer, an OS (Operating System) for executing basic computer processing may be installed in addition to the application for executing the image processing according to the present embodiment. In this case, the program according to the present embodiment may execute processing by calling necessary modules from among program modules provided as part of the OS in a predetermined sequence at a predetermined timing. In other words, the program according to the present embodiment itself may not include the above-mentioned modules, and may execute processing in cooperation with the OS. The program according to the present embodiment may also be in a form that does not include some of these modules.

[0041] Furthermore, the program according to the present embodiment may be provided by being incorporated into a part of another program. In this case, the program itself does not include the modules included in the other program to be combined as described above, and executes processing in cooperation with the other program. In other words, the program according to the present embodiment may be in a form incorporated into such other program. Note that some or all of the functions provided by the execution of the program may be implemented as dedicated hardware circuits.

[0042] Fig. 3 is a schematic diagram showing an example of the hardware configuration of an image processing device. As shown in Fig. 3, the image processing device 20 includes a CPU (Central Processing Unit) 201, which is an example of a processor, a main memory 202, a storage 203, a camera interface 204, an input interface 205, a display controller 206, a communication interface 207, and a data reader / writer 208. These components are connected to each other via a bus 209 so as to be able to communicate data with each other.

[0043] The CPU 201 loads the programs 230 installed in the storage 203 into the main memory 202 and executes them in a predetermined order to perform various calculations. The main memory 202 typically includes a volatile storage device such as a dynamic random access memory (DRAM), and stores images 50 acquired from the camera 14 in addition to the programs 230 read from the storage 203. The storage 203 includes, for example, a hard disk drive, a solid state drive, or the like.

[0044] The CPU 201 executes the program 230 to operate as the identification unit 210 or the inspection unit 211 .

[0045] The identifying unit 210 identifies the relative position and orientation of the cable 2 with respect to the connector 3 based on the projection positions of the line lasers 10a, 12a on the cable 2 in the image 50 and the projection positions of the line lasers 10a, 12a on the connector 3 in the image 50.

[0046] The inspection unit 211 inspects whether the relative position and orientation identified by the identification unit 210 is within a predetermined reference range. For example, the inspection unit 211 inspects whether the relative position and orientation identified from the image 50 captured by the camera 14 after the assembly operation by the robot 16 is completed is within the reference range. The reference range is a range within which the relative position and orientation of the cable 2 with respect to the connector 3 can be taken when the cable 2 is properly assembled to the connector 3. Thus, the inspection result of the inspection unit 211 indicates whether the cable 2 is properly assembled to the connector 3. If the relative position and orientation is not within the predetermined reference range, the inspection unit 211 may output an error notification.

[0047] The camera interface 204 mediates data transmission between the CPU 201 and the camera 14. The camera interface 204 is connected to the camera 14 and temporarily stores the image 50 received from the camera 14.

[0048] The input interface 205 mediates data transmission between the CPU 201 and the input device 212. That is, the input interface 205 accepts input information that the user inputs to the input device 212. For example, the input interface 205 may accept a request to display the result of the identification of the relative position and orientation by the identification unit 210.

[0049] The display controller 206 is connected to the display 213 and controls the screen of the display 213 so as to notify the user of the processing results of the CPU 201, etc. For example, the display 213 displays an image 50 received from the camera 14. Alternatively, the display 213 displays the relative position and orientation identified by the identification unit 210. Alternatively, the display 213 displays the inspection result by the inspection unit 211 or an error notification indicating that the relative position and orientation is not within a predetermined reference range.

[0050] The communication interface 207 mediates data transmission between the CPU 201 and an external device (for example, the controller 30). The communication interface 207 is typically implemented by Ethernet (registered trademark) or USB (Universal Serial Bus).

[0051] Data reader / writer 208 mediates data transmission between CPU 201 and memory card 215, which is a recording medium. That is, memory card 215 stores and distributes programs to be executed by image processing device 20, and data reader / writer 208 reads the programs from memory card 215. In addition, data reader / writer 208 writes images received from camera 14 and / or results of image processing to memory card 215 in response to an internal command from CPU 201. Note that memory card 215 may be a general-purpose semiconductor storage device such as SD (Secure Digital), a magnetic storage medium such as a flexible disk, or an optical storage medium such as a CD-ROM (Compact Disk Read Only Memory).

[0052] <Assembly equipment processing flow> 4 is a flowchart showing the main processing flow of the assembly apparatus according to this embodiment. First, in step S1, light source 10 projects line laser 10a (first line laser) onto cable 2 and connector 3. Then, in step S2, light source 12 projects line laser 12a (second line laser) onto cable 2 and connector 3. Note that steps S1 and S2 may be performed in parallel or sequentially. Steps S1 and S2 may be performed in response to a command from CPU 201 of image processing device 20.

[0053] In the next step S3, the camera 14 captures an image of the cable 2 and the connector 3. The camera 14 outputs the image 50 obtained by capturing the image to the image processing device 20. Step S3 may be performed in response to a command from the CPU 201 of the image processing device 20.

[0054] In the next step S4, the CPU 201 of the image processing device 20 determines the relative position and posture of the cable 2 with respect to the connector 3 based on the projection positions of the line lasers 10a, 12a on the cable 2 in the image 50 and the projection positions of the line lasers 10a, 12a on the connector 3 in the image 50.

[0055] If the cable 2 is not present in the field of view of the camera 14, the cable 2 is not included in the image 50. In this case, the CPU 201 determines that the relative position and orientation of the cable 2 with respect to the connector 3 cannot be identified.

[0056] In the next step S5, the assembly apparatus 1 performs a predetermined process based on the identified relative position and posture. After step S5, the process ends.

[0057] The predetermined process is, for example, control of the robot 16 by the controller 30. Specifically, the controller 30 controls the robot 16 holding the cable 2 so that the relative position and posture of the cable 2 with respect to the connector 3 approaches a predetermined target state. The target state may include multiple states (first state to Nth state), where N is an integer equal to or greater than 2. The Nth state is a state in which the cable 2 is properly assembled to the connector 3. In this case, steps S3 and S4 are repeated multiple times. Specifically, if the relative position and posture of the cable 2 with respect to the connector 3 has not reached the kth state (k is an integer from 1 to N-1), the controller 30 controls the robot 16 holding the cable 2 so that the relative position and posture approaches the kth state. If the relative position and posture of the cable 2 with respect to the connector 3 has reached the kth state, the controller 30 controls the robot 16 holding the cable 2 so that the relative position and posture approaches the k+1th state. If the relative position and posture of the cable 2 with respect to the connector 3 has reached the Nth state, the controller 30 controls the robot 16 to end holding the cable 2. This allows the robot 16 to assemble the cable 2 that it is holding into the connector 3.

[0058] Alternatively, the predetermined process may be an inspection of the state of assembly of the cable 2 to the connector 3. Specifically, the CPU 201 of the image processing device 20 inspects whether the relative position and orientation identified in step S4 is within a reference range. In this case, steps S1 to S5 are performed after the robot 16 has completed the assembly work of the cable 2 to the connector 3.

[0059] <Image example> Fig. 5 is a diagram showing an example of an image including a cable and a connector. The image 50 shown in Fig. 5 includes a cable 2 having a dark color. If the cable 2 has a dark color, the line lasers 10a, 12a projected onto the cable 2 become difficult to see. In the example shown in Fig. 5, the lighting device 18 illuminates the area on the cable 2 onto which the line lasers 10a, 12a may be projected in a whitish light. This makes the red line lasers 10a, 12a more easily visible.

[0060] In this way, the lighting device 18 is used to make it easier to see the projection position of the line laser 10a in the image 50. The type, placement position, color of the illumination light, and light intensity of the lighting device 18 are adjusted appropriately depending on the colors of the line lasers 10a and 12a, and the colors and glossiness of the cable 2 and the connector 3, etc.

[0061] <Method for determining relative position and orientation> A method for identifying the relative position and orientation of the cable 2 with respect to the connector 3 will be described with reference to FIGS. 2 and 6 to 10. FIG.

[0062] Fig. 6 is a diagram showing the positional relationship between the cable, the connector, and the two line lasers. As shown in Fig. 6, the line lasers 10a and 12a are projected so as to straddle the upper surface 2a of the cable 2 and the upper surface 3a of the connector 3. The line lasers 10a and 12a are projected at a predetermined interval in the Y-axis direction. As described above, the line lasers 10a and 12a are irradiated in a fan shape along the X-axis direction.

[0063] As shown in FIG. 2, the projection directions of the line lasers 10a and 12a are inclined with respect to an imaginary plane 14a including an optical axis 14b of the camera 14. The imaginary plane 14a is parallel to the divergence direction (X-axis direction) of the line lasers 10a and 12a. Angles θ1 and θ2 formed by the projection directions of the line lasers 10a and 12a and the imaginary plane 14a are preferably 45°. This simplifies calculations for determining the relative position and orientation. Furthermore, the projection directions of the line lasers 10a and 12a are preferably symmetrical with respect to the imaginary plane 14a.

[0064] The relative position and orientation of the cable 2 with respect to the connector 3 is expressed by three-dimensional coordinates (x, y, z, Rx, Ry, Rz). The three-dimensional coordinates (x, y, z, Rx, Ry, Rz) represent the amount of deviation from the state when the cable 2 is ideally assembled to the connector 3. Specifically, the coordinate value x indicates the amount of translation along the X-axis. The coordinate value y indicates the amount of translation in the Y-axis direction. The coordinate value z indicates the amount of translation in the Z-axis direction. The coordinate value Rx indicates the amount of rotation around the X-axis. The coordinate value Ry indicates the amount of rotation around the Y-axis. The coordinate value Rz indicates the amount of rotation around the Z-axis.

[0065] The three-dimensional coordinates (x, y, z, Rx, Ry, Rz) are calculated based on the positions of the cable 2 and connector 3 on the XY plane in the image 50 and height information of the cable 2 and connector 3 obtained from the projection positions of the line lasers 10a and 12a using the light section method. The calculation method for each coordinate will be described in detail below.

[0066] FIG. 7 is a diagram illustrating a method for calculating coordinate values ​​x, y, and Rz. The CPU 201, operating as the identification unit 210, extracts an edge 2e of the cable 2 and an edge 3e of the connector 3 from the image 50 using a known pattern matching technique or edge extraction technique. Edge 2e is the edge of the cable 2 on the connector 3 side. Edge 3e is the edge of the connector 3 on the cable 2 side. The CPU 201 identifies the X and Y coordinates of the endpoints 2b and 2c of the edge 2e. The CPU 201 calculates the X and Y coordinates of the midpoint 2d of the endpoints 2b and 2c based on the X and Y coordinates of the endpoints 2b and 2c. Similarly, the CPU 201 identifies the X and Y coordinates of the endpoints 3b and 3c of the edge 3e. The CPU 201 calculates the X and Y coordinates of the midpoint 3d of the endpoints 3b and 3c based on the X and Y coordinates of the endpoints 3b and 3c.

[0067] The CPU 201 calculates the coordinate value x by subtracting the X coordinate of the midpoint 3d from the X coordinate of the midpoint 2d, and calculates the coordinate value y by subtracting the Y coordinate of the midpoint 3d from the Y coordinate of the midpoint 2d.

[0068] The CPU 201 calculates the angle between the edge 2e and the edge 3e as a coordinate value Rz. The coordinate value Rz represents the amount of rotation in the counterclockwise direction when viewed from the positive side of the Z axis.

[0069] Fig. 8 is a diagram for explaining a method for calculating the coordinate value z. An image 50 is shown in the upper part of Fig. 8. A diagram showing the positional relationship between the edges 2e and 3e and the line lasers 10a and 12a when viewed from the X-axis direction is shown in the lower part of Fig. 8.

[0070] The CPU 201 identifies the Y coordinates of points 2f and 2g, respectively, onto which the line lasers 10a and 12a are projected at the edge 2e in the image 50. Similarly, the CPU 201 identifies the Y coordinates of points 3f and 3g, respectively, onto which the line lasers 10a and 12a are projected at the edge 3e in the image 50.

[0071] The CPU 201 calculates a difference d1 between the Y coordinate of point 2f and the Y coordinate of point 3f. The CPU 201 calculates the difference in the Z-axis direction between points 2f and 3f from the difference d1 using a light-section method. The difference in the Z-axis direction between points 2f and 3f is expressed as d1 / tan(θ1) using the angle θ1 formed between the projection direction of line laser 10a and imaginary plane 14a including optical axis 14b of camera 14. When angle θ1 is 45°, the difference in the Z-axis direction between points 2f and 3f coincides with the difference d1 between the Y coordinate of point 2f and the Y coordinate of point 3f.

[0072] Similarly, CPU 201 calculates difference d2 between the Y coordinate of point 2g and the Y coordinate of point 3g. CPU 201 calculates the difference in the Z-axis direction between points 2g and 3g from difference d2 using the light-section method. The difference in the Z-axis direction between points 2g and 3g is expressed as d2 / tan(θ2) using angle θ2 formed between the projection direction of line laser 12a and imaginary plane 14a including the optical axis of camera 14. When angle θ2 is 45°, the difference in the Z-axis direction between points 2g and 3g matches difference d2 between the Y coordinate of point 2g and the Y coordinate of point 3g.

[0073] The CPU 201 calculates the average value of the difference between points 2f and 3f in the Z-axis direction and the difference between points 2g and 3g in the Z-axis direction as the coordinate value z. When the angles θ1 and θ2 are 45°, the CPU 201 calculates (d1+d2) / 2 as the coordinate value z.

[0074] Fig. 9 is a diagram for explaining a method for calculating the coordinate value Rx. The upper part of Fig. 9 shows an image 50. The lower part of Fig. 9 shows the positional relationship between the edges 2e and 3e and the line lasers 10a and 12a when viewed from the X-axis direction.

[0075] The CPU 201 calculates the difference d1 / tan(θ1) in the Z-axis direction between points 2f and 3f and the difference d2 / tan(θ2) in the Z-axis direction between points 2g and 3g according to the method described with reference to Fig. 8. Furthermore, the CPU 201 calculates the difference d between the Y coordinate of point 2f and the Y coordinate of point 2g.

[0076] The CPU 201 calculates the angle θ3 that satisfies the following formula as the coordinate value Rx. θ3=arctan[{d1 / tan(θ1)-d2 / tan(θ2)} / d] If the angles θ1 and θ2 are 45°, the above equation is transformed into the following equation: θ3=arctan{(d1-d2) / d] The coordinate value Rx represents the amount of counterclockwise rotation when viewed from the positive side of the X axis (the amount of clockwise rotation when viewed from the negative side of the X axis).

[0077] FIG. 10 is a diagram illustrating a method for calculating the coordinate value Ry. An image 50 is shown in the center of the upper part of FIG. 10. The left side of the upper part of FIG. 10 shows a projection of the vicinity of the end on the positive side of the Y-axis on the upper surface 2a of the cable 2 when viewed from the Y-axis direction. The right side of the upper part of FIG. 10 shows a projection of the vicinity of the end on the negative side of the Y-axis on the upper surface 2a of the cable 2 when viewed from the Y-axis direction. The lower part of FIG. 10 shows a diagram illustrating the positional relationship between the upper surface 2a of the cable 2 and the line lasers 10a and 12a when viewed from the X-axis direction.

[0078] The CPU 201 identifies the X and Y coordinates of each of points 2f and 2i in the image 50, which are both ends of the line laser 10a projected onto the upper surface 2a of the cable 2. Similarly, the CPU 201 identifies the X and Y coordinates of each of points 2g and 2j in the image 50, which are both ends of the line laser 12a projected onto the upper surface 2a of the cable 2.

[0079] The CPU 201 calculates the difference d3 between the Y coordinate of point 2f and the Y coordinate of point 2i. Then, the CPU 201 calculates the difference d3 / tan(θ1) in the Z-axis direction between point 2f and point 2i according to the same method as that described with reference to Figure 8. Furthermore, the CPU 201 calculates the difference d4 between the X coordinate of point 2f and the X coordinate of point 2i.

[0080] The CPU 201 calculates the angle θ4 that satisfies the following formula. θ4=arctan[{d3 / tan(θ1)} / d4] If the angle θ1 is 45°, the above equation is transformed into the following equation: θ4=arctan(d3 / d4) Angle θ4 is the inclination angle of the upper surface 2a of the cable 2 in the vicinity of where the line laser 10a is projected, relative to the horizontal plane.

[0081] Similarly, CPU 201 calculates difference d5 between the Y coordinate of point 2g and the Y coordinate of point 2j. Then, CPU 201 calculates difference d5 / tan(θ2) between point 2g and point 2j in the Z-axis direction using the same method as described with reference to Figure 8. Furthermore, CPU 201 calculates difference d6 between the X coordinate of point 2g and the X coordinate of point 2j.

[0082] The CPU 201 calculates the angle θ5 that satisfies the following formula. θ5=arctan[{d5 / tan(θ2)} / d6] If the angle θ2 is 45°, the above equation is transformed into the following equation: θ5=arctan(d5 / d6) Angle θ6 is the inclination angle of the upper surface 2a of the cable 2 in the vicinity of where the line laser 12a is projected, relative to the horizontal plane.

[0083] The CPU 201 calculates the average value of the angles θ4 and θ5 as the representative value Ry of the inclination angle of the upper surface 2a of the cable 2 relative to the horizontal plane. cable It is calculated as follows.

[0084] Alternatively, the CPU 201 calculates the difference d7 between the Y coordinate of the point 2i and the Y coordinate of the point 2j. The CPU 201 calculates the difference d8 between the Y coordinate of the point 2f and the Y coordinate of the point 2g. Then, the CPU 201 calculates the representative value Ry according to the following formula: cable may be calculated. Ry cable =arctan{(d7-d8) / 2d4} In the above equation, the difference d6 or the average value of the differences d4 and d6 may be used instead of the difference d4.

[0085] In a similar manner, the CPU 201 calculates the representative value Ry of the inclination angle of the upper surface 3a of the connector 3 relative to the horizontal plane. connecter The CPU 201 calculates the representative value Ry cable and typical value Ry connecter The difference between (=Ry cable -Ry connecter ) is calculated as the coordinate value Ry. The coordinate value Ry represents the amount of counterclockwise rotation when viewed from the positive side of the Y axis (the amount of clockwise rotation when viewed from the negative side of the Y axis).

[0086] <Robot control method> 11 is a diagram illustrating an example of the control of the robot performed in step S5. Steps S51 to S54 shown in FIG. 11 are performed by the controller 30 based on information received from the image processing device 20 after the end effector 16a of the robot 16 has held the cable 2. The information received from the image processing device 20 includes three-dimensional coordinates (x, y, z, Rx, Ry, Rz) that represent the relative position and orientation of the cable 2 with respect to the connector 3. If the image 50 does not include the cable 2, the information received from the image processing device 20 indicates that the relative position and orientation of the cable 2 with respect to the connector 3 cannot be identified.

[0087] In step S51, when the controller 30 receives from the image processing device 20 information indicating that the relative position and orientation of the cable 2 with respect to the connector 3 cannot be determined, the controller 30 controls the robot 16 so that the tip of the cable 2 is included in the field of view of the camera 14. Specifically, the controller 30 controls the robot 16 so that the end effector 16a approaches the vicinity of the field of view of the camera 14. When the cable 2 moves into the field of view of the camera 14 and the controller 30 receives the three-dimensional coordinates (x, y, z, Rx, Ry, Rz) from the image processing device 20, the controller 30 performs the next step S52.

[0088] In step S52, the controller 30 controls the robot 16 so that the coordinate values ​​y, Rx, and Rz approach 0. The controller 30 also controls the robot 16 so that the coordinate value Ry falls within the target tilt range. When assembling the cable 2 to the connector 3, it is preferable that the cable 2 be tilted with respect to the horizontal plane. Therefore, the target tilt range is determined in advance to suit the assembling to the connector 3.

[0089] Furthermore, the controller 30 controls the robot 16 so that the coordinate value x approaches a first target value and the coordinate value z approaches a second target value. The first target value and the second target value are set in advance according to the relative position and posture of the cable 2 with respect to the connector 3 when the edge 2e of the cable 2 is located in front of the connector 3. As a result, the robot 16 moves the end effector 16a holding the cable 2 toward the connector 3 so that the relative position and posture of the cable 2 with respect to the connector 3 approaches the first state (the state in the second column from the right in FIG. 10). When the relative position and posture of the cable 2 with respect to the connector 3 reaches the first state, the controller 30 performs the next step S53.

[0090] In step S53, the controller 30 controls the robot 16 so that the coordinate values ​​x and z approach 0. At this time, the controller 30 controls the robot 16 so that the coordinate value Ry is maintained within the target tilt range. As a result, the robot 16 moves the end effector 16a holding the cable 2 toward the connector 3 so that the relative position and posture of the cable 2 with respect to the connector 3 approaches the second state (the state in the second column from the left in FIG. 10). When the relative position and posture of the cable 2 with respect to the connector 3 reaches the second state, the controller 30 performs the next step S54.

[0091] In step S54, the controller 30 controls the robot 16 so that the coordinate value Ry approaches 0. Furthermore, the controller 30 slightly moves the end effector 16a holding the cable 2 in the positive direction of the X-axis. As a result, the robot 16 moves the end effector 16a so that the relative position and posture of the cable 2 with respect to the connector 3 approaches the third state (the state in the first column from the left in FIG. 10). As a result, the cable 2 is assembled (inserted) into the connector 3.

[0092] <Variation 1> In the above description, the light sources 10, 12 and the camera 14 are installed in fixed positions. However, the light sources 10, 12 and the camera 14 may be installed on the robot 16 so as to move integrally with the end effector 16a.

[0093] Fig. 12 is a side view of an example of an assembling apparatus according to Modification 1. Fig. 13 is a front view of an example of an assembling apparatus according to Modification 1. As shown in Figs. 12 and 13, an assembling apparatus 1A according to Modification 1 differs from the assembling apparatus 1 shown in Figs. 1 and 2 in that it includes an installation table 42 instead of the frame 40, and that the light sources 10, 12 and the camera 14 are installed on the robot 16.

[0094] The robot 16 has a fixed frame 16b attached to the tip of the arm. The fixed frame 16b supports the end effector 16a, the light sources 10 and 12, and the camera 14.

[0095] The light sources 10 and 12 are installed at a predetermined distance from each other along a direction (Y-axis direction) perpendicular to the optical axis 14b (Z-axis direction) of the camera 14. Furthermore, the light sources 10 and 12 irradiate the line lasers 10a and 12a by diverging collimated light in a fan shape in a direction (X-axis direction) perpendicular to the Z-axis and Y-axis.

[0096] End effector 16a holds the vicinity of the tip of cable 2 so that the longitudinal direction of cable 2 is parallel to the X-axis. As a result, line lasers 10a and 12a are projected onto cable 2 so as to intersect with edge 2e at the tip of cable 2. Furthermore, by moving end effector 16a so that cable 2 approaches connector 3, line lasers 10a and 12a are also projected onto connector 3.

[0097] The camera 14 is installed on a fixed frame 16b at a position between the light sources 10 and 12 so that the edge 2e of the cable 2 held by the end effector 16a and the area around the edge 2e are included in the field of view.

[0098] In the first modification, the projection directions of the line lasers 10a and 12a are also inclined with respect to an imaginary plane 14a including an optical axis 14b of the camera 14. The imaginary plane 14a is parallel to the divergence directions of the line lasers 10a and 12a. Furthermore, the angles θ1 and θ2 formed by the projection directions of the line lasers 10a and 12a and the imaginary plane 14a are preferably 45°.

[0099] In the assembly apparatus 1A according to the first modification, the end effector 16a holds the cable 2, so that the tip of the cable 2 and the area around the tip of the cable 2 are included in the field of view of the camera 14. Therefore, by moving the end effector 16a so that the cable 2 approaches the connector 3, the connector 3 is included in the field of view of the camera 14. As a result, the camera 14 can output an image 50 that includes the cable 2 and the connector 3.

[0100] In the first modification, the image processing device 20 also calculates three-dimensional coordinates (x, y, z, Rx, Ry, Rz) representing the relative position and orientation of the cable 2 with respect to the connector 3 based on the image 50, as described with reference to FIGS. 7 to 10. This makes it possible to correct the operation of the robot 16 according to the relative position and orientation. Alternatively, it becomes possible to inspect whether the cable 2 has been properly attached to the connector 3 according to the relative position and orientation. As a result, the tolerance for deviation in the orientation of the connector 3 or the cable 2 is increased, and the flexibility of the equipment for transporting or supporting the connector 3 or the cable 2 is increased.

[0101] <Variation 2> The image processing device 20 may be integrated with the controller 30. That is, the image processing device 20 and the controller 30 may be realized by a single computer.

[0102] <Variation 3> The components that the assembly apparatus 1, 1A assembles are not particularly limited. For example, the assembly apparatus 1, 1A may assemble connectors together. That is, the assembly apparatus 1, 1A may mate a first connector with a second connector. The first connector and the second connector are examples of the "first component" and the "second component" of the present disclosure, respectively.

[0103] §3 Supplementary Note As described above, the present embodiment includes the following disclosures.

[0104] (Configuration 1) An assembly device (1, 1A) that assembles a first component (2) to a second component (3), a first light source (10) that projects a first line laser (10a) onto the first component (2) and the second component (3); a second light source (12) that projects a second line laser (12a) onto the first component (2) and the second component (3) from a direction different from that of the first line laser (10a); a camera (14) for photographing the first part (2) and the second part (33); an identification unit (210) that identifies the relative position and orientation of the first part (2) with respect to the second part (3) based on the projection positions of the first line laser (10a) and the second line laser (12a) on the first part (2) in the image (50) output from the camera (14) and the projection positions of the first line laser (10a) and the second line laser (12a) on the second part (3) in the image (50).

[0105] (Configuration 2) a robot (16) that holds the first part (2); The assembly apparatus (1, 1A) according to configuration 1 further comprises a controller (30) that controls the robot (16) so that the relative position and posture approaches a target state.

[0106] (Configuration 3) The robot (16) has an end effector (16a) that holds the first part (2), The assembly device (1A) according to configuration 2, wherein the first light source (10), the second light source (12), and the camera (14) are installed on the robot (16) so as to move integrally with the end effector (16a).

[0107] (Configuration 4) The assembly apparatus (1, 1A) according to configuration 1 further comprises an inspection unit (211) that inspects whether the relative position and posture are within a reference range.

[0108] (Configuration 5) The assembly device (1, 1A) according to any one of configurations 1 to 4, further comprising an illumination device (18) that irradiates the field of view of the camera (14) with illumination light.

[0109] (Configuration 6) An assembly method for assembling a first part (2) to a second part (3), comprising: A first light source (10) projects a first line laser (10a) onto the first component (2) and the second component (3); a second light source (12) projects a second line laser (12a) onto the first component (2) and the second component (3) from a direction different from that of the first line laser (10a); A camera (14) photographs the first part (2) and the second part (3); An assembly method comprising: a processor determining a relative position and orientation of the first part (2) with respect to the second part (3) based on the projected positions of the first line laser (10a) and the second line laser (12a) on the first part (2) in the image (50) output from the camera (14) and the projected positions of the first line laser (10a) and the second line laser (12a) on the second part (3) in the image (50).

[0110] Although the embodiments of the present invention have been described, the embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is defined by the claims, and it is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0111] 1,1A assembly device, 2 cable, 3 connector, 4 board, 10,12 light source, 10a,12a line laser, 14 camera, 14a virtual plane, 14b optical axis, 16 robot, 16a end effector, 16b fixed frame, 18 lighting device, 20 image processing device, 30 controller, 40 frame, 42 installation stand, 50 image, 202 main memory, 203 storage, 204 camera interface, 205 input interface, 206 display controller, 207 communication interface, 208 data reader / writer, 209 bus, 210 identification unit, 211 inspection unit, 212 input device, 213 display, 215 memory card, 230 program.

Claims

1. An assembly device that assembles a first part to a second part, a first light source that projects a first line laser onto the first component and the second component; a second light source that projects a second line laser onto the first component and the second component from a direction different from that of the first line laser; a camera that captures images of the first part and the second part; an identification unit that identifies a relative position and orientation of the first part with respect to the second part based on projection positions of the first line laser and the second line laser on the first part in an image output from the camera and projection positions of the first line laser and the second line laser on the second part in the image.

2. a robot that holds the first part; The assembly apparatus according to claim 1 , further comprising: a controller that controls the robot so that the relative position and posture approach a target state.

3. the robot has an end effector that holds the first part; The assembly apparatus according to claim 2 , wherein the first light source, the second light source, and the camera are installed on the robot so as to move integrally with the end effector.

4. The assembly apparatus according to claim 1 , further comprising an inspection unit that inspects whether the relative position and attitude are within a reference range.

5. The assembly apparatus according to claim 1 , further comprising an illumination device that irradiates a field of view of the camera with illumination light.

6. An assembly method for assembling a first part to a second part, comprising: a first light source projects a first line laser onto the first component and the second component; a second light source projects a second line laser onto the first component and the second component from a direction different from that of the first line laser; a camera photographing the first part and the second part; An assembly method comprising: a processor determining a relative position and orientation of the first part with respect to the second part based on projected positions of the first line laser and the second line laser on the first part in an image output from the camera and projected positions of the first line laser and the second line laser on the second part in the image.

Images