Robot control system for multi-point fitting

Abstract

The present invention provides a robot control system for performing multi-point interlocking, which comprises: a relative relationship calculation unit, which calculates the relative relationship between a first component and a second component at at least one of the multiple points based on data obtained by a visual sensor; a contact point determination unit, which determines the contact point between the first component and the second component based on the calculated relative relationship; a control point setting unit, which sets the control point based on the determined contact point; and an interlocking control unit, which controls the interlocking of the multiple points based on the set control point.

Description

Technical Field

[0001] The invention relates to a robot control system, in particular to a robot control system for performing multi-point embedding. Background Art

[0002] When using industrial robots to assemble components, it is difficult to fit the components together if the relative relationship (i.e., relative position, relative posture) between the components changes. When one component is stationary, only the position and posture of the other component need to be considered. However, when one component is continuously moving, the relative relationship between the components will continue to change, making it more difficult to fit together. Robot control using visual sensors and force sensors is widely used in fitting operations.

[0003] International Publication No. 1998 / 017444 records a force-controlled robot system that obtains a workpiece position / posture representing the position / posture relationship between a first workpiece and a second workpiece based on image data obtained by a visual sensor, and corrects the position and posture of the robot based on the obtained workpiece position / posture data, thereby performing a mating action while performing force control based on the output from the force sensor.

[0004] It is recorded in Japanese Patent Gazette No. 2010-89238: The contact state of the first part and the second part and the transition information from each contact state to the next target state are associated and stored in advance, the contact state is determined using the detection value of the force sensor and the image data of the shooting device, and the transition information is selected based on the result of the determination, and the fitting control is performed based on the selected transition information.

[0005] Japanese Patent Gazette No. 2015-74058 states: a target image when the endpoint of the movable part of the robot is at the target position and a current image when the endpoint is at the current position are obtained, the distance from the current position to the target position is calculated, and the command value based on visual servoing and the command value based on force control are weighted and synthesized according to the calculated distance to obtain a synthesized command value, and the robot is controlled using the synthesized command value. Summary of the invention

[0006] When the components are mated with each other at multiple points, the force sensor detects the reaction force from the components at each point as a resultant force, so it is difficult to determine the contact point. In addition, various noises are mixed in the data obtained by the force sensor, making it difficult to distinguish whether it is a single-point contact or a multi-point contact. For example, when the rigidity of holding the component is low, noise caused by the rigidity will be mixed. Moreover, the force sensor may generate a mechanism-related noise that generates a force in another direction even when the force is applied in only one direction. In addition, when the gain of the force sensor is high, electrical noise may also be mixed from cables, etc. In addition, noise caused by aging, corrosion, etc. of the force sensor may also occur. As a result, in the multi-point mating operation, it is difficult to determine the position and number of contact points only by the force sensor, so it is impossible to determine where the component should be moved or in which direction the component should be tilted, and the robot may be controlled to the wrong position or posture, resulting in a failure of mating.

[0007] Therefore, a technology for controlling a robot that performs multi-point fitting is needed.

[0008] One scheme of the present disclosure provides a robot control system, comprising: a robot; a hand, which is installed at the end of the robot and holds a second component that is engaged with a first component at multiple points; a visual sensor, which is arranged near the hand or at a fixed position independent of the robot; a relative relationship calculation unit, which calculates the relative relationship between the first component and the second component at at least one of the multiple points based on data obtained by the visual sensor; a contact point determination unit, which determines the contact point between the first component and the second component based on the calculated relative relationship; a control point setting unit, which sets the control point based on the determined contact point; and an engagement control unit, which controls the engagement of multiple points based on the set control point. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is a side view showing a schematic configuration of a robot control system according to one embodiment.

[0010] Figure 2A It is a stereoscopic diagram showing a modified example of the number of visual sensors and the direction of sight.

[0011] Figure 2B It is a perspective view showing a modified example of the arrangement of the visual sensor.

[0012] Figure 3 This is a block diagram showing the structure of a robot control system according to one embodiment.

[0013] Figure 4A This is a diagram showing the relative relationship between components taken from above.

[0014] Figure 4BThis is a diagram showing the relative relationship between components taken from above.

[0015] Figure 4C This is a diagram showing the relative relationship between components taken from above.

[0016] Figure 5A This is a diagram showing the relative relationship between components taken from the side.

[0017] Figure 5B This is a diagram showing the relative relationship between components taken from the side.

[0018] Figure 5C This is a diagram showing the relative relationship between components taken from the side.

[0019] Fig. 6A It is a side view showing the control point when one point is in contact.

[0020] Figure 6B It is a side view showing the control point when one point is in contact.

[0021] Figure 6C It is a side view showing the control points in case of multi-point contact.

[0022] Figure 7 It is a top view showing the exploration range of force control.

[0023] Figure 8 This is a schematic flowchart showing the integration control of the robot control system according to one embodiment. DETAILED DESCRIPTION

[0024] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. The same or similar symbols are used for the same or similar structural elements in each figure. In addition, the embodiments described below do not limit the technical scope of the invention and the meaning of the terms recorded in the claims.

[0025] Figure 1 The schematic structure of the robot control system 1 of the present embodiment is shown. The robot control system 1 includes: a robot 10, a hand 11 installed at the end of the robot 10, a visual sensor 12 provided near the hand 11, and a robot control device 13 for controlling the robot 10 and the hand 11. Although the robot 10 is a multi-joint robot, it may also be other industrial robots such as a parallel linkage robot. Although the hand 11 is a hand with multiple claws, other types of hands such as an adsorption-type hand may also be used. The visual sensor 12 may also be a two-dimensional sensor such as a CCD (charge-coupled device) camera, or a three-dimensional sensor such as a TOF (time-of-flight) camera or a laser scanner.

[0026] The hand 11 holds the second component 16 that is engaged with the first component 15 at multiple engagement points A and B, and the robot 10 performs an assembly operation of assembling the second component 16 to the first component 15. The first component 15 and the second component 16 are structural components of an article having multiple engagement points, such as a frame and a window, a body and a cover, a vehicle and a door, etc. The engagement points may be three or more instead of two. The first component 15 may be continuously moved by being transported by the conveying device 14, or the conveying device 14 may be temporarily stopped and temporarily stationary. Alternatively, the first component 15 may be fixed by a fixing device (not shown) and remain stationary at all times.

[0027] The assembly operation performed by the robot 10 includes: alignment control to align the convex parts 16a, 16b of the second component 16 with the concave parts 15a, 15b of the first component 15; and fitting control to move the second component 16 in the fitting direction 17 with the first component 15 so that the convex parts 16a, 16b fit with the concave parts 15a, 15b. In addition, the convex parts 16a, 16b and the concave parts 15a, 15b can also be set on components opposite to each other. When the first component 15 is stationary, it is sufficient to consider the position and posture of the second component 16, but in the fitting of multiple points, the fitting failure may occur due to a slight deviation in the relative relationship between the components. In addition, when the first component 15 continues to move, the relative relationship between the components will continue to change, so the fitting is more difficult. Therefore, it is preferred to use the visual sensor 12 to calculate the relative relationship between the components, determine the contact point based on the relative relationship, and perform fitting control through visual control.

[0028] The robot control system 1 is further provided with a force sensor 19 between the robot 10 and the hand 11. The force sensor 19 may be a strain gauge type, a piezoelectric type, a capacitive type, or the like, and preferably detects forces and moments in three axes, six axes, or the like. When the relative size (gap) between the convex parts 16a, 16b and the concave parts 15a, 15b is relatively large, the mating can be successfully achieved only by the visual sensor 12, but when the relative size between the convex parts 16a, 16b and the concave parts 15a, 15b is relatively small, the convex parts 16a, 16b may be in contact with the concave edge peripheral surface of the concave parts 15a, 15b, and sometimes the mating can never be successfully achieved. Therefore, the force sensor 19 is used to detect the reaction force from the component, and precise mating control is performed based on the detected force. However, as described above, in the mating of multiple points, it is difficult to determine the position and number of contact points only by the force sensor 19, and as a result, the robot may be controlled to an incorrect position or posture. Therefore, when the relative size of the convex portion and the concave portion is relatively small, the relative relationship between the components is calculated using the visual sensor 12, and the contact point is determined based on the relative relationship, thereby performing fitting control through force control.

[0029] Figure 2A - Figure 2B A modified example of the vision sensor is shown. Figure 1 FIG. 1 shows a state where a visual sensor 12 is set to capture at least one of the plurality of fitting points A and B. However, each fitting point A and B may be hidden in the shadow of the component and it is impossible to determine where the fitting point A and B are in contact. Figure 2A - Figure 2B As shown, a plurality of visual sensors 12a and 12b are provided to capture at least one of the plurality of fitting points A and B in different sight lines. In addition, in order to determine at which points contact is made, a visual sensor 12b that can capture all fitting points A and B may be provided. In this example, one visual sensor 12a captures the fitting points A and B from above, and the other visual sensor 12b captures the fitting points A and B from the side.

[0030] And, if Figure 2B As shown, the visual sensors 12a and 12b may be provided at fixed positions independent of the robot 10. When the visual sensors 12a and 12b are arranged on, for example, a fixed column 22, the visual sensor 12a is arranged on the fixed column 22 in such a manner that at least one of the plurality of fitting points A and B is within the imaging range 20 of the visual sensor 12a on one side, and the visual sensor 12b is arranged in such a manner that at least one of the plurality of fitting points A and B is within the imaging range 21 of the visual sensor 12b on the other side.

[0031] Refer again Figure 1 , the robot control system 1 can set the origin of the robot coordinate system C1 to the base 18 of the robot 10, and set the origin of the tool coordinate system C2 to the vicinity of the hand 11 or the second member 16. At this time, the position of the robot 10 is represented as the origin of the tool coordinate system C2, and the posture of the robot 10 is represented as the orientation of the tool coordinate system C2 relative to the robot coordinate system C1. The robot control system 1 is configured to be able to mutually transform the coordinates of any points in these coordinate systems.

[0032] Figure 3The structure of the robot control system 1 of the present embodiment is shown. The robot control device 13 is a control device having a processor such as a CPU (central processing unit), and controls the robot 10 and the hand 11 by the pre-taught motion program 36. The robot control device 13 includes: a storage unit 30 for storing the motion program 36 and the like, a motion control unit 31 for controlling the motion of the robot 10 and the hand 11 based on the motion program 36, a robot driving unit 32 for driving the robot driving motor 34, and a hand driving unit 33 for driving the hand driving motor 35. The motion program 36 is a program that describes basic motion instructions for the robot 10 and the hand 11 that perform assembly work. The motion control unit 31 sends motion instructions to the robot driving unit 32 and the hand driving unit 33 based on the motion program 36, and the robot driving unit 32 and the hand driving unit 33 supply power to the robot driving motor 34 and the hand driving motor 35 based on the motion instructions.

[0033] In order to be able to perform alignment control between the first component and the second component even when the first component is offset or continuously moving, the robot control device 13 may further include a correction instruction generation unit 40 that corrects the motion instruction. Alternatively, the correction instruction generation unit 40 calculates the offset amount of the first component based on the data obtained by the visual sensor 12 and sends the correction instruction to the motion control unit 31, or calculates the predicted movement amount of the first component based on various historical data in addition to the offset amount and sends the correction instruction to the motion control unit 31. The motion control unit 31 sends the motion instruction to the robot drive unit 32 and the hand drive unit 33 based on the correction instruction. In this way, alignment control is performed to arrange the convex part of the second component just above the concave part of the first component 15.

[0034] In order to be able to perform multi-point fitting control after or while performing alignment control, the robot control device 13 also includes: a relative relationship calculation unit 41 that calculates the relative relationship between the components, a contact point determination unit 42 that determines the contact point based on at least the relative relationship, a control point setting unit 43 that sets the control point based on the contact point, and a fitting control unit 44 that controls the fitting of multiple points. The relative relationship calculation unit 41 calculates the relative relationship between the components based on the data obtained by the visual sensor 12. The relative relationship includes at least the relative position between the components, and preferably also includes the relative posture between the components. Alternatively, the contact point determination unit 42 determines the contact point between the first component and the second component based not only on the relative relationship but also on the data obtained by the force sensor 19.

[0035] Figure 4A - Figure 4C shows the relative relationship between components photographed from above, Figure 5A - Figure 5CThe relative relationship between the components photographed from the side is shown. The relative relationship can be calculated by various methods, for example, it can be calculated as described below. First, the recess 15a of the first component 15 and the protrusion 16a of the second component 16 are detected by pattern matching from the data obtained by the visual sensor. The pre-prepared pattern can be a two-dimensional pattern or a three-dimensional pattern. The pattern is rotated and the scale is changed to perform pattern matching. After the recess 15a and the protrusion 16a are detected, the position of the feature point P1 relative to the feature point P2 is calculated as a relative position. In addition, each rotation angle around the three orthogonal axes extending from the feature point P1 relative to each rotation angle around the three orthogonal axes extending from the feature point P2 can also be calculated as a relative posture.

[0036] The calculated relative position is compared with a predetermined threshold value to determine whether there is a contact point. Alternatively, in addition to the relative position, the relative posture is also compared with a predetermined threshold value to determine whether there is a contact point. In addition, in the case of a force sensor, the presence or absence of a contact point can also be determined based on the data obtained by the force sensor. In addition, the one with higher reliability can be appropriately selected according to each determination result. Figure 4A - Figure 4B and Figure 5A - Figure 5B shows the case with no contact points, Figure 4C and Figure 5C The case with a contact point is shown. Figure 5C When there is a contact point 50 as shown, the position of the contact point 50 is determined based on the relative relationship between the components. In addition, when there is a force sensor, the position of the contact point 50 can also be determined based on the data obtained by the force sensor. In addition, it is preferable to determine the contact point not only for one fitting point A but also for another fitting point B, and determine the number of contact points.

[0037] Refer again Figure 3 If it is determined that there is no contact point, the fitting control unit 44 sends a correction instruction to the motion control unit 31 based on the relative relationship between the components to match the relative relationship between the components. On the other hand, if it is determined that there is a contact point, the control point setting unit 43 sets the control point based on the contact point. Fig. 6A - Figure 6BThe control point 51 for one-point contact is shown. When the first component 15 and the second component 16 are in contact at one point, the control point 51 can be set near the contact point. When the front end of the convex portion 16a is in contact with the peripheral surface of the concave edge of the concave portion 15a, the control point 51 is set at the contact point, and the posture of the robot is controlled around the control point 51 as shown by the matching direction 52. However, when the front end of the convex portion 16a is in contact with the outer side surface of the concave portion 15a, the control point 51 is set at the front end of the convex portion 16a located near the contact point (or the fitting point A when the robot holds the first component 15), and the position of the robot is controlled in such a way that the control point 51 matches the target position.

[0038] Figure 6C The control point 51 in case of multi-point contact is shown. When the first component 15 and the second component 16 are in contact at multiple points, the control point 51 can be set at the midpoint near each contact point. When the front ends of the convex parts 16a and 16b are in contact with the peripheral surfaces of the concave edges of the concave parts 15a and 15b, the control point 51 is set at the midpoint of each contact point, and the position of the robot is controlled in such a manner that the control point 51 matches the target position as shown in the matching direction 53. In addition, in addition to the position control of the robot, the posture of the robot can be controlled around the control point 51 in such a manner that it matches the target posture. However, when the front ends of the convex parts 16a and 16b are in contact with the outer side surfaces of the concave parts 15a and 15b, respectively, the control point 51 can be set at the midpoint of the front ends of the convex parts 16a and 16b located near each contact point (or the midpoint of each fitting point A and B when the robot holds the first component 15), and the position of the robot can be controlled in such a manner that the control point 51 matches the target position.

[0039] Refer again Figure 3 , the fitting control unit 44 sends a correction instruction to the motion control unit 31 based on the control point set as described above. That is, the fitting control unit 44 sends a correction instruction to the motion control unit 31 to control the posture of the robot around the control point when there is a single point of contact, and sends a correction instruction to the motion control unit 31 to match the control point with the target position when there are multiple points of contact.

[0040] When the fitting control is performed by force control, the robot control system 1 may further include a search range limiting unit 45 that limits the search range of the force control based on data acquired by the vision sensor 12 . Figure 7FIG. 5 shows a search range 54 for force control. The search range can be set by various methods, for example, as described below. First, the recess 15a of the first component 15 is detected by pattern matching from the data obtained by the visual sensor 12, and the position of the fitting point A is obtained. Then, the search range 54 for force control is limited to the area of ​​the first component 15 around the fitting point A, that is, the concave edge circumference of the recess 15a of the first component 15.

[0041] Refer again Figure 3 The fitting control unit 44 sends a correction instruction to the motion control unit 31 to scan within the restricted exploration range, calculates the force and torque of the control point based on the data obtained by the force sensor, and sends a correction instruction to the motion control unit 31 to precisely match the control point with the target position based on the calculated force and torque.

[0042] Figure 8 The following is a schematic diagram of the interlocking control of the robot control system 1 of the present embodiment. In step S10, the relative relationship between the first component and the second component at at least one of the multiple interlocking points is calculated. In step S11, the contact point is determined based on the calculated relative relationship. If it is determined that there is no contact point, the process proceeds to step S15 to perform interlocking control to match the relative relationship between the components. If it is determined that there is a contact point, in step S12, a control point is set near the contact point when there is a single point of contact, and a control point is set at the midpoint near each contact point when there are multiple points of contact.

[0043] Then, in step S13, the posture of the robot is controlled around the control point when one point is in contact. In case of multiple points of contact, the position of the robot is controlled in such a way that the control point matches the target position. In addition, in the case of fitting control by force control, the search range of force control is limited in step S14. In step S15, fitting control is performed only by visual control, or fitting control is performed by force control in addition to visual control.

[0044] According to the above embodiment, the robot 10 for multi-point fitting can be controlled. Figure 3 In the embodiment, the correction instruction generating unit 40, the relative relationship calculating unit 41, the contact point determining unit 42, the control point setting unit 43, the fitting control unit 44 and the search range limiting unit 45 may be configured not in the robot control device 13 but in a host computer device.

[0045] The program executed by the above-mentioned processor may be recorded in a computer-readable non-transitory storage medium, such as a CD-ROM, and provided.

[0046] Although various embodiments are described in this specification, the present invention is not limited to the above-described embodiments, and various modifications can be made within the scope described in the claims.

Claims

1. A robot control system, characterized in that: have: robot; a hand mounted on an end of the robot and holding a second component engaged with the first component at a plurality of engagement points; a visual sensor disposed near the hand or at a fixed position independent of the robot; a relative relationship calculation unit that calculates a relative relationship between the first component and the second component at at least one of the plurality of fitting points based on data acquired by the visual sensor; a contact point determination unit that determines a contact point between the first component and the second component based on the calculated relative relationship; a control point setting unit that sets a control point based on the determined contact point; as well as a fitting control unit that controls fitting of the plurality of fitting points based on the set control point, The contact point determination unit determines the positions and number of the contact points where the first component and the second component are in contact. The control point setting unit sets the control point based on the position and number of the contact points.

2. The robot control system according to claim 1, characterized in that: A force sensor is also provided, the force sensor being arranged between the robot and the hand, The fitting control unit performs force control on the fitting of the plurality of fitting points based on data acquired by the force sensor.

3. The robot control system according to claim 1 or 2, characterized in that: When the contact point is only one point, the control point setting unit sets the control point near the contact point.

4. The robot control system according to claim 1 or 2, characterized in that: When there are multiple contact points, the control point setting unit sets the control point at a midpoint near each contact point.

5. The robot control system according to claim 3, characterized in that: When the contact point is only one point, the fitting control unit controls the posture of the robot around the control point.

6. The robot control system according to claim 4, characterized in that: When there are multiple contact points, the fitting control unit controls the position of the robot in a manner that the control point matches the target position, or controls the posture of the robot around the control point in a manner that matches the target posture in addition to controlling the position of the robot.

7. The robot control system according to claim 2, characterized in that: A search range limiting unit is further provided, the search range limiting unit limiting the search range of the force control based on the data acquired by the vision sensor.

8. The robot control system according to claim 7, characterized in that: The search range is limited to an area of ​​the first component around the fitting point.

9. The robot control system according to claim 1 or 2, characterized in that: The visual sensor captures an image of at least one of the plurality of fitting points.

10. The robot control system according to claim 1 or 2, characterized in that: The visual sensor is provided in plurality and photographs at least one of the plurality of fitting points in different sight lines.

11. The robot control system according to claim 1 or 2, characterized in that: The relative relationship includes the relative position of the first component and the second component.

12. The robot control system according to claim 11, characterized in that: The relative relationship further includes a relative posture of the first component and the second component.

Images