Robot system and position correction method of robot system

Abstract

To provide a robot system capable of correcting the position of a workpiece according to a change in the positional relationship between an imaging device fixed to a support member and the workpiece.SOLUTION: A tool is used as a marker for positioning a workpiece 2. A camera 4, in a state of being fixed to a support member 5, captures an image of the workpiece 2 and the tool. A camera controller 7 firstly converts a reference position based on a master workpiece 3 grasped from the image into a position on a robot coordinate system, and causes the converted position to be recorded as an initial position. Further, a position of the workpiece 2 grasped from the image is converted into a position on the robot coordinate system, and the position is set as the position of the workpiece 2. When a state occurs in which the position of the workpiece 2 which is set after the initial position is recorded does not match an actual position, the camera controller generates, from the reference position grasped from the image at that time and the initial position, an expression I for converting the workpiece position in the workpiece position coordinate system at a present time point into the workpiece position in a reference coordinate system at a time point when the initial position is recorded, and corrects the position of the workpiece 2 by using the conversion expression I.SELECTED DRAWING: Figure 15

Description

[Technical field]

[0001] The present invention relates to a robot system that uses a marker to position a workpiece, and a position correction method for the system. [Background technology]

[0002] For example, Patent Document 1 discloses a technique in which markers are placed near a workpiece to be worked on by a robot, an image including the markers is captured by a camera, and the positions of the markers captured in the camera's coordinate system are converted to positions in a coordinate system based on the position of the robot, so that the position of the workpiece is grasped in the robot's coordinate system and positioned. In this technique, a CCD camera 4 is moved above a recognition mark 2 provided on a substrate 1, and an image of the recognition mark 2 is taken into a vision controller 5 to detect the center of gravity of the recognition mark 2. Then, the CCD camera 4 is moved a certain distance in the X or Y direction by an XY robot 8, and the center of gravity of the recognition mark 2 is detected again. Then, a calibration value is calculated from the distance between the two obtained centers of gravity in the memory space of the vision controller 5 and the actual movement distance of the XY robot 8. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 7-208924 Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, the CCD camera 4 is moved a fixed distance by an XY robot 8 to calibrate the position of the marker. However, Patent Document 1 does not take into consideration how to deal with a system in which the camera is fixedly arranged, in which the positional relationship between the camera and the workpiece changes due to environmental changes over time or the application of external forces after the system is actually installed at the site.

[0005] The present invention has been made in consideration of the above-mentioned situation, and its object is to provide a robot system that can correct the position of a workpiece in response to a change in the positional relationship between an imaging device fixed to a support member and the workpiece, and a position correction method for the system. [Means for solving the problem]

[0006] According to the robot system of claim 1, a tool attached to the hand of the robot arm is used as a marker when positioning a workpiece that is an object of the robot. The imaging device takes an image capturing the marker and the workpiece while being fixed to a support member. The reference position setting unit first converts a reference position based on the marker grasped from the image into a position on a robot coordinate system based on the position of the robot, and sets the converted position as an initial position. The workpiece position setting unit converts the position of the workpiece grasped from the image into a position on the robot coordinate system, and sets the converted position as the position of the workpiece.

[0007] Then, if the work position set by the work position setting unit does not match the actual work position after the initial position is set, the reference position setting unit generates a conversion equation for converting the current work position coordinate system to the work position in the reference coordinate system at the time the initial position was set, based on the difference between the reference position grasped from the image and the initial position.The work position setting unit then corrects the work position using the conversion equation.

[0008] For example, if the support member expands or contracts due to environmental changes or the like after the initial position is recorded, or an external force is applied to the support member, causing a shift in the position of the imaging device, the position of the workpiece grasped from the image of the imaging device will also shift. This will make it impossible for the robot to perform the intended work on the workpiece. Therefore, if the reference position setting unit generates a conversion equation for converting the current workpiece position coordinate system to the workpiece position in the reference coordinate system at the time the initial position was set, and the workpiece position setting unit corrects the workpiece position using the conversion equation, the position of the workpiece can be corrected to be correctly positioned on the robot coordinate system. This allows the robot to continue working on the workpiece.

[0009] A tool attached to the end of the robot arm is used as a marker. Generally, a robot arm is always in its initial position and posture, or at its origin position, before starting a task, so the position of the tool at the end of the arm is always the same. Therefore, the tool can be used as a marker, and there is no need to prepare a separate marker.

[0010] According to the robot system of claim 2, the present invention is applied to a system in which a robot is placed on a moving body. In such a system, even if the moving body is at the origin position, it is assumed that a slight deviation occurs before and after the movement. Even if it becomes necessary to correct the position of the workpiece due to the deviation, it can be dealt with. At this time, even if the position of the imaging device is also shifted due to the expansion or contraction of the support member, it can be corrected in the same way.

[0011] According to the robot system of claim 3, the reference position setting unit generates a conversion formula when notified that the robot's hand has failed to access the workpiece via a control device that controls the operation of the robot. The failure to access the workpiece indicates that the position of the workpiece set by the workpiece position setting unit will not match the actual position of the workpiece. Therefore, the reference position setting unit can appropriately correct the position of the workpiece as necessary.

[0012] According to the robot system of claim 4, when the robot's hand fails to access the workpiece, the control device causes the robot to retry the access, increments a counter that counts the number of retries, and resets the counter when the access to the workpiece is successful. When the counter value exceeds a predetermined upper limit, the control device notifies the reference position setting unit, and when the reference position setting unit receives the notification, it generates a conversion formula. As a result, the reference position setting unit generates a conversion formula when it is certain that the robot's hand has failed to access the workpiece. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a robot system according to a first reference example. [Diagram 2] FIG. 1 is a perspective view showing the arrangement of an object work and a master work. [Diagram 3] A diagram showing the position of the robot's hand before and after correction when the camera position is changed from the initial position. [Figure 4] Functional block diagram showing the configuration of the robot system [Diagram 5] A model floor plan with only one master workpiece placed [Figure 6] A schematic diagram showing the change in the position of the object work captured in the image as a result of the camera being shifted from its original position. [Figure 7] A model plan view of four master workpieces [Figure 8] A simple example of a transformation matrix I [Figure 9] A diagram showing the state in which a calibration marker is placed on the tip of a robot arm. [Figure 10] A flowchart showing a process for calculating parameters for transforming a camera coordinate system into a robot coordinate system. [Figure 11] A flowchart showing the process of converting the position of the master work in the camera coordinate system into the robot coordinate system. [Figure 12]Flowchart showing the process when the robot system is actually operated (part 1) [Figure 13] Flowchart showing the process when the robot system is actually operated (part 2) [Figure 14] FIG. 2 is a flowchart illustrating a process performed when the robot system is actually operated according to a second reference example. [Figure 15] FIG. 1 is a diagram illustrating a schematic configuration of a robot system according to a first embodiment. [Figure 16] FIG. 2 is a schematic diagram showing a configuration of a model plan view robot system according to a second embodiment when a tool is used as a master workpiece. [Figure 17] A diagram showing the state in which a calibration marker is placed on the tip of a robot arm. [Figure 18] A flowchart showing the process of converting the position of the master work in the camera coordinate system into the robot coordinate system. [Figure 19] A diagram showing the object work placed on the workbench [Figure 20] 12 is a flowchart showing the process when the robot system is actually operated. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] (1st reference example) A first reference example will be described below with reference to Fig. 1 to Fig. 13. As shown in Fig. 1, the robot system of this reference example includes, for example, a vertical six-axis robot 1, an object work 2 that is a work target of the robot 1, a master work 3 that is a marker used to position the object work 2, and a support member 5 for fixing and supporting a camera 4 that is an imaging device above the robot 1. Note that Fig. 1 shows a part of the system as a model. The camera 4 is a so-called 3D camera that can capture images of the object work 2 and the master work 3 from above the robot 1 and measure the distance of an object captured in the image three-dimensionally.

[0015] As shown in Fig. 2, four cylindrical master workpieces 3a to 3d are used in this reference example. The object workpieces 3a to 3d are arranged so that the object workpiece 2 is located within a roughly rectangular area formed by the four master workpieces 3a to 3d. Note that the object workpiece 2 shown in Fig. 2 does not have the shape of an actual object that the robot 1 will work on, but is intended to indicate the corners of a rectangular parallelepiped as the workpiece position recognized by the robot 1, as shown in Fig. 3.

[0016] Figure 4 is a functional block diagram showing the configuration of the robot system. The arm of the robot 1 is controlled by a robot controller 6. The camera controller 7 is, for example, a personal computer, and acquires image data captured by the camera 4 and performs positioning processing and the like. The robot controller 6 and the camera controller 7 communicate with each other. The above constitutes the robot system 11. The robot controller 6 corresponds to a control device, and the camera controller 7 corresponds to a reference position setting unit and a work position setting unit.

[0017] In this reference example, as shown in Fig. 1, correction is performed to resolve a state in which the position of the camera 4 shifts from the initial position due to changes in the support member 5 over time, expansion or contraction of a part of the support member 5 due to the influence of temperature, or the application of an external force, causing the position of the object work 2 on the camera coordinate system to change, and the arm end point of the robot 1 is unable to properly access the object work 2. Here, "a state in which the object work 2 cannot be properly accessed" refers to, for example, a state in which the object work 2 cannot be grasped by the end point or cannot be properly machined by a tool as a result of a position shift, as shown in Fig. 3 before correction.

[0018] Fig. 5 is a model plan view in which only one master work 3 is arranged. Fig. 6 shows a schematic diagram of a change in the position of the object work 2 captured in the image of the camera 4 as a result of the camera 4 being shifted from the initial position. Fig. 7 is a plan view in which four master workpieces 3a to 3d are arranged, similar to Figs. 1 and 2.

[0019] The above correction process will be explained briefly. The matrix of the master work position coordinate system, which is a coordinate system for grasping the position of the master work 3 at the initial position of the camera 4, is defined as M, and the matrix of the master work position coordinate system in a state in which the position of the camera 4 is displaced from the initial position is defined as A. In Fig. 1 and Fig. 2, the three-dimensional coordinates shown within the area surrounded by the master workpieces 3a to 3d are the master work position coordinate system. If the matrix for converting from the coordinate system A to the coordinate system M is defined as I, then M=IA I=MA -1 As shown in the figure. Then, if the work position in coordinate system M is W' and the work position in coordinate system A is W, W'=IW The work position W' can be obtained by

[0020] Here, for ease of explanation, a specific example of a transformation matrix I in a two-dimensional coordinate system and correction using the transformation matrix I will be shown. As shown in FIG. The relationship between the two coordinates, expressed by the transformation coordinate matrix including translation and rotation from coordinates (x, y) to coordinates (x', y'), is given by equation (1). If the camera position change amount is (tx=-1, ty=-1, θ=0), the transformation coordinate matrix I is expressed by equation (2). In coordinate system A after the camera position change, if the position W of the detected object work is (x=0, y=0), the coordinates of the corrected work position W' can be calculated as (x'=-1, y'=-1) as shown in equation (3).

[0021] Next, the operation of this reference example will be described. Figure 9 shows a state in which a calibration marker 8 is placed at the tip of the arm of the robot 1 in order to calculate parameters for converting the coordinate system of the camera 4 into a coordinate system based on the position of the robot 1, that is, the robot coordinate system, when the robot system 11 is installed on-site.

[0022] 10, the camera controller 7 reads the predetermined number N of times an image is captured by the camera 4 (S1). Then, via the robot controller 6, the arm tip of the robot 1, i.e., the calibration marker 8, is moved to the Pth imaging position (S2), and the calibration marker 8 is imaged by the camera 4 at that position (S3). "P" is a counter that counts the number of times imaging is performed, and a position to which the arm tip of the robot 1 is moved is predetermined for each imaging time.

[0023] Then, after incrementing the counter P (S4), it is determined whether the counter P has exceeded the number of photographs taken N (S5), and if "P≦N" (No), the process returns to step S2. If "P>N" is reached (Yes), the process calculates calibration parameters for converting camera coordinates to robot coordinates, and stores and saves them in a memory or the like that is a storage unit inside the camera controller 7 (S6).

[0024] 11 shows the process of converting the positions of the master workpieces 3a-3d in the camera coordinate system into the robot coordinate system. Images of the master workpieces 3a-3d are captured by the camera 4 (S11), and the position of the master workpiece 3 in the camera coordinate system is obtained from the images (S12). Next, these positions are converted into the robot coordinate system using calibration parameters (S13), and the converted coordinate positions are recorded in the memory of the camera controller 7 (S14), ending the process.

[0025] Incidentally, the "position of the master work 3" in step S12, if there is one master work 3, is the position of the center of gravity of the work 3 itself. However, when using master workpieces 3a to 3d as in this reference example, the center of gravity of a figure drawn with these four workpieces 3a to 3d as each point is taken as the "position of the master work 3". This is also true when the number of master workpieces 3 used is "3" or "5" or more. The "position of the master work 3" corresponds to the reference position.

[0026] 12 and 13 show the processing when the robot system 11 is actually operated. When an image capturing the object work 2 is taken by the camera 4 (S21), the position of the object work 2 in the camera coordinate system is obtained from the image (S22). Next, these positions are transformed into the robot coordinate system using the calibration parameters (S23), and the transformed coordinate positions are recorded in the memory of the camera controller 7 (S24).

[0027] When the position of the object work 2 in the robot coordinate system is transmitted to the robot controller 6 (S25), the robot controller 6 moves the hand of the robot 1 to that position and has the tool grasp the object work 2 (S26). Then, it is determined whether or not the grasping has failed (S27). If the grasping has succeeded (No), the process returns to step S1. If the grasping has failed (Yes), the same processes as steps S11 to S14 are performed (S28 to S31). However, the position of the master work 3 in step S31 is the position after the position of the camera 4 has changed.

[0028] 13, using the difference between the position coordinates of the master work 3 recorded in step S31 and the position coordinates of the master work 3 recorded in step S14, a coordinate conversion formula I for converting the coordinate system of the master work 3 at the time of step S31 to the position of the object work 2 in the reference coordinate system of the master work 3 at the time of step S14 is calculated, and recorded in the memory of the camera controller 7 (S32). After performing the same processes as in steps S21 to S23 (S33 to S35), the position of the object work 2 is corrected using the coordinate conversion formula I (S36). After the corrected position of the object work 2 is recorded in the memory of the camera controller 7 (S37), the position is transmitted to the robot controller 6 (S38).

[0029] As described above, according to this reference example, the master work 3 is used to position the object work 2, which is the work target of the robot 1. The camera 4, fixed to the support member 5, captures images of the object work 2 and the master work 3. The camera controller 7 first converts a reference position based on the master work 3 grasped from the image into a position on the robot coordinate system based on the position of the robot 1, and records the converted position in memory as an initial position. In addition, the position of the object work 2 grasped from the image into a position on the robot coordinate system, and sets the converted position as the position of the object work 2.

[0030] Then, if the set position of the object work 2 does not match the actual position of the object work 2 after the initial position is recorded, the camera controller 7 generates a conversion formula I for returning the object work 2 from the current work position coordinate system to the position in the reference coordinate system at the time the initial position was recorded, based on the difference between the reference position grasped from the image at that time. Thereafter, the position of the object work 2 is corrected using the conversion formula I.

[0031] As a result, even if the position of the camera 4 shifts due to the support member 5 expanding or contracting due to an environmental change after the initial position is recorded, or due to an external force being applied to the support member 5, the position of the object work 2 can be corrected so that it is correctly positioned on the robot coordinate system. Therefore, the robot 1 can continue working on the object work 2. In addition, by arranging the four master workpieces 3a to 3d, the camera controller 7 sets the center of gravity of the workpieces 3a to 3d as the reference position, so that the reference position can be set with higher accuracy.

[0032] Furthermore, when the camera controller 7 is notified via the robot controller 6 that the hand of the robot 1 has failed to access the object work 2, the camera controller 7 generates the conversion formula I. Therefore, the position of the object work 2 can be appropriately corrected at the required timing.

[0033] (2nd reference example) Hereinafter, the same parts as those in the first reference example are denoted by the same reference symbols and explanations are omitted, and different parts are explained. In the second reference example, when the robot 1 fails to grasp the object work 2, the robot controller 6 causes the robot 1 to retry the operation of grasping the object work 2. Therefore, instead of steps S26 and S27 shown in FIG. 12, the process shown in FIG. 14 is executed. When a predetermined number N is read as the upper limit of the number of retries (S41), the counter S that counts the number of retries is cleared to zero (S42). Then, the same processes as those in steps S26 and S27 are executed (S43, S44).

[0034] If the answer is "No" in step S43, the process returns to step S42. If the answer is "Yes", the gripping operation of the object work 2 is retried (S45), and the counter S is incremented (S46). Then, the same decision as in step S44 is made (S47), and if the answer is "Yes", it is determined whether the count value of the counter S has exceeded the upper limit number N (S48). If the answer is "No", the process returns to step S45, and if the answer is "Yes", steps S28 to S38 are executed (S49).

[0035] As described above, according to the second reference example, when the hand of the robot 1 fails to access the object work 2, the robot controller 6 retries the access, increments the counter S that counts the number of retries, and resets the counter S when the access to the object work 2 is successful. When the value of the counter S exceeds a predetermined upper limit value N, a notification is given to the camera controller 7, and the camera controller 7 generates the conversion formula I. As a result, the conversion formula I is generated when it is certain that the hand of the robot 1 has failed to access the object work 2.

[0036] (First embodiment) Fig. 15 shows a robot system 12 of the first embodiment. In this embodiment, a tool 13 attached to the arm tip of the robot 1 is used as a positioning marker; a master workpiece. The position of the tool 13 when the robot 1 is in the initial position and posture before starting operation is photographed by the camera 4. In this case, the processes of the first and second reference examples can be applied as they are.

[0037] Second embodiment 16, in a robot system 14 of the second embodiment, a tool 13 is used as a master workpiece as in the first embodiment, but a robot 1 is mounted on an AGV (Automatic Guided Vehicle) 15, which is a moving body. A support member 5 is placed on a workbench 16, and the robot 1 processes an object workpiece placed on the workbench 16. In this case, even if there is no aging deterioration in the support member 5, etc., positional deviation may occur when the AGV 15 returns to a predetermined position.

[0038] FIG. 17 shows a state in which the calibration marker 8 is placed at the arm tip of the robot 1, similarly to FIG. 9. The robot 1 is in the initial position and posture before starting operation, and the AGV 15 is also in the initial position. Then, the process shown in FIG. 10 is executed. Next, while maintaining the above initial position and posture, as shown in FIG. 16, the tool 13 is attached to the arm tip of the robot 1. As shown in FIG. 18, the robot 1 and AGV 15 are moved to the shooting position, which is the above initial position and posture, so that the tool 1 is within the field of view of the camera 4 (S10). Then, steps S11 to S14 are executed.

[0039] Next, as shown in Fig. 19, the object workpiece 17 is placed on the workbench 16, and steps S21 to S27 are executed as shown in Fig. 20. In step S21, both the tool 13 and the object workpiece 17 are within the field of view of the camera 4, but here only the object workpiece 17 is the subject. If the answer is YES in step S27, the same process as in step S10 is executed (S39), and then step S28 and subsequent steps are executed. The processes when the robot system 14 is actually operated are the same as those in Figs. 12 to 14. As described above, the second embodiment can also be applied to a robot system 14 in which the robot 1 is mounted on an AGV 15 that is a mobile carriage.

[0040] The present invention is not limited to the embodiments described above or illustrated in the drawings, and the following modifications and extensions are possible. The function of the camera controller may be provided to the robot controller 6. The camera is not limited to a 3D camera, and for example, two cameras may be used to measure the distance three-dimensionally based on the difference between the distances measured by the two cameras. Robots are not limited to vertical six-axis types. The moving body is not limited to an AGV, but may be, for example, a movable part that moves on a rail. [Explanation of symbols]

[0041] In the drawings, 1 indicates a robot, 2 indicates an object workpiece, 3 indicates a master workpiece, 4 indicates a camera, 5 indicates a support member, 6 indicates a robot controller, 7 indicates a camera controller, 11 and 12 indicate a robot system, 13 indicates a tool, and 15 indicates an AGV.

Claims

1. Robots and The workpiece that this robot is to work on, A marker used for positioning the workpiece; an imaging device that captures an image of the marker and the workpiece while being fixed to a support member; a reference position setting unit that converts a reference position based on the marker initially grasped from the image into a position on a robot coordinate system based on the position of the robot, and sets the converted position as an initial position; a work position setting unit that converts the position of the workpiece grasped from the image into a position on the robot coordinate system and sets the converted position as the position of the workpiece, A tool attached to the end of the arm of the robot is used as the marker; the reference position setting unit, when the position of the workpiece set by the workpiece position setting unit does not coincide with the actual position of the workpiece after the initial position is set, generates a conversion equation for converting the current workpiece position coordinate system into the workpiece position in the reference coordinate system at the time when the initial position is set, based on a difference between the reference position grasped from the image and the initial position; The workpiece position setting unit is a robot system that corrects the position of the workpiece using the conversion formula.

2. The robot system according to claim 1 , wherein the robot is disposed on a moving body.

3. A control device for controlling an operation of the robot is provided, 3. The robot system according to claim 1, wherein the reference position setting unit generates the conversion formula when notified via the control device that an access to the workpiece by the hand of the robot has failed.

4. the control device, when an access to the workpiece by the hand of the robot fails, retries the access to the workpiece and increments a counter that counts the number of retries, resets the counter when the access to the workpiece is successful, and issues the notification when the value of the counter exceeds a predetermined upper limit value; The robot system according to claim 3 , wherein the reference position setting unit generates the conversion formula upon receiving the notification.

5. Robots and The workpiece that this robot is to work on, A marker is a tool attached to a hand of the robot arm and used to position the workpiece; and and an imaging device that captures an image of the marker and the workpiece while being fixed to a support member, First, a reference position based on the marker grasped from the image is transformed into a position on a robot coordinate system based on the position of the robot, and the transformed position is set as an initial position; The position of the workpiece grasped from the image is converted into a position on the robot coordinate system, and the converted position is set as the position of the workpiece; If, after the initial position is set, the set position of the workpiece does not coincide with the actual position of the workpiece, a conversion formula is generated based on the difference between the reference position grasped from the image and the initial position, for converting the current workpiece position coordinate system into the workpiece position in the reference coordinate system at the time when the initial position was set; A position correction method for a robot system, which corrects the position of the workpiece using the transformation formula.

6. 6. The method for correcting a position of a robot system according to claim 5, wherein the robot is disposed on a moving body.

7. The position correction method for a robot system according to claim 5 or 6, wherein the conversion formula is generated when an access to the workpiece by the hand of the robot fails.

8. 8. A position correction method for a robot system according to claim 7, further comprising the steps of: when an attempt to access the workpiece by the robot's hand fails, retrying the access to the workpiece and incrementing a count value that counts the number of retries; when an attempt to access the workpiece is successful, resetting the count value; and when the count value exceeds a predetermined upper limit value, generating the conversion formula.

Images