Robot system

Abstract

To accurately correct a positioning error during cycle in which a work program is repeatedly executed.SOLUTION: A control device 4 of a robot system is provided with: a homogeneous transformation matrix memorizing part 52 that memorizes a homogeneous transformation matrix including an error parameter; a first data generating part 53 that generates first data including at least a difference in measurement position of a robot at two different time points, which are measured with a camera; and a second data generating part 54 that generates second data including at least a difference in calculation position of the robot at two different time points, which are calculated based on the homogeneous transformation matrix; and an error parameter correcting part 55 that corrects the error parameter so that the second data approach the first data.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a robot system that corrects positioning errors in a robot model. [Background technology]

[0002] One of the challenges in controlling the motion of a robot is to accurately correct the robot's positioning error. Known factors that cause positioning error include, for example, joint deflection and backlash. Patent Document 1 calculates the position error between an actual position measured by a measuring device or the like and a calculated position calculated by a robot mechanism model, and identifies joint deflection and backlash so as to minimize this position error. Under the assumption that joint deflection dominates angle errors when joint torque is large at rest, Patent Document 1 measures the robot in a posture in which the joint torque at rest is large, and distinguishes between joint deflection and backlash for identification. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-154527 Summary of the Invention [Problem to be solved by the invention]

[0004] Some recent industrial robots contain aluminum alloys for weight reduction, and as a result of repeated cycles of executing a work program, the link lengths and other parameters can change due to thermal expansion. To address these cycle-related factors, it is desirable to be able to perform positioning error correction processing not only during teaching but also during the cycle.

[0005] However, in Patent Document 1, it is necessary to measure the actual position in a posture that increases the joint torque when stationary. Measurements with such posture restrictions can be performed during teaching, but cannot be performed during a cycle. Therefore, it is not possible to address factors that occur during a cycle, and there is a risk that positioning errors will increase during the cycle.

[0006] The present invention has been made in view of the above-mentioned problems, and a main object of the present invention is to provide a robot system that can accurately correct positioning errors during cycles in which a work program is repeatedly executed. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, the present invention provides a robot system comprising a robot, a camera that measures the position of the robot, and a control device that controls the operation of the robot and the camera, wherein the control device comprises: a homogeneous transformation matrix storage unit that stores a homogeneous transformation matrix that includes error parameters related to the robot; a first data generation unit that generates first data measured by the camera and including at least a difference between measured positions of the robot at two different points in time; a second data generation unit that generates second data calculated based on the homogeneous transformation matrix and including at least a difference between calculated positions of the robot at the two points in time; and an error parameter correction unit that corrects the error parameters so that the second data becomes closer to the first data.

[0008] The error parameter correction unit may learn the error parameters using a deep metric learning method, with the first data as training data and the second data as training data.

[0009] The error parameter correction unit may also learn the error in the link length of the robot, the error in the link torsion angle, the error in the distance between the links, the resistance force of joint bending, and the magnitude of backlash as the error parameters. [Effects of the Invention]

[0010] The present invention provides a robot system that can accurately correct positioning errors during cycles of repeating execution of a task program. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an outline of a robot system according to an embodiment of the present invention. [Figure 2] A block diagram showing the configuration of the control device of FIG. 1. [Figure 3] 1 is a flowchart showing an example of a process flow realized by the control device of FIG. [Figure 4] FIG. 4 is a diagram for explaining the flowchart of FIG. 3. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Fig. 1 is a diagram illustrating an outline of a robot system according to an embodiment of the present invention. As shown in Fig. 1, the robot system 1 includes a robot 2 having a robot arm 21, a camera 3 that measures the position of the robot 2, and a control device 4 that controls the operation of the robot 2 and the camera 3.

[0013] The robot 2 has an installation base 22 that supports a robot arm 21, and a tool (end effector) 23 is attached to the tip of the robot arm 21. The robot arm 21 is composed of a link mechanism having multiple links and has joints that connect the links. Each joint is provided with a drive motor (not shown). An example of the robot 2 is a vertical multi-joint robot with six joints. However, the present invention is also applicable to robots with five or fewer joints or seven or more joints, horizontal multi-joint robots, etc.

[0014] The camera 3 is, for example, a three-dimensional sensor capable of measuring the three-dimensional position of the tool 23 of the robot 2. The three-dimensional sensor generates point cloud data, which is a collection of points indicating the positions of detected points on the surface of an object. The point cloud data generated by the three-dimensional sensor is Cartesian coordinate point cloud data expressed in three-dimensional Cartesian coordinates (x, y, z). The three-dimensional sensor may be, for example, an active stereo camera, a passive stereo camera, a three-dimensional LiDAR (Light Detection And Ranging) sensor, or a ToF (Time of Flight) sensor. Examples include, but are not limited to, a flight camera.

[0015] The camera 3 is not limited to a three-dimensional sensor, and may be a two-dimensional camera, and is not particularly limited. Furthermore, the camera 3 may be installed on the robot 2 (tool 23, etc.) or may be fixed to a location other than the robot 2 using a stand or the like, as long as it can capture an image of a measurement target such as the tool 23.

[0016] The control device 4 includes a robot control device 4a that controls the operation of the robot 2, and a camera control device 4b that controls the operation of the camera 3. The robot control device 4a may be built into the installation base 22 of the robot 2, or may be installed outside the robot 2. In the latter case, the robot 2 and the robot control device 4a are communicatively connected via a communication cable or wirelessly. Similarly, the camera control device 4b may be integrated with the camera 3, or may be separate. In the latter case, the camera 3 and the camera control device 4b are communicatively connected via a communication cable or wirelessly. Furthermore, the robot control device 4a and the camera control device 4b are communicatively connected via a communication cable or wirelessly.

[0017] The CPU (Central Processing Unit) 41a, memory 42a, auxiliary storage device 43a, and input / output interface 44a of the robot control device 4a are connected via a bus 45a. The CPU 41a loads a control program stored in advance in the auxiliary storage device 43a or the like into the memory 42a and sequentially executes a plurality of commands. The auxiliary storage device 43a is a hard disk drive, solid state drive, or the like, and stores data used in the processing described below. The input / output interface 44a inputs and outputs signals from the robot 2, camera control device 4b, other devices, and the like.

[0018] The camera control device 4b's CPU (Central Processing Unit) 41b, memory 42b, auxiliary storage device 43b, and input / output interface 44b are connected via a bus 45b. The CPU 41b loads a control program stored in advance in the auxiliary storage device 43b or the like into the memory 42b and sequentially executes a plurality of commands. The auxiliary storage device 43b is a hard disk drive, solid state drive, or the like, and stores data used in the processing described below. The input / output interface 44b inputs and outputs signals from the camera 3, the robot control device 4a, and other devices.

[0019] 1 shows two control devices 4, namely, a robot control device 4a and a camera control device 4b, but the number of control devices 4 may be one or three or more. In the following, the robot system 1 will be described as having two control devices 4, a robot control device 4a and a camera control device 4b, and these control devices 4 execute processing in cooperation with each other.

[0020] Fig. 2 is a block diagram showing the configuration of the control device in Fig. 1. The robot control device 4a and the camera control device 4b have the configuration shown in Fig. 2 through cooperation between software such as a control program and hardware resources such as CPUs 41a and 41b.

[0021] The robot control device 4a includes a robot operation control unit 51, a homogeneous transformation matrix storage unit 52, a first data generation unit 53, a second data generation unit 54, and an error parameter correction unit 55. The robot operation control unit 51 controls the rotation and stopping of the drive motor of the robot arm 21 based on a control program, commands from an external device, etc.

[0022] The homogeneous transformation matrix storage unit 52 stores a homogeneous transformation matrix including error parameters related to the robot 2. Before describing the error parameters and homogeneous transformation matrix, the notation of mathematical expressions used in this specification will be explained. Generally, superscripts and subscripts are used to denote subscripts in mathematical expressions. However, because superscripts and subscripts are difficult to read in text, curly brackets {} are used instead of superscripts, and square brackets [] are used instead of subscripts when denoting subscripts in text. For example, when subscript i is added to the lower right of x, it is written as x[i]. Similarly, when subscript 0 is added to the upper left of T and subscript n is added to the lower right, it is written as {0}T[n]. However, when not in text, but in Mathematical Expressions 1, 2, and FIG. 4 described below, superscripts and subscripts are used to denote subscripts.

[0023] In order for the robot control device 4a to calculate the position and posture of the tool 23 of the robot 2, etc., it is necessary to set a coordinate system for each link. The relationship between the coordinate systems of the links is expressed by link parameters. A known method for setting the coordinate systems of the links is, for example, the Denavit-Hartenberg notation (DH method). Link parameters using the DH method are also called DH parameters.

[0024] The link coordinate system using the DH method is defined as follows: 1. Links and joints are numbered in order of proximity to the base of robot 2. 2. For link i, a coordinate system Σ[i] is set with joint axis i as the z[i] axis. 3. The x[i] axis is the common perpendicular to the z[i] axis and z[i+1] axis, and the direction toward the z[i+1] axis is considered positive. 4. The y[i] axis is set to form a right-handed coordinate system. 5. The link parameters are: (1) link length a[i] = distance from z[i-1] axis to z[i] axis along x[i-1] axis, (2) link torsion angle α[i] = angle from z[i-1] axis to z[i] axis relative to x[i-1] axis, (3) link-to-link distance d[i] = distance from x[i-1] axis to x[i] axis along z[i] axis, and (4) link-to-link angle θ[i] = angle from x[i-1] axis to x[i] axis relative to z[i] axis.

[0025] In an embodiment of the present invention, error parameters are introduced to expand the link parameters in order to correct positioning errors. The error parameters include five: link length error Δa[i], link torsion angle error Δα[i], link-to-link distance error Δd[i], indirect deflection resistance force c[i], and backlash magnitude w[i]. Here, backlash refers to a gap intentionally created in the direction of motion when a pair of gears mesh with each other and move. The backlash magnitude w[i] is the angle of this gap.

[0026] The errors expressed by these error parameters include manufacturing errors, assembly errors, etc. Furthermore, if the robot 2 includes an aluminum alloy material, changes in link length, etc. due to thermal expansion may occur during repeated cycles of execution of a task program, which may manifest as errors. In other words, the values ​​of some error parameters may change during the robot 2 cycle.

[0027] Hereinafter, link parameters extended by introducing error parameters are referred to as extended link parameters. The design value of the link length is a'[i], the design value of the link torsion angle is α'[i], the design value of the link-to-link distance is d'[i], the input value of the link-to-link angle (= joint angle) is θ'[i], and the input value of the joint torque is τ[i]. Furthermore, when the torque is 0, torque that changes from a positive direction to 0 is represented as +0, and torque that changes from a negative direction to 0 is represented as -0. In this case, the extended link parameters a[i], α[i], d[i], and θ[i], which introduce the error parameters Δa[i], Δα[i], Δd[i], c[i], and w[i], are defined by the following equations (1) to (4).

[0028]

number

[0029] The sign in equation (4) takes the value of w[i] when the joint torque τ[i] is positive or changes from a positive direction to 0, and takes the value of -w[i] when the joint torque τ[i] is negative or changes from a negative direction to 0. In other words, the sign is a function whose sign is determined by which side the pair of gears are in contact with each other.

[0030] The homogeneous transformation matrix storage unit 52 stores a homogeneous transformation matrix expressed by the extended link parameters shown in equations (1) to (4), in other words, a homogeneous transformation matrix including error parameters. Here, "storing a homogeneous transformation matrix" means storing not only the formula of the homogeneous transformation matrix but also the values ​​of the constants included in the homogeneous transformation matrix. That is, the homogeneous transformation matrix storage unit 52 also stores the values ​​of the design value a'[i] of the link length, the design value α'[i] of the link torsion angle, and the design value d'[i] of the inter-link distance, which are included in equations (1) to (4). For example, if the robot 2 is a six-axis vertical articulated robot, the homogeneous transformation matrix {i-1}T[i] from the coordinate system Σ[i-1] to the coordinate system Σ[i] and the coordinate system Σ[t] of the tool are expressed by the following equations:

[0031]

number

[0032] The definition of the extended link parameters is not limited to equations (1) to (4) and may be any differentiable value. For example, the inter-link angle θ[i] may be defined as follows: "If the absolute value of the change in the joint angle θ'[i] is smaller than the magnitude of backlash w[i], the inter-link angle θ[i] does not change."

[0033] The first data generation unit 53 generates first data measured by the camera 3 and including the difference between at least the measured positions of the robot at two different times. The second data generation unit 54 generates second data calculated based on a homogeneous transformation matrix and including the difference between at least the calculated positions of the robot at the two times. The error parameter correction unit 55 corrects the error parameters so that the second data approaches the first data. The first data and second data may include not only the difference between the measured positions and calculated positions of the robot at the two times, but also the difference between the measured attitudes and calculated attitudes of the robot at the two times.

[0034] The error parameter correction unit 55 learns the value of the error parameter by machine learning such as deep learning. For example, the error parameter correction unit 55 uses the first data as training data and the second data as training data to learn the error parameter using a learning method of deep metric learning, and stores the learning result in the homogeneous transformation matrix storage unit 52 as a new error parameter.

[0035] The deep metric learning learning method is a method for learning whether two pieces of input data are semantically similar or dissimilar by providing training data indicating whether the data are "same" or "different." In an embodiment of the present invention, the magnitude of the difference to be calculated, i.e., the difference in the actual movement of the robot 2 (=first data), is known. Therefore, the error parameter correction unit 55 substitutes input data (=joint angles and joint torques) at two points in time into the same model (=homogeneous transformation matrix), and learns error parameters so that the difference in the output data (=second data) approaches the actual difference (=first data).

[0036] Details of deep metric learning are disclosed below. ·Lev V. Utkin, Maxim S. Kovalev, Ernest M. Kasimov, “An explanation method for Siamese neural networks”, [online], November 18, 2019, arXiv, [Retrieved February 1, 2024], Internet<URL:https: / / arxiv.org / ftp / arxiv / papers / 1911 / 1911.07702.pdf>

[0037] The camera control device 4b includes a camera operation control unit 61 and a measurement data providing unit 62. The camera operation control unit 61 controls the operation of the camera 3 based on a control program, commands from an external device, etc. The measurement data providing unit 62 provides the robot control device 4a with three-dimensional Cartesian coordinates indicating the position of the robot 2 measured by the camera 3. The measurement data providing unit 62 may also provide the robot control device 4a with roll angle, pitch angle, and yaw angle indicating the attitude of the robot 2 measured by the camera 3.

[0038] Fig. 3 is a flowchart showing an example of the flow of processing realized by the control device of Fig. 1. Fig. 3 particularly shows the flow of processing by the error parameter correction unit 55. The error parameter correction unit 55 learns five error parameters of the robot 2: the link length error Δa[i], the link torsion angle error Δα[i], the inter-link distance error Δd[i], the joint deflection resistance force c[i], and the backlash magnitude w[i].

[0039] As shown in FIG. 3, the first data generation unit 53 of the robot control device 4a generates first data measured by the camera 3, including the difference between the measured positions (and measured attitudes, if necessary) of the robot 2 at two different times A and B (step S1). Specifically, the first data generation unit 53 acquires three-dimensional Cartesian coordinates indicating the measured positions of specific locations on the robot 2 at the two different times A and B from the measurement data provision unit 62 of the camera control device 4b. Furthermore, the first data generation unit 53 acquires roll angles, pitch angles, and yaw angles indicating the measured attitudes of the specific locations on the robot 2 at the two different times A and B from the measurement data provision unit 62 of the camera control device 4b, if necessary. The first data generation unit 53 changes at least one of the times A and B to a different time each time it returns from step S4 and executes step S1.

[0040] In the embodiment of the present invention, there are no limitations on the position and posture of the robot 2 at two points in time A and B. Furthermore, the order of the points in time may be in chronological order or in an order not dependent on time, and is not particularly limited.

[0041] For example, when the camera 3 is installed on the tool 23, a marker recognizable by the camera 3 is installed on the tool 23. The marker is installed within the shooting range of the camera 3. In this case, the position and posture of the robot 2 at two points in time A and B are the position and posture of the marker. Then, the first data generation unit 53 generates the difference between the measured position (and measured posture, if necessary) of the marker at the two points in time A and B as the first data.

[0042] Furthermore, when the camera 3 is installed in a location other than the robot 2, the position and posture of the robot 2 at two points in time A and B are, for example, the position and posture of a specific location of the tool 23. Then, the first data generation unit 53 generates the difference between the measured position (and measured posture, if necessary) of the specific location of the tool 23 at the two points in time A and B as the first data.

[0043] Next, the second data generation unit 54 substitutes the values ​​of the joint angles and joint torques at two different times A and B into a homogeneous transformation matrix to calculate calculated positions (and calculated postures, if necessary) at the two different times A and B. Then, the second data generation unit 54 generates second data including the difference between the calculated positions (and calculated postures, if necessary) at the two different times A and B (step S2).

[0044] More specifically, the second data generation unit 54 calculates the homogeneous transformation matrices of the formulas (5) and (6) by substituting the joint angle for θ'[i] and the joint torque for τ[i] in formula (4), and calculates the calculated position. For the error parameter value, 0 or a value stored in the homogeneous transformation matrix storage unit 52 is used for the first calculation, and a value adjusted in step S3 (described later) is used for the second calculation or later. The second data generation unit 54 then determines the difference between the calculated positions at the two points in time A and B as the second data. Furthermore, if a calculated posture is required, the second data generation unit 54 calculates the calculated posture based on the homogeneous transformation matrix related to the posture, and determines the difference between the calculated postures at the two points in time A and B as the second data.

[0045] The second data generating unit 54 may, for example, acquire the measurement value of the joint angle from an encoder mounted on the drive motor of each joint. Alternatively, the second data generating unit 54 may, for example, calculate the joint torque based on data from a sensor installed in the robot 2 or a design value of the robot 2.

[0046] Next, the error parameter corrector 55 learns error parameters using a deep metric learning technique so that the second data generated in step S2 becomes closer to the first data generated in step S1 (step S3). For example, the error parameter corrector 55 learns the error parameters so that a loss function representing the deviation between the first data, which is training data, and the second data, which is training data, is minimized. An example of the loss function is squared error, but is not particularly limited thereto.

[0047] Next, the error parameter corrector 55 checks whether the learning termination condition is met (step S4). If the learning termination condition is not met (No in step S4), the error parameter corrector 55 repeats from step S1. In the next step S2, the error parameter corrector 55 uses the error parameter value adjusted in step S3. If the learning termination condition is met (Yes in step S4), the error parameter corrector 55 stores the learning result as a new error parameter in the homogeneous transformation matrix storage unit 52 (step S5) and ends the processing. The learning termination condition may be, for example, that the number of times steps S1 to S4 are repeated reaches a predetermined number, that the value of the loss function becomes equal to or less than a predetermined value, or any other condition.

[0048] FIG. 4 is a diagram illustrating the flowchart of FIG. 3. As shown in FIG. 4, the first data serving as training data is the difference between the measured position M[A] at time A and the measured position M[B] at time B. The model of the robot 2 is represented by the extended link parameters shown in equations (1) to (4) and the homogeneous transformation matrices shown in equations (5) and (6). When the six-axis joint angles θ′{A}[1] to θ′{A}[6] at time A and the six-axis joint torques τ{A}[1] to τ{A}[6] at time A are input to the model of the robot 2, a calculated position C[A] at time A is output. When the six-axis joint angles θ′{B}[1] to θ′{B}[6] at time B and the six-axis joint torques τ{B}[1] to τ{B}[6] at time B are input to the model of the robot 2, a calculated position C[B] at time B is output. The difference between the calculated position C[A] at time A and the calculated position C[B] at time B is the second data as learning data. Furthermore, the loss function L is the squared error between the first data and the second data, that is, L=((C[B]-C[A])-(M[B]-M[A]))^2. Note that "X^2" means the square of X. Then, the error parameter correction unit 55 learns the error parameters so that the loss function L is minimized.

[0049] In the above description, the error parameters to be learned are the five shown in Equations (1) to (4), and the input data are joint angles and joint torques. However, the types of error parameters to be learned and the input data are not limited to these. For example, the error parameters to be learned may be only the link length error Δa[i], the link torsion angle error Δα[i], and the link-to-link distance error Δd[i], and Equation (4) may be set to θ[i] = θ'[i], and the input data may be only the joint angles. Furthermore, for example, the error parameters to be learned may be not only the five shown in Equations (1) to (4), but also other parameters may be added, and extended link parameters that more precisely model joint deflection and backlash may be used. In this case, the input data may include other parameters in addition to the joint angles and joint torques.

[0050] As described above, in the robot system 1 according to the embodiment of the present invention, the control device 4 generates first data measured by the camera 3 and including at least the difference between the measured positions of the robot 2 at two different points in time. Next, the control device 4 generates second data calculated based on a homogeneous transformation matrix and including at least the difference between the calculated positions of the robot at the two points in time. The control device 4 then corrects the error parameters so that the second data approaches the first data.

[0051] For example, as in Patent Document 1, if there are restrictions on the position and posture of the robot 2, it is difficult to correct the error parameters during the cycle. In the robot system 1 according to the embodiment of the present invention, there are no restrictions on the position and posture of the robot 2 and it can be applied to any work program, so it is possible to correct the error parameters during the cycle.

[0052] Furthermore, in conventional techniques such as Patent Document 1, the absolute position of the robot 2 is measured to correct the error parameters. However, it is difficult to accurately measure the absolute position of the robot 2 due to errors caused by various factors, such as an installation error of the robot 2. In an embodiment of the present invention, the absolute position of the robot 2 is not used, but the difference between the positions measured at two points in time is used. Therefore, in an embodiment of the present invention, the influence of errors caused by various factors, such as an installation error of the robot 2, can be reduced, and ultimately the error parameters can be corrected with high accuracy.

[0053] Furthermore, the robot control device 4a according to the embodiment of the present invention uses the first data as training data and the second data as training data to learn error parameters using a deep metric learning learning method. As described above, deep metric learning is a method for learning whether two pieces of input data are semantically similar or dissimilar by providing training data indicating whether the data are "same" or "different." Therefore, deep metric learning is suitable for cases where the training data is the difference between measured positions at two points in time, rather than the absolute position of the robot 2, as in the embodiment of the present invention. Furthermore, since deep metric learning can be applied regardless of how the model is defined, the learning method can be applied as is even if the definition of the extended link parameters is changed depending on the type of robot 2, the environment, etc. Furthermore, according to the embodiment of the present invention, a learning model can be constructed with a small number of parameters, and training data can be acquired in real time during the repeated execution of the task program. Therefore, the error parameters can be accurately corrected during the cycle.

[0054] While the preferred embodiments of the robot system and the like according to the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to these examples. It is clear that those skilled in the art can conceive of various modifications and alterations within the scope of the technical ideas disclosed herein, and it is understood that these modifications and alterations also fall within the technical scope of the present invention. [Explanation of symbols]

[0055] 1. Robot system 2. Robot 3. Camera 4. Control device 4a...Robot control device 4b: Camera control device 51: Robot operation control unit 52: Homogeneous transformation matrix storage section 53...First data generation unit 54...Second data generation unit 55: Error parameter correction section 61: Camera operation control section 62...Measurement data provision section

Claims

1. A robot system including a robot, a camera that measures the position of the robot, and a control device that controls operations of the robot and the camera, The control device a homogeneous transformation matrix storage unit that stores a homogeneous transformation matrix including error parameters related to the robot; a first data generating unit configured to generate first data measured by the camera and including a difference between at least the measured positions of the robot at two different points in time; a second data generation unit that generates second data calculated based on the homogeneous transformation matrix and including a difference in at least the calculated positions of the robot at the two time points; an error parameter correction unit that corrects the error parameter so that the second data becomes closer to the first data; A robot system comprising:

2. The error parameter correction unit uses the first data as training data and the second data as training data to learn the error parameters using a deep metric learning method.

2. The robot system according to claim 1.

3. The error parameter correction unit learns the error of the robot's link length, the error of the link torsion angle, the error of the distance between the links, the resistance force of joint bending, and the magnitude of backlash as the error parameters.

3. The robot system according to claim 2.

Images