Program generation device, robot system, and program generation method

Abstract

A program generation device (10) comprises at least one memory and at least one processor, wherein the memory stores a first point sequence indicating pair information comprising a pair of a time and a position of at least one point on an object (30) when causing a target movement to be performed on the object (30), and the processor generates a movement program for causing a robot (20) holding the object (30) to reproduce the target movement on the basis of the first point sequence stored in the memory, causes the robot (20) to move according to the generated movement program, generates a second point sequence indicating pair information at the time of said movement, and corrects a parameter related to the movement of the robot (20) in the movement program on the basis of a comparison between the first point sequence and the second point sequence.

Description

Program generation device, robot system and program generation method

[0001] The present disclosure relates to a program generation device, a robot system, and a program generation method.

[0002] A control device is known that controls a sprayer, a painting robot that paints while moving the sprayer, and a sensor that detects position data of the sprayer (see, for example, Patent Document 1). In order to achieve uniform painting, this control device calculates the movement speed of the sprayer based on position data of the sprayer at each time detected by the sensor, and adjusts the painting conditions of the sprayer according to the calculated movement speed.

[0003] Japanese Patent Application Laid-Open No. 2004-337710

[0004] In some cases, a painting robot is made to imitate the painting actions of a worker to perform painting work on behalf of the worker. However, if the painting actions of the worker include sudden accelerations and sudden stops, the painting robot may not be able to accurately imitate the painting actions of the worker, resulting in an inability to produce a uniform coating. Therefore, it is desirable to have a robot that can more accurately imitate the actions of a worker.

[0005] One aspect of the present disclosure is a program generation device comprising at least one memory and at least one processor, wherein the memory stores a first sequence of points indicating paired information consisting of a pair of position and time of at least one point on an object when the object is made to perform a target action, and the processor generates an operation program, based on the first sequence of points stored in the memory, for causing a robot holding the object to reproduce the target action, causes the robot to operate in accordance with the generated operation program, generates a second sequence of points indicating the paired information at that time, and modifies parameters related to the robot's operation in the operation program based on a comparison between the first sequence of points and the second sequence of points.

[0006] Fig. 3 is a schematic diagram showing a robot system according to an embodiment of the present disclosure; Fig. 4 is a side view showing a tool in the robot system of Fig. 1; Fig. 5 is a block diagram showing a configuration of a program generation device according to an embodiment of the present disclosure; Fig. 6 is an example of a first point sequence generated by a processor of the program generation device of Fig. 3; Fig. 7 is a flowchart showing a program generation method according to an embodiment of the present disclosure;

[0007] A robot system 1 and a program generation device according to an embodiment of the present disclosure will be described below with reference to the drawings. As shown in Fig. 1, the robot system 1 according to this embodiment includes a robot 20, a control device 10, a tool (object) 30 attached to the tip of the robot 20, and a sensor 40 disposed outside the robot 20. The control device 10 controls the robot 20, the tool 30, and the sensor 40, and also functions as a program generation device according to the present disclosure.

[0008] 1, the robot 20 is, for example, a six-axis vertical articulated robot that is a painting robot that imitates the painting motion (target motion) of a worker P. The robot 20 includes a base 21 that is placed on a mounting surface such as a horizontal floor, and a rotating body 22 that is rotatable relative to the base 21 about a vertical first axis A. The robot 20 also includes a first arm 23 that is rotatable relative to the rotating body 22 about a horizontal second axis B, and a second arm 24 that is rotatable relative to the first arm 23 about a third axis C that is parallel to the second axis B. The robot 20 also includes a three-axis wrist unit 25 supported at the tip of the second arm 24.

[0009] The tool 30 is, for example, a paint gun that sprays paint onto a workpiece (not shown). As shown in Fig. 1, the tool 30 is detachably attached to a flange surface 25f provided at the tip of the wrist unit 25 of the robot 20. That is, by operating the robot 20, the tool 30 can be moved to a desired position and posture. Furthermore, the tool 30 detached from the flange surface 25f can also be manually operated by an operator P, as shown in Fig. 1.

[0010] 1 and 2, the sensor 40 is, for example, a motion capture device, and includes markers 41 and 42 attached to the outer surface of the tool 30 and a camera 43 that captures images of the markers 41 and 42. The marker 41 is attached to the outer surface near the nozzle 31 at the tip of the tool 30. On the other hand, the marker 42 is attached to the outer surface on the same side as the marker 41, near the rear end of the tool 30. In other words, the straight line O connecting the markers 41 and 42 is arranged parallel to the direction in which the nozzle 31 extends, i.e., the direction in which the paint is sprayed.

[0011] When the camera 43 is to capture markers 41, 42 attached to the tool 30 attached to the robot 20, the camera 43 is installed at a position away from the robot 20 by a distance that allows the tool 30 operated by the robot 20 to fit within the angle of view. Similarly, when the camera 43 is to capture markers 41, 42 attached to the tool 30 held by the worker P, the camera 43 is installed at a position away from the worker P by a distance that allows the tool 30 operated by the worker P to fit within the angle of view.

[0012] The sensor 40 includes at least one processor, such as a CPU (not shown), and at least one memory, such as a ROM or RAM. The processor in the sensor 40 calculates position data of the markers 41, 42 at each time based on a plurality of captured images acquired by the camera 43 capturing images of the markers 41, 42 within the field of view at predetermined time intervals. The position data is pair information that associates the positions of the markers 41, 42 in the global coordinate system with the times when the markers 41, 42 were captured at those positions. The calculated pair information is stored in the memory in the sensor 40.

[0013] When the camera 43 photographs the tool 30 operated by the worker P, the sensor 40 acquires paired information I11, I12 of the positions and times in the global coordinate system of the markers 41, 42 attached to the tool 30 operated by the worker P. When the camera 43 photographs the tool 30 operated by the robot 20, the sensor 40 acquires paired information I21, I22 of the positions and times in the global coordinate system of the markers 41, 42 of the tool 30 operated by the robot 20.

[0014] 3, the control device 10 includes at least one memory 11 such as a ROM or a RAM, and at least one processor 12 such as a CPU. The control device 10 also includes an input device 13 such as a keyboard, and a transmitting / receiving device 14 that transmits and receives signals to and from the robot 20, the tool 30, and the sensor 40.

[0015] The memory 11 stores a system program that performs the basic functions of the control device 10. When the processor 12 receives pair information I11 and I12 from the sensor 40 via the transmitting / receiving device 14, it generates a first point sequence based on the pair information I11 and I12. The generated first point sequence is stored in the memory 11.

[0016] As shown in FIG. 4, the first point sequence is a data group indicating the positions of the markers 41 and 42 attached to the tool 30 operated by the worker P at each time. The position information a of the markers 41 and 42 included in the first point sequence is １ ~a ｎ , b １ ~b ｎ is obtained by converting the position data in the global coordinate system of the pair information I11 and I12 into the robot coordinate system and chronologically arranging them. The robot coordinate system in this case is a coordinate system that serves as a reference for controlling the robot 20, and is a three-axis orthogonal coordinate system whose origin is the intersection of the horizontal floor surface on which the base 21 is installed and the first axis A, and is defined at a position different from that of the global coordinate system.

[0017] The processor 12 calculates the position information a included in the first point sequence. １ ~a ｎ , b １ ~b ｎ The processor 12 calculates a movement path of the tool 30 operated by the worker P, which is converted from the movement path in the global coordinate system to the robot coordinate system, based on the position information a １ ~a ｎ , b １ ~b ｎ and time information t １ ~t ｎ Based on this, the moving speed of the tool 30 operated by the worker P at each position on the moving path is calculated.

[0018] Then, based on the movement path and movement speed calculated from the first point sequence, the processor 12 generates an operation program for causing the robot 20, to which the tool 30 is attached, to reproduce the painting operation performed by the worker P. The generated operation program is stored in the memory 11.

[0019] When the processor 12 executes the operation program stored in the memory 11, the robot 20 moves the tool 30 attached to the flange surface 25f so as to reproduce the movement of the tool 30 operated by the worker P. At that time, the processor 12 also causes the sensor 40 to acquire paired information I21 and I22 of the positions and times in the global coordinate system of the markers 41 and 42 attached to the tool 30 moved by the robot 20.

[0020] Then, similar to the first point sequence, the processor 12 generates a second point sequence based on the acquired pair information I21 and I22 and compares it with the first point sequence stored in the memory 11. Based on the result of comparing the two point sequences, the processor 12 modifies the command values ​​(parameters) that define the movement speed of the robot 20 and are included in the operation program stored in the memory 11. The modified operation program is overwritten and stored in the memory 11.

[0021] A program generation method using the program generation device (control device) 10 according to this embodiment configured as described above will be described with reference to the flowchart shown in FIG.

[0022] First, a worker P holding the tool 30 performs painting work, and the markers 41, 42 attached to the tool 30 are photographed by the camera 43. As a result, the sensor 40 acquires pair information I11, I12 that associates the positions in the global coordinate system of the markers 41, 42 attached to the tool 30 operated by the worker P with each time (step S1).

[0023] The paired information I11 and I12 acquired by the camera 43 are sent to the processor 12 of the program generation device 10 by the transmitting / receiving device 14. Then, as shown in Fig. 4, the processor 12 performs coordinate transformation on the acquired paired information I11 and I12 and aligns them in time series to generate a first sequence of points (step S2). Based on the generated first sequence of points, the processor 12 calculates the movement path, posture, and movement speed of the tool 30 operated by the worker P, which are obtained by transforming the movement path into the robot coordinate system.

[0024] Then, based on the calculated movement path, posture, and movement speed of the tool 30, the processor 12 generates an operation program for causing the robot 20 to reproduce the painting operation performed by the worker P (step S3). The operation program can basically be configured by setting the coordinates of multiple teaching points located on the movement path of the tool tip point set in the robot 20, and the movement speed toward the teaching points. The generated operation program is stored in the memory 11.

[0025] Thereafter, the processor 12 attaches the tool 30 with the markers 41 and 42 attached to the flange surface 25f at the tip of the robot 20, and executes the operation program read from the memory 11 while operating the camera 43. As a result, the robot 20 is operated so as to reproduce the movement of the tool 30 operated by the worker P (step S4). At this time, the markers 41 and 42 attached to the tool 30 operated by the robot 20 are photographed by the camera 43.

[0026] As a result, the sensor 40 acquires paired information I21, I22 indicating the positions at each time of the markers 41, 42 attached to the tool 30 operated by the robot 20 that reproduces the movement of the worker P (step S5). Then, the acquired paired information I21, I22 is sent to the processor 12 of the program generation device 10 by the transmitting / receiving device 14.

[0027] Next, the processor 12 performs coordinate transformation on the acquired paired information I21 and I22, similar to the first point sequence, and generates a second point sequence by chronologically aligning them (step S6). The processor 12 then compares the second point sequence with the first point sequence read from the memory 11. The comparison between the first point sequence and the second point sequence is performed using, for example, dynamic time warping (DTW).

[0028] Specifically, based on the position information of the first sequence of points and the second sequence of points, the system searches for a path that minimizes the time difference between the corresponding points. That is, the system compares the points associated with each other through the searched paths. Therefore, even if there is a time difference between the first sequence of points and the second sequence of points, the system can directly compare the two sequences of points by allowing for nonlinear expansion and contraction along the time axis, enabling accurate comparison of the two sequences of points.

[0029] The processor 12 calculates the time difference between the points in the first sequence of points and the second sequence of points that are associated using the DTW method, and determines whether the difference is less than or equal to a predetermined threshold value that has been stored in advance in the memory 11 (step S7).

[0030] If the result of the determination is that the time difference between corresponding points in both sequences of points is greater than a predetermined threshold, the processor 12 determines that the robot 20 is not able to accurately reproduce the painting action of the worker P. The processor 12 then calculates a coefficient based on the time difference between corresponding points in both sequences of points, multiplies the coefficient by a command value that defines the movement speed of the robot 20, included in the operation program, and modifies the operation program (step S8). As a result, the command value that defines the movement speed of the robot 20 in the operation program is modified so that the time difference between corresponding points in both sequences of points becomes smaller. The modified operation program is overwritten in the memory 11. The controls of steps S4 to S8 are repeated until the result of the determination in step S7 becomes equal to or less than the predetermined threshold.

[0031] On the other hand, if the result of the judgment in step S7 is below the predetermined threshold, the processor 12 determines that the robot 20 is able to accurately reproduce the painting operations of the worker P and finalizes the operation program without modifying it.

[0032] As described above, according to this aspect, the robot 20 detects the movement of the tool 30 when replicating the painting action of the worker P, and calculates the difference between that movement and the movement of the tool 30 operated by the worker P. Then, the operation commands included in the operation program are modified so that the calculated difference becomes smaller. As a result, the robot 20 can apply the modified operation program the next time the robot 20 performs a reproduction operation, and the robot 20 can more accurately reproduce the painting action of the worker P.

[0033] In this case, the DTW method was used to compare the painting actions performed by the worker P with the actions reproduced by the robot 20. This allows for a time difference between the two actions to be tolerated, and based on the comparison results, the actions reproduced by the robot 20 can be adjusted to match the painting actions of the worker P. Therefore, even if the painting actions of the worker P include quick movements such as sudden acceleration or sudden stops, the robot 20 can accurately reproduce the painting actions.

[0034] In this embodiment, the command value that defines the movement speed of the robot 20, which is included in the operation program, is modified based on a comparison between the first sequence of points and the second sequence of points. Alternatively, the command that defines the acceleration of the robot 20 or the command that defines the movement path, which are included in the operation program, may be modified. Alternatively, both the command that defines the movement speed of the robot 20 and the command that defines the movement path may be modified.

[0035] In this embodiment, the processor 12 generates the first sequence of points and the operation program based on the paired information I11 and I12 acquired by the sensor 40. Alternatively, if information such as the movement path and movement speed of the painting operation to be performed by the robot 20 is already known, the information may be stored in advance in the memory 11. In this case, the processor 12 can omit the step of detecting the movement of the tool 30 operated by the worker P using the sensor 40 before generating the first sequence of points and the operation program.

[0036] In this embodiment, the processor 12 acquires the pair information I21 and I22 by having the camera 43 capture an image of the tool 30 operated by the robot 20 that reproduces the painting action of the worker P. Alternatively, the processor 12 may estimate the pair information I21 and I22 based on angle information of each joint of the robot 20 that reproduces the painting action of the worker P.

[0037] In this case, the posture information of the robot 20 and the position or posture of the tool 30 operated by the robot 20 may be associated by calibration. As a result, the processor 12 acquires the posture of the robot 20 based on the detection values ​​of the angle sensors provided at each joint of the robot 20, and estimates pair information I21 and I22 of the robot 20 that reproduces the painting operation based on the posture.

[0038] In addition, in this embodiment, if the sensor 40 has been calibrated in advance with the robot coordinate system, the pair information acquired by the sensor 40 may be used to generate the first and second point sequences without undergoing coordinate conversion.

[0039] In addition, in this embodiment, the camera 43 photographs the object within the field of view at a predetermined time interval, but the timing of photographing may be arbitrary as long as the interval allows detection of the movement of the object.

[0040] In the present embodiment, the program generation device is configured as an integral part of the control device 10 that controls the robot 20, etc. Alternatively, the program generation device may be configured as a separate entity from the control device 10. For example, the program generation device may be configured to be connected to the control device 10 only when an operation program is generated.

[0041] Furthermore, in this embodiment, the sensor 40 that detects the movement of the tool 30 operated by the worker P and the sensor 40 that detects the movement of the tool 30 operated by the robot 20 may be the same or different. When different sensors 40 are used, it is not necessary to move the installation position of the sensor 40 each time the movement of the tool 30 operated by the worker P is detected and the movement of the tool 30 operated by the robot 20 is detected.

[0042] Furthermore, in this embodiment, the sensor 40 is exemplified as a motion capture sensor that captures the movement of an object by capturing images of the markers 41 and 42 attached to the object, but the sensor 40 is not limited to this. For example, the sensor 40 may be a motion sensor that includes multiple sensors that can capture position information themselves, such as a gyro sensor and an acceleration sensor. In this case, it is sufficient to simply attach the sensor 40 to the object, and there is no need to capture images of the markers attached to the object. Therefore, even if there are objects around the object that may obstruct the capture of the image, the position data of the object can be accurately detected.

[0043] In the present embodiment, the sensor 40 is described as having a processor and calculating pair information from the acquired images. Alternatively, the sensor 40 may acquire images, and the processor 12 of the program generation device 10 may calculate pair information from the images acquired by the sensor 40.

[0044] In this embodiment, the movement of the tool 30 when the worker P operates the tool 30 is detected as a target movement by the sensor 40. Alternatively, the pair information I12, I22 or the first point sequence in the target movement may be generated offline on a computer.

[0045] Although the embodiments of the present disclosure have been described in detail above, the present disclosure is not limited to the individual embodiments described above. Various additions, substitutions, modifications, partial deletions, etc. are possible to these embodiments without departing from the gist of the invention or the concept and spirit of the present invention derived from the content of the claims and their equivalents. For example, in the above-described embodiments, the order of each operation and the order of each process are shown as examples and are not limited to these.

[0046] The following supplementary notes are further disclosed regarding the above-described embodiments and variations. (Supplementary Note 1) A program generation device including at least one memory and at least one processor, wherein the memory stores a first sequence of points indicating paired information consisting of a pair of a position of at least one point on an object and a time when the object is made to perform a target motion, and the processor generates an operation program for causing a robot holding the object to reproduce the target motion based on the first sequence of points stored in the memory, operates the robot in accordance with the generated operation program, generates a second sequence of points indicating the paired information at that time, and modifies a parameter related to the robot's motion in the operation program based on a comparison between the first sequence of points and the second sequence of points. (Supplementary Note 2) The program generation device according to Supplementary Note 1, wherein the target motion is the motion of the object when the object is moved by an operator. (Supplementary Note 3) The program generation device according to Supplementary Note 1 or Supplementary Note 2, wherein the parameter is the motion speed of the robot. (Supplementary Note 4) The program generation device according to any one of Supplementary Note 1 to Supplementary Note 3, wherein the processor modifies the parameters by multiplying them by a coefficient calculated based on the difference in time between corresponding points in the first sequence of points and the second sequence of points. (Supplementary Note 5) The program generation device according to Supplementary Note 4, wherein the processor repeatedly modifies the parameters until the difference becomes smaller than a preset threshold. (Supplementary Note 6) The program generation device according to Supplementary Note 1, wherein the processor compares the first sequence of points and the second sequence of points using a DTW (Dynamic Time Warping) method. (Supplementary Note 7) A robot system comprising the program generation device according to any one of Supplementary Note 1 to Supplementary Note 6, a sensor that detects the pair information, the robot, and the target object.(Supplementary Note 8) A program generation method comprising: storing a first sequence of points indicating paired information consisting of a pair of position and time of at least one point on an object when the object is made to perform a target motion; generating an operation program based on the stored first sequence of points to cause a robot holding the object to reproduce the target motion; operating the robot in accordance with the generated operation program; generating a second sequence of points indicating the paired information at that time; and modifying parameters related to the robot's operation in the operation program based on a comparison between the first sequence of points and the second sequence of points.

[0047] REFERENCE SIGNS LIST 1 Robot system 11 Memory 12 Processor 10 Control device (program generating device) 20 Robot 30 Tool (object) 40 Sensor I11, I12 Paired information I21, I22 Paired information P Worker

Claims

1. A program generation device comprising at least one memory and at least one processor, wherein the memory stores a first point sequence indicating pair information consisting of a pair of at least one point position and time on the object when the object is caused to perform a target operation, the processor generates an operation program for causing a robot holding the object to reproduce the target operation based on the first point sequence stored in the memory, operates the robot according to the generated operation program, and generates a second point sequence indicating the pair information at that time, and modifies a parameter related to the operation of the robot in the operation program based on a comparison between the first point sequence and the second point sequence.

2. The program generation device according to claim 1, wherein the target operation is the operation of the object when the object is moved by an operator.

3. The program generation device according to claim 1 or claim 2, wherein the parameter is the operation speed of the robot.

4. The program generation device according to any one of claims 1 to 3, wherein the processor multiplies and corrects the parameter by a coefficient calculated based on the time difference at corresponding points between the first point sequence and the second point sequence.

5. The program generation device according to claim 4, wherein the processor repeats the correction of the parameter until the difference becomes smaller than a preset threshold value.

6. The program generation device according to claim 1, wherein the processor compares the first point sequence and the second point sequence by the DTW (Dynamic Time Warping) method.

7. A robot system comprising the program generation device according to any one of claims 1 to 6, a sensor for detecting the pair information, the robot, and the object.

8. A program generation method that stores a first point sequence indicating pair information consisting of a pair of the position and time of at least one point on the object when the object is made to perform a target operation, generates an operation program for causing a robot holding the object to reproduce the target operation based on the stored first point sequence, operates the robot according to the generated operation program, generates a second point sequence indicating the pair information at that time, and corrects a parameter related to the operation of the robot in the operation program based on a comparison between the first point sequence and the second point sequence.

Images