Robot systems and robot control devices

Abstract

The robot system (100, 500) of the present invention includes: a robot (10, 520); and a robot control device (50, 530), which controls the robot (10, 520) to perform a specific task based on the detection result of the sensor on the detection object; and the robot control device (50, 530) includes an anomaly detection processing unit (153, 636), which controls the robot (10, 520) to return and stop at the position at the time when the cause of the anomaly was detected in response to the detection of an anomaly performed by the sensor.

Description

[Technical Field]

[0001] This invention relates to a robot system. [Previous Technology]

[0002] Previously, it was known that a robot system comprising the following components: a conveying device for conveying items; a robot for performing specific operations on the conveyed items; a robot control device; and a camera device for detecting items, etc. (for example, see Patent Document 1).

[0003] Patent Document 2, concerning a control device for a work robot performing specific tasks on a production line, describes a control device for a work robot having the following function: when a monitoring device that monitors the working status of the work device performing specific tasks on a workpiece identifies an abnormality in the working status, it temporarily returns the work device to the position where the abnormality was identified and restarts the operation from that position (page 2, bottom right column, columns 14-20). [Prior Art Documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-188508 [Patent Document 2] Japanese Patent Application Publication No. Sho 62-278610 [Summary of the Invention]

[0005] [The problem the invention aims to solve]

[0006] Considering the teaching and adjustment scenarios of robot systems as described above, namely: a conveying device, a robot performing specific tasks on the conveyed items, a robot control device, and a camera device. Generally, in such robot systems, the process of photographing items or the conveying device and detecting their position using a camera device is repeated at high speed, and the robot follows the items while performing the task. During this process, it is difficult for the operator to confirm whether the processing at each location is proceeding without problems. Therefore, teaching and adjusting when problems occur is time-consuming and laborious, and the system startup time is long. [Technical means to solve the problem]

[0007] One embodiment of the robot system disclosed herein includes: a robot; and a robot control device that controls the robot to perform a specific task based on the detection results of a detection object by a sensor; and the robot control device includes an anomaly detection processing unit that controls the robot to return and stop at a position corresponding to the detection of an anomaly in the detection action performed by the sensor. [Effects of the Invention]

[0008] Based on the above configuration, since the robot returns and stops at the location where the cause of the anomaly is detected, the operator can teach and adjust the location of the problem without having to perform heavy tasks such as moving the robot to the location of the problem. Therefore, the time and labor required for teaching and adjusting the robot system when problems occur can be greatly reduced, and the time required for robot system startup can be shortened.

[0009] The objects, features and advantages of the present invention, as well as other objects, features and advantages, are more apparent from the detailed description of the typical embodiments of the present invention shown in the accompanying drawings.

Implementation Method

[0011] Next, embodiments of the present disclosure will be described with reference to the drawings. In the referenced drawings, the same reference numerals are given to the same constituent parts or functional parts. For ease of understanding, the scale of these drawings may be changed. Furthermore, the forms shown in the drawings are examples for implementing the present invention, and the present invention is not limited to the forms shown in the drawings.

[0012] Figure 1 is a diagram showing the configuration of a robot system 100 according to an embodiment. As shown in Figure 1, the robot system 100 includes: a robot 10, which has a hand 30 mounted at the forearm of its arm; a robot control device 50, which controls the robot 10; a conveying device (conveyor 25 and conveyor control device 20), which conveys items; three camera devices 71-73; and a host control device 60. The robot control device 50 and the conveyor control device 20 are connected to the host control device 60, and the host control device 60 comprehensively controls the robot control device 50 and the conveyor control device 20 in a manner that allows specific tasks to be performed in the robot system 100. In addition, as shown in Figure 1, a teaching pendant 40 can be connected to the robot control device 50.

[0013] The host control device 60 is, for example, a PLC (programmable logic controller), which comprehensively controls the action sequence of the robot control device 50 and the conveyor control device 20 according to the control program used to control the action sequence.

[0014] The camera devices 71-73 are connected to the robot control device 50 and operate under the control of the robot control device 50. In this embodiment, the robot control device 50 has the functions of controlling each camera device and performing image processing on the image data captured by each camera device. The camera devices 71-73 may be cameras that capture images of varying shades or colors, or stereo cameras or three-dimensional sensors that can acquire distance images or three-dimensional point groups. In this embodiment, it is assumed that each camera device is a camera that acquires two-dimensional images.

[0015] The conveyor control device 20 controls the conveyor 25 to transport workpieces by driving the motor 26 of the conveyor 25.

[0016] The teaching device 40 is used, for example, to adjust the teaching content and program. The teaching device 40 may be, for example, a teaching operation panel, a tablet terminal, a smartphone, or other information processing device.

[0017] In the robot system 100, the robot control device 50 controls the robot 10 to return and stop at the position at the time the abnormality was detected, in response to the detection of an anomaly by the camera devices 71-73. This allows the operator to adjust the position where the problem occurred without moving the robot.

[0018] This embodiment can be applied to robot systems that can use vision sensors to perform various processes (detection, judgment, etc.). However, here, as an example, the configuration of robot 10 is described as performing the operation of holding workpiece 91 against support 2 and installing it onto workpiece 81 being transported on conveyor 25. As an example, assume that workpiece 91 is a vehicle door and workpiece 81 is the body of the vehicle on which the door is installed. Robot 10 repeatedly performs the operation of installing workpiece 91 onto workpiece 81 being transported one by one on conveyor 25 in the direction of the arrow in Figure 1. During this process, the camera devices 71-73 repeatedly perform the actions of imaging and detecting the objects to be detected.

[0019] Figure 2 is a diagram showing an example of the hardware configuration of the host control device 60 and the robot control device 50. The host control device 60 may be configured as a general computer, with a memory 62 (ROM, RAM, non-volatile memory, etc.) connected to the processor 61 via a bus, a display unit 63, an operation unit 64 consisting of an input device such as a keyboard (or soft keyboard), and an input / output interface 65. The robot control device 50 may be configured as a general computer, with a memory 52 (ROM, RAM, non-volatile memory, etc.) connected to the processor 51 via a bus, an input / output interface 53, and an operation unit 54 including various operation switches. Furthermore, the teaching pendant 40 may also be configured as a general computer, having a processor, memory, a display unit, an operation unit, and various input / output interfaces.

[0020] Figure 3 is a functional block diagram of the robot control device 50 and the upper control device 60. As shown in Figure 3, the robot control device 50 includes: a motion control unit 151, which controls the motion of the robot 10 (and the hand 30) according to instructions or motion programs 155 from the teaching device 40; an image processing unit 152, which executes the control of the camera devices 71-73 and performs image processing on the camera images; and an anomaly detection processing unit 153, which controls the robot 10 to return to the position at the time of anomaly detection in response to an anomaly detected by any of the camera devices 71-73. An information recording unit 154 acquires and records the position of the robot 10 and other information at the time of anomaly detection. The anomaly detection processing unit 153 also has the function of sending a signal indicating the intention to the upper control device 60 in response to an anomaly detected in response to an anomaly detected by any of the camera devices 71-73.

[0021] The upper-level control device 60 includes: a control unit 161, which controls the action sequence of the robot control device 50 and the conveyor control device 20 according to the control program 164; and an anomaly handling unit 162, which controls the conveyor 25 to return to the position at the time of an anomaly detection in response to a signal received from the robot control device 50 indicating that an anomaly has been detected. An information recording unit 163 acquires and records the position of the conveyor 25 and other information at the time of an anomaly detection.

[0022] Here, an example of object detection performed by camera devices 71-73 between specific tasks performed by robot 10 will be described. Camera device 71 is a fixed camera that captures only one image of the workpiece 81 held against support 2 by robot 10, and detects the holding deviation.

[0023] Camera devices 72 and 73 are fixed to the front end of the arm of robot 10. Camera devices 72 and 73 are set to photograph the vicinity of the installation position when robot 10 installs workpiece 91 onto workpiece 81. Camera devices 72 and 73 repeatedly photograph the workpiece 91 while robot 10 is moving it and installing it onto workpiece 81.

[0024] The positioning control of workpiece 91 on workpiece 81, performed as part of such an installation operation, using camera device 72 or 73, will be described. Here, the case of using the image captured by camera device 72 will be described. In the positioning control, during the stage when robot 10 is moved to a position where workpiece 91 can be installed on workpiece 81, the camera device 72 captures this state, and control is performed to adjust the position of robot 10.

[0025] Figure 4 shows a camera image taken by the camera device 72 in a normal position where the robot 10 can be mounted on the workpiece 81 (here, the position is defined as where the workpiece 91 can be moved vertically downward and the pin of the workpiece 91 can be inserted into the hole of the workpiece 81). Since this image is taken when the robot 10 is in the correct position, it is also referred to below as reference image 261.

[0026] The image processing unit 152 of the robot control device 50 has the function of detecting the feature quantity of a preset characteristic portion, i.e., a characteristic part, in workpieces 81 and 91. Furthermore, the image processing unit 152 has the function of calculating the difference between the feature quantity of workpiece 81 and the feature quantity of workpiece 91 as a relative quantity. Based on the calculated relative quantity, the image processing unit 152 generates instructions to cause the robot 10 to move.

[0027] In the robot system 100, the positioning of workpiece 91 relative to workpiece 81 transported by conveyor 25 is performed based on images captured by camera devices 72 and 73. Specifically, positioning is performed when the holes 82a and 83a (see Figure 6) of the protrusions 82 and 83 of workpiece 81 (body) are inserted into the pins (not shown) of workpiece 91 (door).

[0028] Before the robot 10 performs actual work, the image processing unit 152 calculates the relative position in the reference image 261 using the following steps (A1) to (A3). (A1) The image processing unit 152 detects the upper surface of the protrusion 82 as a first feature part for detecting the position of the workpiece 81, and detects the upper surface of the protrusion 92 of the workpiece 91 as a second feature part for detecting the position of the workpiece 91. In addition, a part of the workpiece, a pattern formed on the surface of the workpiece, a line or diagram recorded on the surface of the workpiece, etc., can be used as feature parts. As a method for detecting feature parts, a base image that serves as the reference for each workpiece 81 and 91 can be prepared in advance. The feature parts in the image captured by the camera device 72 can be detected by means of template matching or the base image and the image captured by the camera device 72.

[0029] (A2) Next, the image processing unit 152 detects a first feature quantity related to the position of the first feature portion and a second feature quantity related to the position of the second feature portion. A screen coordinate system 252 is set in the image captured by the imaging device 72. The screen coordinate system 252 is a coordinate system with any point in the image set as the origin. The screen coordinate system 252 has mutually orthogonal u-axis and v-axis, and corresponds to the visual sensor coordinate system of the imaging device 72. In this embodiment, the position-related feature quantity is the coordinate value of the u-axis and the coordinate value of the v-axis of the screen coordinate system 252 in the image. The image processing unit 152 can detect the positions of set points P1 and P2 set at the feature portions based on the feature portions detected in the reference image 261. The image processing unit 152 detects the coordinate values ​​(u1b, v1b) of the screen coordinate system 252 at the set point P1, and uses them as the first feature quantity. The feature quantity detection unit also detects the coordinate values ​​(u2b, v2b) of the screen coordinate system 252 at the set point P2, and uses them as the second feature quantity.

[0030] (A3) Next, the image processing unit 152 calculates the relative quantity between the first feature quantity and the second feature quantity in the reference image. The image processing unit 152 calculates the relative position quantity as a relative quantity in order to control the position of the robot 10. The relative position quantity is the difference between the first feature quantity and the second feature quantity. For example, the image processing unit 152 calculates the difference between the coordinate values ​​of the first feature quantity and the coordinate values ​​of the second feature quantity (u1b-u2b, v1b-v2b) as the relative position quantity. The calculated relative position quantity in the reference image 261 is memorized as a reference relative position quantity.

[0031] Thus, the image processing unit 152 can calculate the relative position quantity in the reference image 261. Furthermore, in this embodiment, it is assumed that the relative position quantity in the reference image 261 is pre-calculated and stored in the memory unit.

[0032] A flowchart of the positioning control is shown in Figure 5. This process is executed under the control of the processor 51 of the robot control device 50. After the robot control device 50 controls the approach to a position where the workpiece 91 can be installed on the workpiece 81, in step S101, the motion control unit 151 uses the camera device 72 to photograph the workpieces 81 and 91.

[0033] Figure 6 shows image 262 captured by camera device 72 in step S101. Image 262 includes an image of the upper surface of the first feature portion, namely the protrusion 82, and an image of the upper surface of the second feature portion, namely the protrusion 92. In image 262, workpiece 91 is offset relative to workpiece 81 towards the positive side of the u-axis of screen coordinate system 252 as indicated by arrow 201.

[0034] In step S102, the image processing unit 152 detects the first feature region and the second feature region of the image 262 captured by the camera device 72. Here, the upper surface of the protrusion 82 of the workpiece 81 is detected as the first feature region, and the upper surface of the protrusion 92 of the workpiece 91 is detected as the second feature region.

[0035] Next, in step S103, the image processing unit 152 detects the first feature quantity and the second feature quantity in the image captured by the camera device 72. The image processing unit 152 detects the coordinate value (u1m, v1m) of the screen coordinate system 252 of the set point P1 as the first feature quantity related to the first feature portion, and calculates the coordinate value (u2m, v2m) of the screen coordinate system of the set point P2 as the second feature quantity related to the second feature portion.

[0036] Next, in step S104, the image processing unit 152 calculates the difference between the first feature quantity and the second feature quantity as the relative position quantity. The relative position quantity in the image 262 captured by the camera device 72 is the difference between the coordinate values ​​of the first feature quantity and the coordinate values ​​of the second feature quantity (u1m-u2m, v1m-v2m).

[0037] Next, in step S105, the image processing unit 152 calculates the difference between the relative position quantity in image 262 and the relative position quantity in reference image 261, i.e., the difference in relative position quantity. The difference in relative position quantity can be represented by [(u1m-u2m)-(u1b-u2b),(v1m-v2m)-(v1b-v2b)], and is used as a value related to each u-axis and v-axis.

[0038] Next, in step S106, it is determined whether the difference in relative position is within a preset range. If the difference in relative position is within a preset range (S120: Yes), it is considered that the positioning of workpiece 91 relative to workpiece 81 is completed, and this process ends.

[0039] On the other hand, if the difference in relative position is outside the preset range (S120: No), it can be determined that workpiece 91 has not reached the desired position relative to workpiece 81. In this case, the process proceeds to step S107.

[0040] In step S107, the image processing unit 152 sets the driving method of the robot 10 based on the difference in relative position quantities. The image processing unit 152 sets the movement direction and movement amount of the robot in the reference coordinate system. In this embodiment, based on the relative positional relationship between the screen coordinate system 252 and the reference coordinate system of the robot 10, the movement direction of the robot 10 relative to the difference in relative position quantities is preset. For example, the movement direction of the robot's position in the reference coordinate system is determined in the following way, depending on whether the difference in relative position quantities related to the u-axis of the screen coordinate system is positive or negative. When the difference in relative position quantities related to the u-axis is positive, the movement direction (1,1,0) is determined using the coordinate values ​​of the X-axis, Y-axis, and Z-axis of the reference coordinate system. Also, when the difference in relative position quantities related to the y-axis is positive, the movement direction (0,0,1) is determined using the coordinate values ​​of the X-axis, Y-axis, and Z-axis of the reference coordinate system.

[0041] Furthermore, the method for calculating the robot's position movement based on the difference in relative position quantities is determined as follows: The robot's position movement in the direction corresponding to the u-axis can be obtained by multiplying the value related to the u-axis ((u1m-u2m)-(u1b-u2b)) by a preset coefficient. Similarly, the robot's position movement in the direction corresponding to the v-axis can be obtained by multiplying the value related to the v-axis ((v1m-v2m)-(v1b-v2b)) by a preset coefficient. Thus, the position movement of the robot 10 can be calculated in the directions corresponding to each axis of the screen coordinate system 252.

[0042] Next, in step S108, the robot 10 is driven based on the movement direction and amount of the robot's position calculated as described above. The image processing unit 152 generates a movement command to drive the robot 10 based on the movement direction and amount of the robot's position. The image processing unit 152 sends the movement command to the motion control unit 151. The motion control unit 151 controls the position of the robot 10 based on the movement command. Then, the processing from step S115 onwards is repeated. Through such control, the position of the workpiece 91 can be gradually brought closer to the desired position.

[0043] The image processing unit 152 has the function of detecting abnormalities in the detection operation performed by any of the imaging devices 71-73. As an example of abnormality detection, the following situations may occur: (B1) The state in which no detection object (e.g., a feature part) is detected in the image is continuous. (B2) When continuously capturing images of a moving detection object, the position of the detected detection object is discontinuous (located in an unexpected position).

[0044] Figure 7 is a flowchart showing the anomaly detection process performed when an anomaly is detected by the camera device in the robot system 100. The processes shown in steps S1 to S6 of the anomaly detection process are executed under the control of the processor 51 of the robot control device 50, and the processes shown in steps S7 to S10 are executed under the control of the processor 61 of the host control device 60.

[0045] Suppose that due to the occurrence of the situation (B1) or (B2) illustrated above, any of the camera devices 71-73 detects an abnormality in the detection (step S1).

[0046] In step S2, the robot control device 50 (abnormal detection processing unit 153) sends a signal to the upper control device 60 indicating that an abnormality has been detected.

[0047] Next, during anomaly detection, the processing unit 153 (information recording unit 154) obtains the current position (position A) of the robot 10 (step S3). Furthermore, the position of the robot 10 also includes its posture. Then, the robot control device 50 (processing unit 153 during anomaly detection) decelerates and stops the robot 10 to ensure its safe stopping (step S4).

[0048] Next, the robot control device 50 (abnormal detection processing unit 153) determines whether the robot 10 has stopped (step S5). The robot control device 50 (abnormal detection processing unit) waits for the robot to stop (S5: No).

[0049] When the robot stops (S5: Yes), the robot control device 50 (abnormal detection processing unit 153) moves the robot 10 and stops it at position A at the time when the abnormality is detected.

[0050] When the robot control device 50 receives a signal indicating an anomaly detection, the upper control device 60 (information recording unit 163) obtains the current position (position B) information of the conveyor (step S7). Next, the upper control device 60 (anomaly handling unit 162) considers safety and causes the conveyor 25 to decelerate and stop (step S8).

[0051] Next, the upper control device 60 (abnormality processing unit 162) determines whether the conveyor 25 has stopped (step S9). The upper control device 60 (abnormality processing unit 162) waits until the conveyor 25 stops (D9: No). When the conveyor 25 stops (S9: Yes), the upper control device 60 (abnormality processing unit 162) moves the conveyor 25 and stops it at the position where the abnormality occurred, i.e., position B (step S10). Based on the above processing, when an abnormality is detected in the detection action of the camera device, the robot 10 and the conveyor 25 can be returned to the position at the time when the abnormality was detected.

[0052] In addition to returning robot 10 and conveyor 25 to the position where the anomaly was detected, one or more of the following actions (C1) to (C3) may be performed in robot system 100: (C1) When each machine constituting the robot system returns to the state where the anomaly was detected, video is captured by each camera device, and the video image is displayed. (C2) Information related to the content of the anomaly is displayed. (C3) Information related to the operating conditions of the robot system at the time the anomaly was detected is displayed.

[0053] The above-mentioned action (C1) can be achieved by the abnormality detection processing unit 153 activating each camera device 71-73 via the image processing unit 152 after the robot 10 and the conveyor 25 return to the point where the abnormality was detected, and displaying the camera image on the display unit 41 of the teaching device 40. Since the abnormal state can be reproduced in such a camera image, the operator can observe such a camera image while ensuring that the adjustment of the position of the robot 10, the adjustment of the timing of the robot 10's actions, or the adjustment of the setting position of the camera device in the action program is carried out reliably.

[0054] The above-mentioned action (C2) can be achieved, for example, by the cooperation of the information recording unit 154 of the robot control device 50 with the image processing unit 152 or the motion control unit 151, thereby obtaining information related to the content of the anomaly when an anomaly is detected. The anomaly detection processing unit 153 displays the information related to the content of the anomaly obtained by the information recording unit 154 on the teaching pendant 40. Figure 8 shows an example of a situation where the content of the anomaly detected by the camera device is displayed on the display screen (display unit 41) of the teaching pendant 40 by the anomaly detection processing unit 153. In this example, an error display screen 145 is displayed, indicating that no object was detected in the camera device (camera #2 in this example), as the cause of the anomaly. In this case, the operator can use the information related to the content of the anomaly displayed here as a clue to ensure that the adjustment of the position of the robot 10, the adjustment of the action sequence of the robot 10, and the adjustment of the setting position of the camera device in the motion program are carried out in a reliable manner.

[0055] That is, by adopting information related to the operator's prompts and the content of the anomaly, the adjustment for responding to the anomaly can proceed efficiently.

[0056] As an example of the aforementioned action (C3), the information recording unit 154 of the robot control device 50 may be configured to record information related to the action conditions (e.g., robot position, posture, movement speed, etc.) of the robot 10 at the time when an anomaly is detected. Furthermore, the information recording unit 163 of the upper control device 60 may be configured to record information related to the action conditions (position, speed, etc.) of the conveyor 25 at the time when an anomaly is detected. Information related to the action conditions of the robot 10 and the conveyor 25 may also be displayed, for example, on the display unit 41 of the teaching pendant 40. Such information related to action conditions can also improve the efficiency of operator adjustments.

[0057] In addition, the prompts for the above actions (C1) to (C3) can also be made on a device (e.g., a host control device 60) that has a display unit in the robot system 100, in addition to the examples mentioned above.

[0058] As explained above, according to this embodiment, since the robot returns and stops at the location where the cause of the anomaly is detected, the operator can teach and adjust the location of the problem without the need for laborious tasks such as moving the robot to the problem site. Therefore, the time and labor required for teaching and adjusting the robot system when problems occur can be greatly reduced, and the time required for robot system startup can be shortened.

[0059] The present invention has been described above using typical embodiments, but it is obviously understood that anyone skilled in the art can make changes and various other changes, omissions, and additions to the above embodiments without departing from the scope of the present invention.

[0060] The configuration of the robot system 100 shown in Figure 1 is illustrative and should be understood as not all of these components being essential. For example, the robot system 100 may also have a system configuration that omits the conveyor 25 and the conveyor control device 20.

[0061] The above-described embodiment relates to a robot system that detects objects and performs specific tasks by means of a camera device as a vision sensor. However, the present invention can be applied to robot systems that have other types of sensors for detecting objects (proximity sensors, distance sensors, etc.). Furthermore, in the case where such other types of sensors are used in the robot system, the output result of the sensor is displayed in the above-described action (C1).

[0062] The above-described embodiment, which reduces the startup time of a system that returns the robot or each movable body together with the robot to the position where the problem occurred and makes various adjustments, can be applied to robot systems of various types.

[0063] Here, referring to Figures 9 and 10, another embodiment will be described. The embodiment shown here relates to a robot system configured to correct the robot's position relative to the workspace by detecting markers placed in the workspace using a vision sensor mounted on the robot. Figure 9 is a diagram showing the machine configuration of the robot system 500. The robot system 500 includes: a machine tool 510, a robot 520, a robot control device 530 for controlling the robot 520, and a conveying device 581 for conveying the robot 520 and the robot control device 530. The robot 520 is mounted on the conveying device 581 and positioned at a specific location in front of the machine tool 510, and performs specific operations such as loading / unloading workpieces into the machine tool 510. That is, the robot system 500 is configured as an automated system that automates the loading / unloading of workpieces into the machine tool 510 by means of the robot 520. The conveying device 581 is, for example, a trolley or AGV (Automated Guided Vehicle).

[0064] The teaching pendant 550 is wirelessly or wiredly connected to the robot control device 530 and is used to teach the robot 520. In addition, in the actual use of the robot system 500, since the control program made using the teaching pendant 550 is registered in the robot control device 530, the teaching pendant 550 can be omitted from the robot system 500.

[0065] In the robot system 500 shown in Figure 9, when the robot 520 performs operations such as loading / unloading workpieces, the position of the conveying device 581 equipped with the robot 520 changes. Therefore, the robot 520 must be configured to measure the positional offset of the robot 520 relative to the machine tool 510 and to perform operations correctly on the machine tool 510. Therefore, a vision sensor 571 is mounted on the front end 521 of the robot 520 arm, and the robot 520 (robot control device 530) is configured to use the vision sensor 571 to detect the positional offset of the robot 520 relative to the workspace (machine tool 510), correct the positional offset, and perform the operation.

[0066] The teaching pendant 550 provides the function of creating a program (hereinafter also referred to as a measurement program). This program uses a vision sensor 571 mounted on the forearm 521 of the robot 520 to measure the three-dimensional position of a marker 504 set at a specific location in the workspace (machine tool 510), and measures the positional deviation from the desired position in the workspace of the robot 520. The control program, which includes the measurement program created using the teaching pendant 550, is registered in the robot control device 530. Subsequently, the robot 520 (robot control device 530) can detect the positional deviation from the desired position in the workspace of the robot 520, make position corrections, and perform specific operations.

[0067] The vision sensor 571 may be a two-dimensional camera or a three-dimensional position detector. In this embodiment, it is assumed that the vision sensor 571 is a two-dimensional camera. The vision sensor 571 is connected to the robot control device 530. It is assumed that the robot control device 530 has functions for controlling the vision sensor 571 and performing various image processing functions on the images captured by the vision sensor 571. Furthermore, it is assumed that calibration data containing data representing the position of the vision sensor 571 relative to the robot 520 is pre-memorized in the memory of the robot control device 530.

[0068] Figure 10 is a functional block diagram of the robot control device 530. As shown in Figure 10, the robot control device 530 includes: a memory unit 631 for storing control programs and other various information; a motion control unit 632 for controlling the actions of the robot 520 according to the control program; a marker position measurement unit 633; ​​a relative position calculation unit 634; a measurement accuracy evaluation unit 635; and an anomaly detection processing unit 636.

[0069] The marker position measurement unit 133 uses a vision sensor 571 to measure the three-dimensional position of the marker 504. In this embodiment, as an example, the marker position measurement unit 633 uses a vision sensor 571, which is a two-dimensional camera, to measure the position of the marker 504 using a stereoscopic measurement method. That is, the marker position measurement unit 633 changes the position of the vision sensor 571, which includes a two-dimensional camera, to photograph the same marker 504 from two different positions, and calculates the three-dimensional position of the marker 504. This method has the advantage of enabling a position measurement system to be implemented at low cost by using a less expensive two-dimensional camera. In addition, other methods known in the art for measuring the position of markers (also known as target markings or visual markers) can be used.

[0070] The memory unit 631 stores calibration data indicating the position of the 2D camera (vision sensor 571) based on the coordinate system (mechanical coordinate system) set on the forearm 521 of the robot 520. On the other hand, the robot control device 530 (marker position measuring unit 633) can grasp the position and posture of the forearm 521 of the robot 520 during its movements. Therefore, the robot control device 530 (marker position measuring unit 633) converts the mechanical coordinate system into the robot coordinate system in response to the robot 520's movements, thus establishing a correspondence between the sensor coordinate system of the 2D camera (vision sensor 571) during image capture and the robot coordinate system. In this way, the position of the object (marker 504) can be obtained in the marker position measuring unit 633 as its three-dimensional position in the robot coordinate system.

[0071] The relative position calculation unit 634 calculates the relative position between the workspace (machine tool 510) and the robot 520 based on the measured mark position (in other words, the offset of the robot 520 from the desired position in the workspace).

[0072] The motion control unit 632 controls the robot 520 to perform the operation in a manner that the robot 520 performs the operation in a correct position and posture after the correction of the specified position and posture, based on the calculated relative position relationship between the work space and the robot (the offset of the robot 520 from the desired position in the work space).

[0073] The measurement accuracy evaluation unit 635 has the function of evaluating the accuracy of the measurement result after the mark position measurement unit 633 measures the position of a mark 504.

[0074] The marker position measuring unit 633 may be configured to increase the number of measuring markers when the measuring accuracy evaluation unit 635 evaluates that the measuring accuracy is low. As an example, when using 3 markers for measurement, it is performed as follows: The 3 markers are positioned around the teaching point. The position of the first measured marker is set as the origin position, the position of the second measured marker is set as the position in the X-axis direction, and the position of the third measured marker is set as being located on the XY plane, thereby determining the position and orientation of the coordinate system (the coordinate system of the workspace).

[0075] After the robot control device 530 has grasped the relative positional relationship between the robot 520 and the workspace (machine tool 510) using the above method, it performs a correction action in a way that maintains the positional relationship between the robot 520 and the workspace as the desired positional relationship, and at the same time makes the robot 1 perform a specific operation (loading / unloading of workpieces into the machine tool 510, etc.).

[0076] When the anomaly detection processing unit 636 detects an anomaly caused by a correction action based on the detection result using the marker during the execution of a specific task by the robot 1, it controls the robot 520 to return and stop the vision sensor 571 at the marked camera position (i.e., the position for detecting the cause of the anomaly). Here, the anomaly caused by the correction action is, for example, a situation where the robot 520 cannot position the workpiece correctly (positioning error, etc.). Thus, when an anomaly caused by the correction action is detected, the robot 520 (vision sensor 571) automatically returns to the camera position (i.e., the position for taking the camera to generate the correction action), so the operator can quickly and easily make adjustments related to the marker detection function (parameter adjustment, etc.).

[0077] The robot system 100 described with reference to Figures 1 to 8 above and the robot system 500 described with reference to Figures 9 to 10 include: a robot; and a robot control device that controls the robot to perform a specific task based on the detection result of the sensor on the detection object; and the robot control device includes an anomaly detection time processing unit that controls the robot to return and stop at the position at the time when the detection action performed by the sensor is detected in response to an anomaly.

[0078] The function allocation shown in the function block diagrams of Figures 3 and 10 is an example, and there may be various examples of function allocation. For example, there may also be a configuration in which the vision sensor control device having the function of the image processing unit 152 as described above is configured as a separate device from the robot control device.

[0079] The functional blocks of the robot control device and the upper control device shown in Figures 3 and 10 can be implemented by the processor of the device executing various software stored in the memory device, or can be implemented by hardware such as ASIC (Application Specific Integrated Circuit).

[0080] The programs for various processing such as the positioning control in Figure 5 and the abnormal detection processing in Figure 7 of the above-described embodiments can be recorded on various computer-readable recording media (e.g., semiconductor memory such as ROM, EEPROM, flash memory, magnetic recording media, CD-ROM, DVD-ROM, etc.). [Simplified Explanation of the Diagram]

[0010] Figure 1 is a diagram showing the configuration of a robot system in one embodiment. Figure 2 is a diagram showing an example of the hardware configuration of the robot control device and the upper-level control device. Figure 3 is a functional block diagram of the robot control device and the upper-level control device. Figure 4 is an example of a reference image showing a detection performed by a camera device. Figure 5 is a flowchart of positioning control using a camera device. Figure 6 is an example of a camera image during positioning control. Figure 7 is a flowchart showing the handling of an anomaly. Figure 8 is a diagram showing an example of information related to the content of the anomaly. Figure 9 is a diagram showing the configuration of a robot system in another embodiment. Figure 10 is a functional block diagram of the robot control device of the robot system in Figure 9.

Claims

1. A robot system comprising: a robot; and a robot control device that controls the robot to perform a specific task based on the detection result of a sensor on a detected object; wherein the robot control device includes an anomaly detection processing unit that controls the robot to return and stop at a position at which the cause of the anomaly was detected when an anomaly is detected by the sensor; wherein the anomaly detection processing unit activates the sensor when the robot returns and stops at the position at which the anomaly was detected, and displays the output result of the sensor on a display unit.

2. The robot system of claim 1, wherein the robot control device detects the object to be detected by the aforementioned sensor and causes the robot to perform the aforementioned specific task; and the aforementioned abnormality detection processing unit controls the robot to return and stop at the position at the time when the abnormality is detected in response to the detection action performed by the aforementioned sensor.

3. The robot system of claim 1 further comprises: a conveying device for conveying items; and a host control device for controlling the conveying device and the robot control device; and the aforementioned abnormality detection processing unit further outputs a signal indicating that the aforementioned abnormality has been detected to the host control device in response to the detection of the aforementioned abnormality; the aforementioned host control device comprises an abnormality processing unit that controls the conveying device to return and stop at the position at the time when the aforementioned abnormality was detected in response to receiving the aforementioned signal.

4. The robot system of any one of claims 1 to 3, wherein the aforementioned sensor is a camera device; and the aforementioned output result is a camera image of the aforementioned camera device.

5. The robot system according to any one of claims 1 to 3, wherein when the aforementioned anomaly detection is performed, the processing unit detects the aforementioned anomaly and displays information related to the content of the aforementioned anomaly on the aforementioned display unit.

6. The robot system according to any one of claims 1 to 3, wherein the aforementioned anomaly detection processing unit displays information related to the robot's action conditions at the time the aforementioned anomaly was detected on the aforementioned display unit.

7. The robot system of any one of claims 1 to 3 further comprises a teaching device connected to the aforementioned robot control device and used to teach the aforementioned robot; the aforementioned display unit is disposed on the aforementioned teaching device.

8. The robot system of claim 1, wherein the aforementioned sensor is a vision sensor mounted on the robot, and the robot control device is configured to perform a correction action to adjust the position of the robot relative to the work space by detecting one or more markers disposed in the work space using the aforementioned vision sensor; the aforementioned abnormality is caused by the aforementioned correction action; and when the aforementioned abnormality is detected, the processing unit controls the robot in such a way that the aforementioned vision sensor returns to the position where the aforementioned marker was detected.

9. A robot control device comprising: a motion control unit that controls the robot to perform a specific task based on the detection result of a sensor on a detection object; and an anomaly detection processing unit that controls the robot to return and stop at a position at the time point when the detection of the cause of the anomaly is detected, in response to the detection of an anomaly in the detection action performed by the aforementioned sensor; and the anomaly detection processing unit actuates the aforementioned sensor when the robot returns and stops at the position at the time point when the anomaly is detected, and displays the output result of the aforementioned sensor on a display unit.

Images