Robotic monitoring system

Abstract

The present invention relates to a robot monitoring system, providing a system for determining whether a connecting rod has been displaced from a target braking position while the power supply of the robot is not turned on. The robot monitoring system is a robot monitoring system for a robot having a joint that connects the connecting rod in a rotatable manner, a robot control unit that controls the rotation of the connecting rod, and a braking mechanism for braking the rotation of the connecting rod, wherein the system comprises: a photographing unit that photographs the robot in a state where the braking mechanism is working; an estimating unit that estimates the angle of the joint using image data photographed by the photographing unit; and a determining unit that determines whether the connecting rod has been rotated beyond a predetermined angle range based on the estimated angle estimated by the estimating unit.

Description

Technical Field

[0001] The present disclosure relates to a robotic monitoring system. Background Art

[0002] In the past, in industrial robots, there is a known structure having a connecting rod connected by a joint, a driving device for rotating the connecting rod, and a braking mechanism for braking the rotation of the connecting rod (for example, Patent Document 1). During a period when the power of the robot is not turned on, the rotation of the connecting rod is braked by the braking mechanism, and the position of the connecting rod is maintained at a target braking position. There are various methods in the braking mechanism. For example, in the brake device described in Patent Document 1, the rotation of the connecting rod is braked by making a brake member that can mesh with a gear included in the driving device mesh with the gear.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2017-064802 Summary of the invention

[0006] Problems to be solved by the invention

[0007] Regardless of the various methods of the braking mechanism, the braking performance may sometimes decrease due to the deterioration of the components of the braking mechanism. In this case, when the braking mechanism is driven and the connecting rod is braked, the position of the connecting rod may be displaced from the target braking position. If the position of the connecting rod is displaced from the target braking position, there are the following problems. For example, during a period when the power of the robot is not turned on, etc., the operator sometimes performs maintenance on the robot. In this case, the operator performs maintenance work on the premise that the connecting rod is maintained at the target braking position. Therefore, if the position of the connecting rod is displaced from the target braking position, the operability may decrease. In addition, if the connecting rod is displaced during the absence of the operator, the displacement of the connecting rod will develop due to the response delay, and the connecting rod may interfere with the installation surface of the robot. Therefore, it is desired to have a system that determines whether the connecting rod has been displaced from the target braking position while the braking mechanism is working.

[0008] Means for solving problems

[0009] The present disclosure can be implemented as the following aspects.

[0010] (1) According to one embodiment of the present disclosure, a robot monitoring system is provided. The robot monitoring system is a robot monitoring system for a robot having a joint that connects a connecting rod in a rotatable manner, a robot control unit that controls the rotation of the connecting rod, and a braking mechanism for braking the rotation of the connecting rod, wherein the system comprises: a photographing unit that photographs the robot when the braking mechanism is in operation; an estimating unit that estimates the angle of the joint using image data photographed by the photographing unit; and a determining unit that determines whether the connecting rod has rotated beyond a predetermined angle range based on the estimated angle estimated by the estimating unit. According to this embodiment, by using the photographing unit to photograph the robot when the braking mechanism is in operation while the braking mechanism is in operation, it is possible to determine whether the connecting rod of the robot has displaced from a target braking position.

[0011] (2) In the robot monitoring system of the above-mentioned embodiment, a first output unit may be further provided, which outputs a reporting signal to a reporting device for causing the reporting device to report that the connecting rod has been displaced, when the determining unit determines that the connecting rod is not within the angular range. According to this embodiment, when the connecting rod is displaced, the reporting device can report the displacement. Thus, the operator can know the displacement of the connecting rod.

[0012] (3) In the robot monitoring system of the above-mentioned method, it is possible to further include a second output unit, which outputs a displacement signal for displacing the position of the connecting rod to the target braking position to the robot control unit when the determination unit determines that the position is not within the angle range. According to this method, when the connecting rod is displaced, the connecting rod can be displaced to the target braking position. Thus, even when the operator is not near the robot, the connecting rod position of the robot can be reset to the target braking position without human hands.

[0013] (4) In the robot monitoring system of the above-mentioned method, there may be a storage unit for storing a skeleton model composed of the joints and the links, and the estimating unit estimates the angle based on the image data and uses the estimated positions of the joints and the skeleton model. According to this method, the angle of the joint can be estimated with high accuracy using the skeleton model.

[0014] (5) In the robot monitoring system of the above-mentioned embodiment, a setting unit may be further provided, which sets the estimated angle estimated by the estimating unit using the image data at the start time point when the braking mechanism starts to operate and the braking of the rotation of the connecting rod as a reference angle, and sets a predetermined range centered on the reference angle as the angle range. According to this embodiment, the displacement relative to the position of the connecting rod at the start time point can be monitored.

[0015] (6) In the robot monitoring system of the above aspect, the robot may include a plurality of the joints, and the determination unit may determine whether the estimated angle is within the angle range for each of the plurality of joints. According to this aspect, when the robot includes a plurality of joints, the displacement of at least one joint can be determined.

[0016] The present disclosure can also be implemented in various forms other than the robot monitoring system, for example, in the form of a robot monitoring method, a control method of a robot monitoring system, a computer program that implements the control method, a non-transitory recording medium that records the computer program, and the like. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a schematic diagram showing the structure of a monitoring system.

[0018] Figure 2 is a block diagram showing the structure of an arm.

[0019] Figure 3 This is a diagram illustrating a skeleton model.

[0020] Figure 4 This is a flowchart of the monitoring process.

[0021] Figure 5 It is a diagram for explaining the angle range used for determination.

[0022] Figure 6 It is a schematic diagram showing the configuration of a monitoring system according to a second embodiment.

[0023] Figure 7 This is a flowchart of the monitoring process according to the second embodiment. DETAILED DESCRIPTION

[0024] A. First embodiment:

[0025] A1. Structure of monitoring system:

[0026] Figure 1 2 is a schematic diagram showing the structure of the robot monitoring system 100 that monitors the robot 200 . Figure 2 2 is a block diagram showing the structure of the arm 210. The robot 200 is a six-axis vertical multi-joint industrial robot. The robot 200 performs operations such as welding and painting on a workpiece.

[0027] The robot 200 includes an arm 210 and a robot control unit 220. The arm 210 includes a base 250, first to fifth links L1 to L5, and an end effector 256. The base 250 is fixed to the installation surface of the robot 200. The first link L1, the second link L2, the third link L3, the fourth link L4, the fifth link L5, and the end effector 256 are connected in series in sequence.

[0028] The base 250 and the first link L1 are connected by the first joint J1 in a rotatable manner. The first link L1 rotates in a manner that changes the angle between the base 250 and the first link L1, with the first axis CA1 of the first joint J1 as a fulcrum. The first link L1 and the second link L2 are connected by the second joint J2 in a rotatable manner. The second link L2 rotates in a manner that changes the angle between the first link L1 and the second link L2, with the second axis CA2 of the second joint J2 as a fulcrum. It should be noted that the angle between the first link L1 and the second link L2 is also referred to as the angle of the second joint J2. The angle of the second joint J2 is the angle between the straight line connecting the joint J at both ends of the first link L1, i.e., the first joint J1 and the second joint J2, and the straight line connecting the joint J at both ends of the second link L2, i.e., the second joint J2 and the third joint J3. The angles of the third joint J3 to the fifth joint J5 described below are also defined in the same way. The second link L2 and the third link L3 are connected by the third joint J3 in a rotatable manner. The third link L3 rotates in a manner that changes the angle formed by the second link L2 and the third link L3, i.e., the angle of the third joint J3, with the third axis CA3 of the third joint J3 as a fulcrum. The third link L3 and the fourth link L4 are connected by the fourth joint J4 in a rotatable manner. The fourth joint J4 has a fourth axis CA4 parallel to the direction in which the fourth link L4 extends. The fourth link L4 rotates with the fourth axis CA4 as the center axis. The fourth link L4 and the fifth link L5 are connected by the fifth joint J5 in a rotatable manner. The fifth link L5 rotates in a manner that changes the angle formed by the fourth link L4 and the fifth link L5, i.e., the angle of the fifth joint J5, with the fifth axis CA5 of the fifth joint J5 as a fulcrum. The fifth link L5 and the end effector 256 are connected by the sixth joint J6 in a rotatable manner. The sixth joint J6 has a sixth axis CA6 parallel to the direction in which the fifth link L5 extends. The end effector 256 rotates around the sixth axis CA6 of the sixth joint J6. The end effector 256 is, for example, a welding gun or a painting gun. The first to fifth links L1 to L5 are collectively referred to as links L. The first to fifth joints J1 to J5 are collectively referred to as joints J.

[0029] The robot control unit 220 is configured as a computer having a CPU (Central Processing Unit) and a memory, and is connected to the arm 210 in a communicable manner. The robot control unit 220 controls the arm 210. Specifically, the robot control unit 220 controls the on / off of the power supply of the arm 210, controls the rotation of the link L, and controls the brake mechanism 240 described later.

[0030] like Figure 2 As shown, each joint J has a motor 230 and a brake mechanism 240. Each link L rotates by the driving force of the connected motor 230. The motor 230 is an electromagnetic motor having a stator 231 and a rotor 232.

[0031] The brake mechanism 240 is a brake mechanism of a non-excitation type. The brake mechanism 240 includes a friction plate 241, a spring 242, and an electromagnet 243. The friction plate 241 is arranged opposite to a fixed plate (not shown) fixed to the motor 230, sandwiching the rotor 232 in the axial direction of the rotor 232. The spring 242 is installed in the axial direction of the rotor 232 in such a manner as to apply force to the friction plate 241 toward the fixed plate. When the robot control unit 220 does not command braking, the electromagnet 243 is set to an energized state, and the friction plate 241 is electromagnetically attracted by the electromagnet 243 against the force of the spring 242. Thus, the rotor 232 can rotate freely. In contrast, when the robot control unit 220 commands braking, the electromagnet 243 is set to a non-energized state, and the electromagnetic attraction of the electromagnet 243 to the friction plate 241 is released. Thus, the friction plate 241 is pressed against the rotor 232 by the force of the spring 242. Thus, the rotation of the rotor 232 is restricted, and the rotor 232 is held in the braking position.

[0032] like Figure 1 As shown, the robot monitoring system 100 includes a camera 10 as a photographing unit and an information processing device 20. The camera 10 has an image sensor and is arranged in a manner that can photograph the entire arm 210. The camera 10 photographs the robot 200 in a state where the brake mechanism 240 is in operation. Here, the state in which the brake mechanism 240 is in operation refers to a state in which the brake mechanism 240 is braking the rotation of the rotor 232. The information processing device 20 includes a CPU 30 and a storage unit 40. The storage unit 40 is implemented by a memory such as a RAM (Random Access Memory) and a ROM (Read Only Memory). The CPU 30 functions as a setting unit 31, an estimation unit 32, a determination unit 33, and a first output unit 34 by executing a program stored in the storage unit 40. A skeleton model 41 and a monitoring processing program 42 are stored in the storage unit 40.

[0033] The estimation unit 32 estimates the angle of the joint J using the image data of the robot 200 captured by the camera 10. The determination unit 33 determines whether the link L has rotated beyond a predetermined angle range based on the angle estimated by the estimation unit 32, i.e., the estimated angle. The first output unit 34 outputs a report signal Siga when the determination unit 33 determines that the angle is not within the angle range. The setting unit 31 sets the angle range used by the determination unit 33 in the determination. Specifically, the setting unit 31 sets the estimated angle estimated by the estimation unit 32 using the image data at the start time point as a reference angle θIn, and the start time point is the time point when the braking mechanism 240 operates to start braking the rotation of the link L. Furthermore, the setting unit 31 sets a predetermined range centered on the reference angle θIn as the angle range.

[0034] A monitor 300 as a reporting device is communicatively connected to the information processing device 20. The monitor 300 has a display screen 301. The information transmitted from the information processing device 20 is displayed on the display screen 301.

[0035] As described above, the brake mechanism 240 in the present embodiment works when the electromagnet 243 is not energized. During the operation of the robot 200, the power supply of the arm 210 is turned on even when the brake mechanism 240 is working. In contrast, during the period when the production line is interrupted, the period from the end of production to the next production, the period of emergency stop in an emergency, etc., the operation performed by the robot 200 is not performed, so the power supply of the arm 210 is turned off. The specific process is: after the electromagnet 243 is set to be non-energized in order to make the brake mechanism 240 work, the power supply of the arm 210 is turned off. Even if the power supply of the arm 210 is turned off, since the brake mechanism 240 is working, the arm 210 is also maintained in the initial braking position, that is, the target braking position. However, the brake mechanism 240 sometimes has a reduced braking function due to, for example, the deterioration of the spring 242, resulting in the so-called slippage of the joint J from the target braking position. In this case, the arm 210 cannot maintain the target braking position, and typically, it will droop toward the installation surface due to its own weight, and sometimes interfere with the installation surface. In addition, when the power of the arm 210 is disconnected, the operator sometimes performs maintenance on the robot 200. In this case, the operator performs maintenance and other operations on the premise that the connecting rod L is maintained at the target braking position. Therefore, if the position of the connecting rod L is displaced from the target braking position, the workability may be reduced. Therefore, the inventors have planned a robot monitoring system 100 that monitors the displacement of the connecting rod L during the period when the power of the arm 210 is disconnected. Thus, the displacement of the arm 210 of the robot 200 can also be monitored during the period when the power of the arm 210 is disconnected. It should be noted that the period when the power of the arm 210 is disconnected means a state in which the braking mechanism 240 is working with respect to all joints J of the arm 210. As described later in other embodiments, the monitoring process described later can also be performed during the period when the power of the arm 210 is connected. In this case, it can be determined whether or not a portion of the joint J where the braking mechanism 240 is operating has been displaced from the target braking position.

[0036] The robot monitoring system 100 estimates the displacement of the joint J using the skeleton model 41 . Figure 3 : is a diagram for explaining the skeleton model 41. The skeleton model 41 is composed of tracking points TP and straight lines connecting two tracking points TP, namely, model links ML. In the present embodiment, the skeleton model 41 is composed of five tracking points TP and four model links ML. Specifically, the joints J at which the angle formed by the two connected links L changes, namely, the first joint J1, the second joint J2, the third joint J3, the fifth joint J5 and the sixth joint J6 as the end joint J are respectively assigned to the first tracking point TP1 to the fifth tracking point TP5. Here, regarding Figure 1The fourth joint J4 shown in the figure does not change the angle formed by the third link L3 and the fourth link L4 connected thereto, and the relative positional relationship does not change, so it is not set as the tracking point TP. The skeleton model 41 is a data group represented by the three-dimensional position information of the tracking point TP and the angle formed by the two model links ML connected to the tracking point TP, that is, the model angle θMn of the tracking point TP. It should be noted that "n" in the model angle θMn represents the number of the tracking point TP. The skeleton model 41 is pre-made using motion capture technology and stored in the storage unit 40. By using the skeleton model 41, the position information of the two-dimensional joint J can be used to infer the position of the three-dimensional joint J.

[0037] A2. Monitoring and processing

[0038] Figure 4 This is a flowchart of the monitoring process. Figure 5 4 is a diagram for explaining the angle range used in the monitoring process. The monitoring process is performed by the CPU 30 executing the monitoring process program 42. After the operator turns off the power of the arm 210, for example, the start switch for receiving the start of the monitoring process provided in the robot monitoring system 100 is turned on. If the CPU 30 receives the start instruction via the start switch, the monitoring process starts. In this embodiment, the case of monitoring the second joint J2, the third joint J3, and the fifth joint J5 among the six joints J is exemplified for explanation.

[0039] First, the CPU 30 causes the camera 10 to photograph the arm 210 of the robot 200 (step S10). The photographed image data is sent to the information processing device 20. Using the sent image data, the estimation unit 32 estimates the position of each joint J in the image (step S12). Specifically, the estimation unit 32 binarizes the image data and detects the edge of the image representing the outline of the arm 210 through image processing. After that, the detected edge data is used as input, and AI (artificial intelligence) is used to estimate the position of the joint J in the image.

[0040] The estimated angle θn of each joint J is calculated by the estimation unit 32 (step S14). Here, "n" of the estimated angle θn represents the number of the joint J. For example, the estimated angle θ2 represents the estimated angle of the second joint J2. The same applies to the reference angle θIn described later. As described above, the angle of the joint J is the angle formed by two straight lines drawn when the two links L connected to the joint J each draw a straight line connecting the two joints J at the two ends of the link L. However, in the present embodiment, as Figure 1As shown, regarding the third joint J3, in order to match the skeleton model 41, the angle formed by the second link L2 and the third link L3 is replaced by the angle formed by the straight line connecting the second joint J2 and the third joint J3 and the straight line connecting the third joint J3 and the fifth joint J5 is defined as the estimated angle θ3. The same is true for the fifth joint J5. The angle formed by the straight line connecting the third joint J3 and the fifth joint J5 and the straight line connecting the fifth joint J5 and the sixth joint J6 is defined as the estimated angle θ5. Figure 4 Specifically, in step S14, the three-dimensional position of the joint J is estimated using the estimated position of the joint J in the image and the position information of the tracking point TP of the skeleton model 41. Then, the estimating unit 32 estimates the three-dimensional description information of the link L connecting the joint J using AI to calculate the estimated angle θn.

[0041] The setting unit 31 sets the angle range using the estimated angle θn calculated by the estimating unit 32 as the reference angle θIn (step S16). Figure 5 As shown, the angle range is a range in which an angle obtained by subtracting the allowable angle θa from the reference angle θIn is a minimum angle and an angle obtained by adding the allowable angle θa to the reference angle θIn is a maximum angle. Figure 5 The second joint J2, the third joint J3, and the fifth joint J5 shown represent the positions of the joints J estimated from the image data at the start time. In the present embodiment, the angle range is determined based on the brake position at the start time when the monitoring process starts.

[0042] like Figure 4 As shown, the CPU 30 determines whether a stop instruction has been received (step S18). When the operator ends the monitoring process, the operator turns on the end switch (not shown) of the robot monitoring system 100 for receiving the end of the monitoring process. If the CPU 30 receives the end instruction via the end switch, it determines that a stop instruction has been received. If the CPU 30 determines that a stop instruction has been received (step S18: Yes), the present processing routine ends. If the CPU 30 determines that a stop instruction has not been received (step S18: No), the camera 10 is caused to photograph the arm 210 of the robot 200 (step S20). The position of each joint J in the image is estimated in the same way as step S12 (step S22). The estimated angle θn of each joint J is calculated by the estimation unit 32 in the same way as step S14 (step S24).

[0043] The determination unit 33 determines whether the second link L2 is rotating beyond the angle range based on the estimated angle θ2 of the second joint J2 (step S26). When the estimated angle θ2 is within the angle range, the determination unit 33 determines that the second link L2 is not rotating beyond the angle range. On the other hand, when the estimated angle θ2 is not within the angle range, the determination unit 33 determines that the second link L2 is rotating beyond the angle range. Specifically, the determination unit 33 determines whether the estimated angle θ2 is larger than the angle obtained by subtracting the allowable angle θa from the reference angle θI2 and smaller than the angle obtained by adding the allowable angle θa to the reference angle θI2. If the determination unit 33 determines that the second link L2 is rotating beyond the angle range, that is, the estimated angle θ2 is less than the angle obtained by subtracting the allowable angle θa from the reference angle θI2 or is greater than the angle obtained by adding the allowable angle θa to the reference angle θI2 (step S26: No), the process moves to step S32. This is because the second link L2 is rotating beyond the angle range and a slip occurs at the second joint J2. In contrast, if the determination unit 33 determines that the second link L2 is rotating without exceeding the angle range, that is, the estimated angle θ2 is larger than the angle obtained by subtracting the allowable angle θa from the reference angle θI2 and is smaller than the angle obtained by adding the allowable angle θa to the reference angle θI2 (step S26: yes), the process moves to step S28. This is because no slip occurs at the second joint J2, and the determination is then made about the third joint J3.

[0044] The determination unit 33 determines whether the third link L3 is rotating beyond the angle range based on the estimated angle θ3 of the third joint J3 (step S28). Specifically, the determination unit 33 determines whether the estimated angle θ3 is larger than the angle obtained by subtracting the allowable angle θa from the reference angle θI3 and smaller than the angle obtained by adding the allowable angle θa to the reference angle θI3. If the determination unit 33 determines that the estimated angle θ3 is not within the angle range (step S28: No), it can be inferred that slippage has occurred in the third joint J3, and thus the process moves to step S32. In contrast, if the determination unit 33 determines that the estimated angle θ3 is within the angle range (step S28: Yes), it can be inferred that no slippage has occurred in the third joint J3, and thus the process moves to step S30.

[0045] The determination unit 33 determines whether the fifth link L5 is rotating beyond the angle range based on the estimated angle θ5 of the fifth joint J5 (step S30). Specifically, the determination unit 33 determines whether the estimated angle θ5 is larger than the angle obtained by subtracting the allowable angle θa from the reference angle θI5 and smaller than the angle obtained by adding the allowable angle θa to the reference angle θI5. If the determination unit 33 determines that the estimated angle θ5 is not within the angle range (step S30: No), it can be inferred that slippage has occurred in the fifth joint J5, and thus moves to step S32. In contrast, if the determination unit 33 determines that the estimated angle θ5 is within the angle range (step S30: Yes), it can be inferred that no slippage has occurred in the fifth joint J5, and thus after a predetermined time has passed, it moves to step S18. Thus, the displacement of the arm 210 of the robot 200 can be continuously monitored. The predetermined time is, for example, about several minutes.

[0046] In step S32, the first output unit 34 outputs a report signal Siga to the monitor 300 (step S32). The report signal Siga is a signal for causing the monitor 300 to report the displacement of the link L. For example, when it is determined that the estimated angle θ3 of the third joint J3 is not within the angle range, the monitor 300 includes information such as a display message "3-axis slippage occurs" including information for determining the number of the joint J. If the monitor 300 receives the report signal Siga, it displays the included display message on the display screen 301. Thus, the operator can know that the link L has been displaced from the target braking position. After the execution of step S32, this processing routine ends.

[0047] According to the first embodiment described above, the robot monitoring system 100 includes the camera 10, the estimation unit 32, the determination unit 33, and the first output unit 34. Thus, by imaging the robot 200 with the braking mechanism 240 in operation, it is possible to determine whether the link L of the robot 200 has displaced from the target braking position.

[0048] When the determination unit 33 determines that the link L has rotated beyond the angle range based on the estimated angle θn, the first output unit 34 outputs a notification signal Siga to the monitor 300 for notifying the monitor 300 of the displacement of the link L. Thus, when the link L is displaced, the monitor 300 can be notified of the displacement. Thus, the operator can know the displacement of the link L.

[0049] The storage unit 40 stores a skeleton model 41 in advance. The estimating unit 32 estimates the angle of the joint J using the position of the joint J estimated from the image data and the skeleton model 41. Thus, the angle of the joint J can be estimated with high accuracy using the skeleton model 41.

[0050] The setting unit 31 sets the estimated angle θn estimated by the estimating unit 32 using the image data at the start time point when the braking mechanism 240 is operated and the braking of the rotation of the connecting rod L is started as the reference angle θIn, and sets a predetermined range centered on the reference angle θIn as the angle range. In this way, the displacement relative to the position of the connecting rod L at the start time point can be determined.

[0051] The determination unit 33 performs determination on each of the plurality of joints J. Thus, when the robot 200 includes a plurality of joints J, the displacement of at least one joint J can be determined.

[0052] B. Second Implementation

[0053] Figure 6 : is a schematic diagram showing the structure of the robot monitoring system 100 of the second embodiment. In the second embodiment, the robot control unit 220 and the information processing device 20 are connected in a manner capable of communication and the CPU 30 is provided with a second output unit 35 instead of the first output unit 34, which is different from the first embodiment. The same figure marks are marked on the same structures as the first embodiment, and detailed descriptions are omitted. When the determination unit 33 determines that the link L is rotating beyond the angle range, the second output unit 35 outputs a displacement signal Sigb for displacing the position of the link L to the target braking position to the robot control unit 220. Regarding the monitoring process of the second embodiment, similar to the first embodiment, in this embodiment, the second joint J2, the third joint J3 and the fifth joint J5 among the six joints J are monitored as an example.

[0054] Figure 7 This is a flowchart of the monitoring process of the second embodiment. The monitoring process of this embodiment is performed, for example, at night when the operator is not present. During the absence of the operator, if the displacement of the link L occurs, the displacement progresses due to the delay, and the link L may interfere with the installation surface or other equipment. Therefore, in this embodiment, a predetermined angle range is used for judgment. The movement of the arm 210 is performed according to a program predetermined according to the work content of the robot 200. Therefore, in this embodiment, the movement range of the link L in the pre-programmed action is set to an angle range. For example, in a case where the movement range of the joint angle in the pre-programmed action of the second joint J2 is a range greater than 40 degrees and less than 100 degrees, a range greater than 40 degrees and less than 100 degrees is set as the angle range of the second joint J2. Thereby, the displacement of the link L can be judged regardless of the initial braking position, and the above-mentioned interference can be avoided. The angle range is predetermined and stored in the storage unit 40.

[0055] exist Figure 7In the process steps having the same contents as those of the process steps of the first embodiment, the same reference numerals are given, and detailed descriptions are omitted. When the monitoring process starts, the CPU 30 determines whether a stop instruction has been received (step S18). If the CPU 30 determines that a stop instruction has been received (step S18: Yes), the present process routine is terminated. If the CPU 30 determines that a stop instruction has not been received (step S18: No), the camera 10 is caused to photograph the arm 210 of the robot 200 (step S20). The position of each joint J in the image is estimated by the estimation unit 32 (step S22). The estimated angle θn of each joint J is calculated by the estimation unit 32 (step S24).

[0056] The determination unit 33 determines whether the second link L2 has rotated beyond the angle range based on the estimated angle θ2 of the second joint J2 (step S40). Specifically, the determination unit 33 determines whether the estimated angle θ2 is larger than the minimum angle θMIN2 of the angle range and smaller than the maximum angle θMAX2 of the angle range. If the determination unit 33 determines that the second link L2 is rotating beyond the angle range, that is, the estimated angle θ2 is less than the minimum angle θMIN2 of the angle range or greater than the maximum angle θMAX2 of the angle range (step S40: No), the process proceeds to step S46. In contrast, if the determination unit 33 determines that the second link L2 is not rotating beyond the angle range, that is, the estimated angle θ2 is larger than the minimum angle θMIN2 of the angle range and smaller than the maximum angle θMAX2 of the angle range (step S40: Yes), the process proceeds to step S42.

[0057] The determination unit 33 determines whether the third link L3 has rotated beyond the angle range of the third joint J3 based on the estimated angle θ3 of the third joint J3 (step S42). Specifically, the determination unit 33 determines whether the estimated angle θ3 is larger than the minimum angle θMIN3 of the angle range and smaller than the maximum angle θMAX3 of the angle range. If the determination unit 33 determines that the estimated angle θ3 is not within the angle range (step S42: No), it moves to step S46. On the other hand, if the determination unit 33 determines that the estimated angle θ3 is within the angle range (step S42: Yes), it moves to step S44. The determination unit 33 determines whether the fifth link L5 has rotated beyond the angle range of the fifth joint J5 based on the estimated angle θ5 of the fifth joint J5 (step S44). Specifically, the determination unit 33 determines whether the estimated angle θ5 is larger than the minimum angle θMIN5 of the angle range and smaller than the maximum angle θMAX5 of the angle range. If the determination unit 33 determines that the estimated angle θ5 is not within the angle range (step S44: No), the process proceeds to step S46. On the other hand, if the determination unit 33 determines that the estimated angle θ5 is within the angle range (step S44: Yes), the process proceeds to step S18 after a predetermined time has passed.

[0058] In step S46, the displacement signal Sigb is outputted from the second output unit 35 to the robot control unit 220. The displacement signal Sigb is a signal for displacing the position of the link L to the target braking position. Here, the target braking position is the initial braking position of each joint J before the power supply of the arm 210 is turned off after the operation of the arm 210 is stopped. After the robot control unit 220 stops the operation of the arm 210, the angle of each joint J is stored. Thereafter, the power supply of the arm 210 is turned off. In addition, if the displacement signal Sigb is received, the power supply of the arm 210 is turned on, and the arm 210 is controlled in such a way that the angle of each joint J becomes the stored angle. Thus, even if the operator is not near the robot 200, a large displacement of the arm 210 can be avoided, and interference with the installation surface, etc. can be avoided. The position of the link L of the arm 210 can be reset to the target braking position. After the execution of step S46, this processing routine ends.

[0059] According to the second embodiment described above, the second output unit 35 outputs the displacement signal Sigb to the robot control unit 220 in step S46. Thus, when the displacement of the link L occurs, the arm 210 of the robot 200 can be displaced to the target braking position. Thus, even when the operator is not near the robot, the position of the link L of the arm 210 can be reset to the target braking position without the operator's hands.

[0060] C. Other implementation methods:

[0061] (C1) In the above embodiment, the brake mechanism 240 is an electromagnetic type. The brake mechanism 240 is not limited to the electromagnetic type, and may also be a mechanical type. The robot monitoring system 100 is separate from the robot 200, so it is possible to monitor the arm 210 to which the power is not turned on regardless of the type of the brake mechanism 240. In addition, in the above embodiment, the robot 200 is a 6-axis vertical multi-joint industrial robot, but it can also be applied to robots of other types.

[0062] (C2) In the above embodiment, the robot monitoring system 100 includes one camera 10. As another structure, a plurality of cameras 10 may be provided according to the size and movable range of the arm 210. In this case, it is preferable to set the cameras 10 in such a manner that the respective shooting ranges of the plurality of cameras 10 are different from each other and part of the shooting ranges of each other overlap. Specifically, it is preferable to have one camera 10 shoot the first joint J1 to the third joint J3, and have another camera 10 shoot the third joint J3 to the sixth joint J6. Thus, when the arm 210 moves, the arm 210 can be reliably photographed.

[0063] (C3) In the above embodiment, during the monitoring process, the second joint J2, the third joint J3, and the fifth joint J5 are used to determine whether the estimated angle θn is within the angle range, but the order is not limited thereto. For example, the load applied to the joints J may be determined in descending order. Thus, it is possible to make an early determination regarding the joints J that are subject to heavy loads and are prone to displacement.

[0064] (C4) In the first embodiment described above, the monitor 300 is provided as the reporting means. The reporting device is not limited to the monitor 300, and may be a speaker that issues an alarm, a light that turns on, or the like.

[0065] (C5) In the second embodiment, the target braking position is the position of each joint J before the power is turned off after the operation of the arm 210 is stopped. In addition, the target braking position may be a pre-set initial or standard position. According to this structure, the process of storing the angles of each joint J in the robot control unit 220 can be reduced.

[0066] (C6) In the second embodiment described above, the angle range is set to the range of motion of the link L in the pre-programmed motion. As another embodiment, the angle range may be set to a range based on the movable range of the specifications of the arm 210. In addition, the angle range may be set to a range based on the angles of the joints J when the arm 210 interferes.

[0067] (C7) In the robot monitoring system 100 of the first embodiment, the first output unit 34 is provided. As another structure, the robot monitoring system 100 may be provided with both the first output unit 34 and the second output unit 35 of the second embodiment. That is, when the determination unit 33 determines that the estimated angle θn is not within the angle range, the report signal Siga may be sent to the monitor 300, and the displacement signal Sigb may be sent to the robot control unit 220. The same is true for the second embodiment. In addition, in the robot monitoring system 100 of the first embodiment, the second output unit 35 may be provided instead of the first output unit 34. In the robot monitoring system 100 of the second embodiment, the first output unit 34 may be provided instead of the second output unit 35.

[0068] (C8) In the above embodiment, the monitoring process is executed after the power supply of the arm 210 is turned off. Here, the state in which the power supply of the arm 210 is turned off means the state in which the brake mechanism 240 is working on all the joints J of the arm 210. In addition, the monitoring process may be executed while the power supply of the arm 210 is turned on. Thus, when the brake mechanism 240 is working on a part of the joints J of the arm 210, it is possible to determine whether the link L2 has rotated beyond a predetermined angle range with respect to the joint J where the brake mechanism 240 is working. Specifically, during operation, for example, with respect to the second joint J2 among the plurality of joints J, when the angle of the second joint J2 is not changed, by performing the monitoring process on the second joint J2, it is possible to determine whether the second link L2 has rotated beyond a predetermined angle range. In the monitoring process, the angle of each joint J can be estimated regardless of the position of the joint J. Thus, when the brake mechanism 240 is working on a part of the joints J of the arm 210, it is possible to monitor whether slippage has occurred with respect to the joint J where the brake mechanism 240 is working.

[0069] The present disclosure is not limited to the above-mentioned embodiments, and can be implemented in various structures within the scope of its main purpose. For example, the technical features of the embodiments corresponding to the technical features in each method described in the invention content column can be appropriately replaced or combined in order to solve part or all of the above-mentioned problems or to achieve part or all of the above-mentioned effects. In addition, as long as the technical feature is not described as a necessary technical feature in this specification, it can be appropriately deleted.

[0070] Description of Reference Numerals

[0071] 10…camera, 20…information processing device, 30…CPU, 31…setting unit, 32…estimation unit, 33…judgment unit, 34…first output unit, 35…second output unit, 40…storage unit, 41…skeleton model, 42…monitoring processing program, 100…robot monitoring system, 200…robot, 210…arm, 220…robot control unit, 230…motor, 231…stator, 232…rotor, 240…brake mechanism, 241…friction plate, 242…spring, 243…electromagnet, 250…base, 256…end effector, 300…monitor, 301…display screen, CA1…first axis, CA2…second axis, CA3…third axis, CA 4…fourth axis, CA5…fifth axis, CA6…sixth axis, J…joint, J1…first joint, J2…second joint, J3…third joint, J4…fourth joint, J5…fifth joint, J6…sixth joint, L…link, L1…first link, L2…second link, L3…third link, L4…fourth link, L5…fifth link, ML…model link, Siga…report signal, Sigb…displacement signal, TP…tracking point, TP1…first tracking point, TP2…second tracking point, TP3…third tracking point, TP4…fourth tracking point, TP5…fifth tracking point, θMn…model angle, θIn…reference angle, θa…allowable angle, θn…estimated angle.

Claims

1. A robot monitoring system, comprising a robot having a joint that rotatably connects a link, a robot control unit that controls the rotation of the link, and a brake mechanism that brakes the rotation of the link, in, have: A plurality of photographing units for photographing the robot when the braking mechanism is in operation; an estimating unit that estimates the angle of the joint using the image data captured by each of the plurality of imaging units; a determination unit that determines whether the link has rotated beyond a predetermined angle range based on the estimated angle estimated by the estimation unit; a setting unit that uses the image data at the start time point when the braking mechanism operates and the braking of the rotation of the link begins and the estimated angle estimated by the estimating unit as a reference angle, and sets a predetermined range centered on the reference angle as the angle range; as well as The second output unit outputs a displacement signal for displacing the position of the link toward a target braking position to the robot control unit when the determination unit determines that the position is not within the angular range after the braking mechanism has been actuated.

2. The robot monitoring system according to claim 1, The invention further includes a first output unit that outputs a notification signal to a notification device for causing the notification device to notify that the link has been displaced, when the determination unit determines that the link is not within the angular range.

3. The robot monitoring system according to claim 1 or 2, further comprising a storage unit for storing a skeleton model composed of the joints and the links, The estimating unit estimates the position of the joint based on the image data, and estimates the angle using the estimated position of the joint and the skeleton model.

4. The robot monitoring system according to claim 1 or 2, The robot has a plurality of joints. The determination unit determines whether the estimated angle is within the angle range for each of the plurality of joints.

Images