Robot Monitoring System

Abstract

To provide a system for determining whether or not a link is displaced from a target braking position while a robot is not powered on.SOLUTION: A robot has joints that rotatably connect links, a robot control unit that controls rotations of the links, and a brake mechanism that brakes the rotations of the links. A robot monitoring system for the robot includes an imaging unit that captures an image of the robot in a state that the brake mechanism is in operation, an estimation unit that estimates angles of the joints using the image data captured by the imaging unit, and a determination unit that determines whether or not the links rotate beyond a predetermined angle range.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a robot surveillance system. [Background technology]

[0002] Conventionally, there is known a configuration for an industrial robot that includes links connected by joints, a drive unit for rotating the links, and a braking mechanism for braking the rotation of the links (for example, Patent Document 1). During a period when the robot is not powered on, the rotation of the links is braked by the braking mechanism, and the position of the links is held at a target braking position. There are various types of braking mechanisms, and for example, in the braking device described in Patent Document 1, the rotation of the links is braked by meshing a brake member that can mesh with a gear included in the drive unit with the gear. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-064802 A Summary of the Invention [Problem to be solved by the invention]

[0004] Regardless of the various aspects, the braking performance of the braking mechanism may be degraded due to deterioration of components of the braking mechanism. In this case, when the braking mechanism is driven to brake the link, the position of the link may be displaced from the target braking position. If the position of the link is displaced from the target braking position, the following problems may occur. For example, a worker may perform maintenance on the robot during a period when the robot is not powered on. In this case, the worker performs maintenance work on the assumption that the link is maintained at the target braking position, so if the position of the link is displaced from the target braking position, workability may be degraded. In addition, if the link is displaced while the worker is absent, the displacement may progress due to a delayed response, and the link may interfere with the installation surface of the robot. Therefore, a system is desired that determines whether the link is displaced from the target braking position during a period when the braking mechanism is operating. [Means for solving the problem]

[0005] The present disclosure can be realized in the following forms. According to one embodiment of the present disclosure, there is provided a robot monitoring system for a robot having a joint that rotatably connects links, a robot control unit that controls the rotation of the links, and a braking mechanism that brakes the rotation of the links, and the robot is imaged while the braking mechanism is in operation. Multiple An imaging unit, Multiple Imaging unit Each ofa determination unit that determines whether the link has rotated beyond a predetermined angle range based on the estimated angle estimated by the estimation unit; a setting unit that sets the estimated angle estimated by the estimation unit using the image data at a start point when the braking mechanism is activated to start braking the rotation of the link as a reference angle and sets the angle range to a predetermined range centered on the reference angle; and a second output unit that outputs a displacement signal to the robot control unit to displace the position of the link to a target braking position if the determination unit determines that the angle is not within the angle range after the braking mechanism is activated.

[0006] (1) According to one aspect of the present disclosure, a robot monitoring system is provided. The robot monitoring system is a robot monitoring system for a robot having a joint that rotatably connects links, a robot control unit that controls the rotation of the link, and a braking mechanism for braking the rotation of the link, and includes an imaging unit that images the robot in a state in which the braking mechanism is activated, an estimation unit that estimates an angle of the joint using image data captured by the imaging unit, and a determination unit that determines whether the link has rotated beyond a predetermined angle range based on the estimated angle estimated by the estimation unit. According to this aspect, by capturing an image of the robot in a state in which the braking mechanism is activated by the imaging unit during a period in which the braking mechanism is activated, it is possible to determine whether the link of the robot has been displaced from a target braking position. (2) The robot monitoring system of the above aspect may further include a first output unit that outputs a notification signal to cause a notification device to notify that the link has been displaced when the determination unit determines that the angle is not within the angle range. According to this aspect, when a displacement of the link occurs, the notification device can be made to notify the displacement. This allows a worker to know the displacement of the link. (3) The robot monitoring system of the above aspect may further include a second output unit that outputs a displacement signal to the robot control unit to displace the position of the link to a target braking position when the determination unit determines that the angle is not within the angle range. According to this aspect, when a displacement of the link occurs, the link can be displaced to the target braking position. As a result, even if a worker is not near the robot, the link position of the robot can be returned to the target braking position without human intervention. (4) In the robot monitoring system of the above aspect, a storage unit may be provided for storing a skeletal model constituted by the joints and the links, and the estimation unit may estimate the angle by using the estimated position of the joint and the skeletal model based on the image data. According to this aspect, the angle of the joint can be estimated with high accuracy by using the skeletal model. (5) The robot monitoring system of the above aspect may further include a setting unit that sets the estimated angle estimated by the estimating unit as a reference angle using the image data at a start point when the braking mechanism is activated to start braking the rotation of the link, and sets a predetermined range centered on the reference angle as the angle range. According to this aspect, it is possible to monitor the displacement of the link relative to the position at the start point. (6) In the robot monitoring system of the above aspect, the robot may include a plurality of the joints, and the determination unit may determine whether or not the estimated angle is within the angle range for each of the plurality of joints. According to this aspect, when the robot includes a plurality of joints, it is possible to determine the displacement of at least one joint. The present disclosure may be realized in various forms other than a robot monitoring system, such as a robot monitoring method, a control method for a robot monitoring system, a computer program for realizing the control method, a non-transitory recording medium on which the computer program is recorded, etc. [Brief description of the drawings]

[0007] [Figure 1]FIG. 1 is a schematic diagram showing a configuration of a monitoring system. [Diagram 2] FIG. 2 is a block diagram showing a configuration of an arm. [Diagram 3] FIG. 2 is a diagram illustrating a skeletal model. [Figure 4] 13 is a flowchart of a monitoring process. [Diagram 5] FIG. 13 is a diagram illustrating an angle range used for determination. [Figure 6] FIG. 11 is a schematic diagram showing the configuration of a monitoring system according to a second embodiment. [Figure 7] 10 is a flowchart of a monitoring process according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] A. First embodiment: A1. Surveillance system configuration: Fig. 1 is a schematic diagram showing the configuration of a robot monitoring system 100 that monitors a robot 200. Fig. 2 is a block diagram showing the configuration of an arm 210. The robot 200 is a six-axis vertical articulated industrial robot. The robot 200 performs tasks such as welding and painting on a workpiece.

[0009] The robot 200 includes an arm 210 and a robot control unit 220. The arm 210 includes a base 250, a first link L1 to a fifth link L5, and an end effector 256. The base 250 is fixed to a surface on which the robot 200 is installed. The first link L1, the second link L2, the third link L3, the fourth link L4, the fifth link L5, and the end effector 256 are connected in series in this order.

[0010] The base 250 and the first link L1 are rotatably connected at a first joint J1. The first link L1 rotates around a first axis CA1 of the first joint J1 so that the angle between the base 250 and the first link L1 changes. The first link L1 and the second link L2 are rotatably connected at a second joint J2. The second link L2 rotates around a second axis CA2 of the second joint J2 so that the angle between the first link L1 and the second link L2 changes. The angle between the first link L1 and the second link L2 is also called the angle of the second joint J2. The angle of the second joint J2 is the angle between a straight line connecting the first joint J1 and the second joint J2, which are joints J at both ends of the first link L1, and a straight line connecting the second joint J2 and the third joint J3, which are joints J at both ends of the second link L2. The angles of the third joint J3 to the fifth joint J5 described below are defined in the same manner. The second link L2 and the third link L3 are rotatably connected at the third joint J3. The third link L3 rotates around the third axis CA3 of the third joint J3 as a fulcrum so that the angle between the second link L2 and the third link L3, i.e., the angle of the third joint J3, changes. The third link L3 and the fourth link L4 are rotatably connected at the fourth joint J4. The fourth joint J4 has a fourth axis CA4 that is parallel to the direction in which the fourth link L4 extends. The fourth link L4 rotates around the fourth axis CA4 as a central axis. The fourth link L4 and the fifth link L5 are rotatably connected at the fifth joint J5. The fifth link L5 rotates around the fifth axis CA5 of the fifth joint J5 as a fulcrum so that the angle between the fourth link L4 and the fifth link L5, i.e., the angle of the fifth joint J5, changes. The fifth link L5 and the end effector 256 are rotatably connected at a sixth joint J6. The sixth joint J6 has a sixth axis CA6 that is parallel to the direction in which the fifth link L5 extends. The end effector 256 rotates about the sixth axis CA6 of the sixth joint J6 as a central axis. The end effector 256 is, for example, a welding gun or a painting gun. The first link L1 to the fifth link L5 are also collectively referred to as links L. The first joint J1 to the fifth joint J5 are also collectively referred to as joints J.

[0011] The robot control unit 220 is configured as a computer equipped with a CPU (Central Processing Unit) and a memory, and is communicatively connected to the arm 210. The robot control unit 220 controls the arm 210. Specifically, the robot control unit 220 controls the on / off of the power supply for the arm 210, controls the rotation of the link L, and controls the braking mechanism 240, which will be described later.

[0012] 2, each joint J has a motor 230 and a braking mechanism 240. Each link L is rotated by the driving force of the connected motor 230. The motor 230 is an electromagnetic motor having a stator 231 and a rotor 232.

[0013] The braking mechanism 240 is a non-excitation actuation type braking mechanism. The braking mechanism 240 has a friction plate 241, a spring 242, and an electromagnet 243. The friction plate 241 is provided facing a fixed plate (not shown) fixed to the motor 230, sandwiching the rotor 232 in the axial direction of the rotor 232. The spring 242 is attached so as to bias the friction plate 241 toward the fixed plate in the axial direction of the rotor 232. When braking is not commanded by the robot control unit 220, the electromagnet 243 is energized, so that the friction plate 241 is electromagnetically attracted to the electromagnet 243 against the biasing force of the spring 242. This allows the rotor 232 to rotate freely. On the other hand, when braking is commanded by the robot control unit 220, the electromagnet 243 is de-energized, so that the electromagnetic attraction of the friction plate 241 by the electromagnet 243 is released. As a result, the friction plate 241 is pressed against the rotor 232 by the biasing force of the spring 242. Therefore, the rotation of the rotor 232 is restricted and the rotor 232 is held in the braking position.

[0014] As shown in FIG. 1, the robot monitoring system 100 includes a camera 10 as an imaging unit and an information processing device 20. The camera 10 has an image sensor and is installed so as to be able to image the entire arm 210. The camera 10 images the robot 200 in a state in which the braking mechanism 240 is operating. Here, the state in which the braking mechanism 240 is operating refers to a state in which the braking mechanism 240 brakes the rotation of the rotor 232. The information processing device 20 includes a CPU 30 and a storage unit 40. The storage unit 40 is realized by a memory such as a RAM (Random Access Memory) and a ROM (Read Only Memory). The CPU 30 executes a program stored in the storage unit 40 to function as a setting unit 31, an estimation unit 32, a determination unit 33, and a first output unit 34. A skeleton model 41 and a monitoring processing program 42 are stored in the storage unit 40.

[0015] The estimation unit 32 estimates the angle of the joint J using image data of the robot 200 captured by the camera 10. The determination unit 33 determines whether the link L has rotated beyond a predetermined angle range based on the estimated angle estimated by the estimation unit 32. The first output unit 34 outputs a notification signal Siga when the determination unit 33 determines that the link L is not within the angle range. The setting unit 31 sets the angle range used by the determination unit 33 for the determination. Specifically, the setting unit 31 sets the estimated angle estimated by the estimation unit 32 using image data at the start time when the braking mechanism 240 is activated and braking of the rotation of the link L is started as the reference angle θIn. Then, the setting unit 31 sets the angle range to a predetermined range centered on the reference angle θIn.

[0016] A monitor 300 serving as a notification device is communicatively connected to the information processing device 20. The monitor 300 has a display screen 301. Information transmitted from the information processing device 20 is displayed on the display screen 301.

[0017] As described above, the braking mechanism 240 in this embodiment operates when the electromagnet 243 is de-energized. During the operation of the robot 200, the power supply of the arm 210 is on even during the operation of the braking mechanism 240. In contrast, during the period when the production line is interrupted, the period after the end of production until the next production, the period during the emergency stop, and the like, no operation is performed by the robot 200, and the power supply of the arm 210 is turned off. In specific procedures, in order to operate the braking mechanism 240, the electromagnet 243 is de-energized, and then the power supply of the arm 210 is turned off. Even when the power supply of the arm 210 is turned off, the braking mechanism 240 is in operation, so that the arm 210 is held at the target braking position, which is the initial braking position. However, the braking function of the braking mechanism 240 may decrease due to, for example, deterioration of the spring 242, and the joint J may be displaced from the target braking position, which is called slippage. In this case, the arm 210 cannot hold the target braking position, and typically hangs down toward the installation surface due to its own weight, and may interfere with the installation surface. In addition, when the power supply of the arm 210 is turned off, an operator may perform maintenance on the robot 200. In this case, the operator performs work such as maintenance on the assumption that the link L is held at the target braking position, so if the position of the link L is displaced from the target braking position, there is a risk of workability being reduced. Therefore, the inventors have devised a robot monitoring system 100 that monitors the displacement of the link L during a period when the power supply of the arm 210 is turned off. This makes it possible to monitor the displacement of the arm 210 of the robot 200 even during a period when the power supply of the arm 210 is turned off. Note that the period when the power supply of the arm 210 is turned off means a state in which the braking mechanism 240 is activated for all of the joints J of the arm 210. As will be described later in other embodiments, a monitoring process to be described later may be performed during a period when the power supply of the arm 210 is turned on. In this case, it is possible to determine whether or not some of the joints J for which the braking mechanism 240 is actuated are displaced from the target braking position.

[0018] The robot monitoring system 100 estimates the displacement of the joint J by using a skeleton model 41. FIG. 3 is a diagram for explaining the skeleton model 41. The skeleton model 41 is composed of tracking points TP and model links ML that are straight lines connecting two tracking points TP. In this embodiment, the skeleton model 41 is composed of five tracking points TP and four model links ML. Specifically, the first joint J1, the second joint J2, the third joint J3, and the fifth joint J5, which are joints J in which the angle formed by two connected links L changes, and the sixth joint J6, which is the terminal joint J, are assigned to the first tracking point TP1 to the fifth tracking point TP5, respectively. Here, the fourth joint J4 shown in FIG. 1 is not set as a tracking point TP because the angle formed by the connected third link L3 and fourth link L4 does not change and the positional relationship between them does not change. The skeletal model 41 is a data group represented by three-dimensional position information of the tracking point TP, a model angle θMn of the tracking point TP which is the angle between two model links ML connected to the tracking point TP, and the like. Note that "n" in the model angle θMn indicates the number of the tracking point TP. The skeletal model 41 is created in advance using a motion capture technique and stored in the storage unit 40. By using the skeletal model 41, the three-dimensional position of the joint J can be estimated using two-dimensional position information of the joint J.

[0019] A2. Monitoring process FIG. 4 is a flowchart of the monitoring process. FIG. 5 is a diagram for explaining the angle range used in the monitoring process. The monitoring process is performed by the CPU 30 executing the monitoring process program 42. After the operator turns off the power of the arm 210, for example, the operator turns on a start switch for receiving the start of the monitoring process provided in the robot monitoring system 100. When the CPU 30 receives a start instruction via the start switch, it starts the monitoring process. In this embodiment, a case where the second joint J2, the third joint J3, and the fifth joint J5 out of the six joints J are monitored will be described as an example.

[0020] First, the CPU 30 causes the camera 10 to capture an image of the arm 210 of the robot 200 (step S10). The captured image data is transmitted to the information processing device 20. Using the transmitted image data, the estimation unit 32 estimates the position of each joint J in the image (step S12). Specifically, the estimation unit 32 binarizes the image data, and detects edges of the image showing the contour of the arm 210 by image processing. Thereafter, the position of the joint J in the image is estimated using AI (artificial intelligence) with the detected edge data as input.

[0021] The estimation unit 32 calculates an estimated angle θn of each of the joints J (step S14). Here, "n" in the estimated angle θn indicates the number of the joint J, and for example, the estimated angle θ2 indicates the estimated angle of the second joint J2. The same applies to the reference angle θIn described later. As described above, the angle of the joint J is an angle formed by two straight lines drawn by drawing straight lines connecting the two joints J at both ends of the link L for each of the two links L connected to the joint J. However, in this embodiment, as shown in FIG. 1, for the third joint J3, in order to conform to the skeletal model 41, instead of the angle formed by the second link L2 and the third link L3, the angle formed by the straight line connecting the second joint J2 and the third joint J3 and the straight line connecting the third joint J3 and the fifth joint J5 is defined as the estimated angle θ3. Similarly, for the fifth joint J5, the angle between the line connecting the third joint J3 and the fifth joint J5 and the line connecting the fifth joint J5 and the sixth joint J6 is defined as the estimated angle θ5. In step S14 of FIG. 4, specifically, the three-dimensional position of the joint J is estimated using the estimated position of the joint J in the image and the position information of the tracking point TP of the skeletal model 41. Thereafter, the estimation unit 32 generates three-dimensional description information of the link L connecting the joints J using AI, and calculates the estimated angle θn.

[0022] The setting unit 31 sets an angle range in which the estimated angle θn calculated by the estimation unit 32 is the reference angle θIn (step S16). Here, the angle range is a range in which the angle obtained by subtracting the allowable angle θa from the reference angle θIn is the minimum angle and the angle obtained by adding the allowable angle θa to the reference angle θIn is the maximum angle, as shown in FIG. 5. The second joint J2, the third joint J3, and the fifth joint J5 shown in FIG. 5 indicate the positions of the joint J estimated from the image data at the start time. In this embodiment, the angle range is determined based on the braking position at the start time when the monitoring process is started.

[0023] As shown in FIG. 4, the CPU 30 judges whether or not a stop instruction has been received (step S18). When the worker wants to end the monitoring process, he or she turns on an end switch (not shown) provided in the robot monitoring system 100 for receiving the end of the monitoring process. When the CPU 30 receives the end instruction via the end switch, it judges that a stop instruction has been received. When the CPU 30 judges that a stop instruction has been received (step S18: YES), it ends this processing routine. When the CPU 30 judges that a stop instruction has not been received (step S18: NO), it causes the camera 10 to capture an image of the arm 210 of the robot 200 (step S20). As in step S12, the position of each joint J in the image is estimated (step S22). As in step S14, the estimation unit 32 calculates an estimated angle θn of each joint J (step S24).

[0024] The determination unit 33 determines whether the second link L2 has rotated beyond the angle range based on the estimated angle θ2 of the second joint J2 (step S26). When the estimated angle θ2 is within the angle range, the determination unit 33 determines that the second link L2 has not rotated beyond the angle range. On the other hand, when the estimated angle θ2 is not within the angle range, the determination unit 33 determines that the second link L2 has rotated beyond the angle range. Specifically, the determination unit 33 determines whether the estimated angle θ2 is greater than an angle obtained by subtracting the allowable angle θa from the reference angle θI2 and smaller than an angle obtained by adding the allowable angle θa to the reference angle θI2. When the determination unit 33 determines that the second link L2 has rotated beyond the angle range, that is, the estimated angle θ2 is equal to or less than an angle obtained by subtracting the allowable angle θa from the reference angle θI2 or equal to or more than an angle obtained by adding the allowable angle θa to the reference angle θI2 (step S26: NO), the process proceeds to step S32. This is because the second link L2 has rotated beyond the angle range, causing slippage at the second joint J2. On the other hand, when the determination unit 33 determines that the second link L2 has not rotated beyond the angle range, that is, that the estimated angle θ2 is greater than the angle obtained by subtracting the permissible angle θa from the reference angle θI2 and less than the angle obtained by adding the permissible angle θa to the reference angle θI2 (step S26: YES), the process proceeds to step S28. This is because no slippage has occurred at the second joint J2, and next, a determination is made regarding the third joint J3.

[0025] The determination unit 33 determines whether the third link L3 has rotated beyond the angle range based on the estimated angle θ3 of the third joint J3 (step S28). Specifically, the determination unit 33 determines whether the estimated angle θ3 is greater than an angle obtained by subtracting the allowable angle θa from the reference angle θI3 and less than an angle obtained by adding the allowable angle θa to the reference angle θI3. If the determination unit 33 determines that the estimated angle θ3 is not within the angle range (step S28: NO), it can be assumed that slippage has occurred at the third joint J3, and therefore the process proceeds to step S32. On the other hand, if the determination unit 33 determines that the estimated angle θ3 is within the angle range (step S28: YES), it can be assumed that slippage has not occurred at the third joint J3, and therefore the process proceeds to step S30.

[0026] The determination unit 33 determines whether the fifth link L5 rotates beyond the angle range based on the estimated angle θ5 of the fifth joint J5 (step S30). Specifically, the determination unit 33 determines whether the estimated angle θ5 is greater than an angle obtained by subtracting the allowable angle θa from the reference angle θI5 and smaller than an angle obtained by adding the allowable angle θa to the reference angle θI5. If the determination unit 33 determines that the estimated angle θ5 is not within the angle range (step S30: NO), it can be estimated that slippage has occurred in the fifth joint J5, and the process proceeds to step S32. On the other hand, if the determination unit 33 determines that the estimated angle θ5 is within the angle range (step S30: YES), it can be estimated that slippage has not occurred in the fifth joint J5, and the process proceeds to step S18 after a predetermined time has elapsed. This allows the displacement of the arm 210 of the robot 200 to be continuously monitored. The predetermined time is, for example, about several minutes.

[0027] In step S32, the first output unit 34 outputs a notification signal Siga to the monitor 300 (step S32). The notification signal Siga is a signal for notifying the monitor 300 of the displacement of the link L, and includes, for example, a display message such as "3-axis slippage has occurred" including information specifying the number of the joint J when it is determined that the estimated angle θ3 of the third joint J3 is not within the angle range. When the monitor 300 receives the notification signal Siga, it displays the included display message on the display screen 301. This allows the operator to know that the link L has been displaced from the target braking position. After execution of step S32, this processing routine ends.

[0028] According to the first embodiment described above, the robot monitoring system 100 includes the camera 10, the estimation unit 32, the determination unit 33, and the first output unit 34. As a result, by capturing an image of the robot 200 in a state in which the braking mechanism 240 is operating, it is possible to determine whether or not the link L of the robot 200 has been displaced from the target braking position.

[0029] When the determination unit 33 determines that the link L has rotated beyond the angle range based on the estimated angle θn, the first output unit 34 outputs a notification signal Siga for notifying the monitor 300 of the displacement of the link L. This makes it possible to cause the monitor 300 to notify the displacement when the displacement of the link L occurs. This allows the worker to know the displacement of the link L.

[0030] The storage unit 40 stores a skeletal model 41 in advance. Then, the estimation unit 32 estimates the angle of the joint J using the position of the joint J estimated from the image data and the skeletal model 41. In this way, the angle of the joint J can be estimated with high accuracy using the skeletal model 41.

[0031] The setting unit 31 uses image data at the start time when the braking mechanism 240 is actuated to start braking the rotation of the link L to set the estimated angle θn estimated by the estimation unit 32 as the reference angle θIn, and sets a predetermined range centered on the reference angle θIn as the angle range. This makes it possible to determine the displacement of the link L relative to its position at the start time.

[0032] The determination unit 33 performs a determination for each of the multiple joints J. In this way, when the robot 200 includes multiple joints J, the displacement of at least one joint J can be determined.

[0033] B. Second embodiment FIG. 6 is a schematic diagram showing the configuration of a robot monitoring system 100 according to a second embodiment. The second embodiment is different from the first embodiment in that the robot control unit 220 and the information processing device 20 are connected to be able to communicate with each other, and the CPU 30 includes a second output unit 35 instead of the first output unit 34. The same components as those in the first embodiment are given the same reference numerals, and detailed descriptions are omitted. When the determination unit 33 determines that the link L has rotated beyond the angle range, the second output unit 35 outputs a displacement signal Sigb to the robot control unit 220 to displace the position of the link L to the target braking position. As with the first embodiment, the monitoring process according to the second embodiment will be described by taking as an example a case in which monitoring is performed on the second joint J2, the third joint J3, and the fifth joint J5 among the six joints J in this embodiment.

[0034] FIG. 7 is a flowchart of the monitoring process according to the second embodiment. The monitoring process according to this embodiment is executed, for example, at night when the worker is absent. If the link L is displaced while the worker is absent, the response is delayed and the displacement advances, and the link L may interfere with the installation surface or other equipment. Therefore, in this embodiment, a predetermined angle range is used for the determination. The movement of the arm 210 operates according to a program that is predetermined according to the work content of the robot 200. Therefore, in this embodiment, the movement range of the link L in the pre-programmed movement is set to the angle range. For example, if the movement range of the joint angle in the pre-programmed movement of the second joint J2 is a range greater than 40 degrees and less than 100 degrees, a range greater than 40 degrees and less than 100 degrees is set to the angle range of the second joint J2. This makes it possible to determine the displacement of the link L regardless of the initial braking position, and to avoid the above-mentioned interference. The angle range is predetermined and stored in the storage unit 40.

[0035] In Fig. 7, the same process steps as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted. When the monitoring process is started, the CPU 30 judges whether or not a stop instruction has been received (step S18). When the CPU 30 judges that a stop instruction has been received (step S18: YES), this process routine ends. When the CPU 30 judges that a stop instruction has not been received (step S18: NO), the camera 10 captures an image of the arm 210 of the robot 200 (step S20). The estimation unit 32 estimates the position of each joint J in the image (step S22). The estimation unit 32 calculates an estimated angle θn of each joint J (step S24).

[0036] The determination unit 33 determines whether the second link L2 has rotated beyond the angle range based on the estimated angle θ2 of the second joint J2 (step S40). Specifically, the determination unit 33 determines whether the estimated angle θ2 is greater than the minimum angle θMIN2 of the angle range and less than the maximum angle θMAX2 of the angle range. If the determination unit 33 determines that the second link L2 has rotated beyond the angle range, that is, the estimated angle θ2 is equal to or less than the minimum angle θMIN2 of the angle range or equal to or greater than the maximum angle θMAX2 of the angle range (step S40: NO), the process proceeds to step S46. On the other hand, if the determination unit 33 determines that the second link L2 has not rotated beyond the angle range, that is, the estimated angle θ2 is greater than the minimum angle θMIN2 of the angle range and less than the maximum angle θMAX2 of the angle range (step S40: YES), the process proceeds to step S42.

[0037] The determination unit 33 determines whether the third link L3 has rotated beyond the angle range of the third joint J3 based on the estimated angle θ3 of the third joint J3 (step S42). Specifically, the determination unit 33 determines whether the estimated angle θ3 is greater than the minimum angle θMIN3 of the angle range and less than the maximum angle θMAX3 of the angle range. If the determination unit 33 determines that the estimated angle θ3 is not within the angle range (step S42: NO), the process proceeds to step S46. On the other hand, if the determination unit 33 determines that the estimated angle θ3 is within the angle range (step S42: YES), the process proceeds to step S44. The determination unit 33 determines whether the fifth link L5 has rotated beyond the angle range of the fifth joint J5 based on the estimated angle θ5 of the fifth joint J5 (step S44). Specifically, the determination unit 33 determines whether the estimated angle θ5 is greater than the minimum angle θMIN5 of the angle range and less than the maximum angle θMAX5 of the angle range. If the determination unit 33 determines that the estimated angle θ5 is not within the angle range (step S44: NO), the process proceeds to step S46. On the other hand, if the determination unit 33 determines that the estimated angle θ5 is within the angle range (step S44: YES), the process proceeds to step S18 after a predetermined time has elapsed.

[0038] In step S46, the second output unit 35 outputs a displacement signal Sigb to the robot control unit 220. The displacement signal Sigb is a signal for displacing the position of the link L to a target braking position. Here, the target braking position is the initial braking position of each joint J before the power supply of the arm 210 is turned off after the work by the arm 210 is stopped. After the work by the arm 210 is stopped, the robot control unit 220 stores the angle of each joint J. Then, the power supply of the arm 210 is turned off. Then, upon receiving the displacement signal Sigb, the power supply of the arm 210 is turned on, and the arm 210 is controlled so that the angle of each joint J becomes the stored angle. This makes it possible to avoid a large displacement of the arm 210 even when there is no worker near the robot 200, and to avoid interference with the installation surface, etc. The position of the link L of the arm 210 can be returned to the target braking position. After execution of step S46, this processing routine ends.

[0039] According to the second embodiment described above, the second output unit 35 outputs the displacement signal Sigb to the robot control unit 220 in step S46. This makes it possible to displace the arm 210 of the robot 200 to the target braking position when displacement of the link L occurs. This makes it possible to return the position of the link L of the arm 210 to the target braking position without human intervention, even if an operator is not near the robot.

[0040] C. Other embodiments: (C1) In the above embodiment, the braking mechanism 240 is electromagnetic. The braking mechanism 240 is not limited to being electromagnetic, and may be mechanical. The robot monitoring system 100 is separate from the robot 200, and therefore can monitor the arm 210 that is not powered on, regardless of the type of braking mechanism 240. Also, in the above embodiment, the robot 200 is a six-axis vertical articulated industrial robot, but the present invention can also be applied to other types of robots.

[0041] (C2) In the above embodiment, the robot monitoring system 100 includes one camera 10. Alternatively, multiple cameras 10 may be included depending on the size and movable range of the arm 210. In this case, the multiple cameras 10 may be installed so that the imaging ranges of the cameras 10 are different from one another and part of the imaging ranges overlap each other. Specifically, one camera 10 may image the first joint J1 to the third joint J3, and another camera 10 may image the third joint J3 to the sixth joint J6. This allows the arm 210 to be reliably imaged even when the arm 210 moves.

[0042] (C3) In the above embodiment, in the monitoring process, the judgment as to whether the estimated angle θn is within the angle range is performed in the order of the second joint J2, the third joint J3, and the fifth joint J5, but the order is not limited to this. For example, the judgment may be performed in the order of the load applied to the joint J. This allows the judgment to be performed early on for the joint J that is subject to a large load and is likely to be displaced.

[0043] (C4) In the above first embodiment, the monitor 300 is provided as a notification device. The notification device is not limited to the monitor 300, and may be a speaker that issues an alarm, a light that turns on, or the like.

[0044] (C5) In the above second embodiment, the target braking position is the position of each joint J after the work by the arm 210 is stopped and before the power is turned off. Alternatively, the target braking position may be a preset initial or standard position. With this configuration, it is possible to eliminate the process of storing the angle of each joint J by the robot control unit 220.

[0045] (C6) In the second embodiment described above, the angle range is set to the motion range of the link L in a pre-programmed motion. In another embodiment, the angle range may be set to a range based on the movable range according to the specifications of the arm 210. Also, the angle range may be set to a range based on the angle of each joint J when the arm 210 causes interference.

[0046] (C7) The robot monitoring system 100 according to the first embodiment has the first output unit 34. As another configuration, the robot monitoring system 100 may have both the first output unit 34 and the second output unit 35 according to the second embodiment. That is, when the determination unit 33 determines that the estimated angle θn is not within the angle range, the determination unit 33 may transmit the notification signal Siga to the monitor 300 and the displacement signal Sigb to the robot control unit 220. The same applies to the second embodiment. Also, the robot monitoring system 100 according to the first embodiment may have the second output unit 35 instead of the first output unit 34. The robot monitoring system 100 according to the second embodiment may have the first output unit 34 instead of the second output unit 35.

[0047] (C8) In the above embodiment, the monitoring process is executed after the power supply of the arm 210 is turned off. Here, the state in which the power supply of the arm 210 is turned off means a state in which the brake mechanism 240 is activated for all of the joints J of the arm 210. Separately from this, the monitoring process may be executed during a period in which the power supply of the arm 210 is turned on. As a result, when the brake mechanism 240 is activated for some of the joints J of the arm 210, it is possible to determine whether or not the link has rotated beyond a predetermined angle range for the joint J in which the brake mechanism 240 is activated. Specifically, for example, for the second joint J2 among the multiple joints J in the work, when the angle of the second joint J2 is not changed, it is possible to determine whether or not the second link L2 has rotated beyond a predetermined angle range by performing the monitoring process for the second joint J2. In the monitoring process, the angle of each joint J can be estimated regardless of the position of the joint J. Therefore, when the braking mechanism 240 is actuated for some of the joints J of the arm 210, it is possible to monitor whether or not slippage occurs for the joints J for which the braking mechanism 240 is actuated.

[0048] The present disclosure is not limited to the above-mentioned embodiment, and can be realized in various configurations without departing from the spirit of the present disclosure. For example, the technical features of the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention can be appropriately replaced or combined to solve some or all of the above-mentioned problems or to achieve some or all of the above-mentioned effects. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]

[0049] 10...camera, 20...information processing device, 30...CPU, 31...setting unit, 32...estimation unit, 33...determination unit, 34...first output unit, 35...second output unit, 40...storage unit, 41...skeletal model, 42...monitoring processing program, 100...robot monitoring system, 200...robot, 210...arm, 220...robot control unit, 230...motor, 231...stator, 232...rotor, 240...braking mechanism, 241...friction plate, 242...spring, 243...electromagnet, 250...base, 256...end effector, 300...monitor, 301...display screen, CA1...first axis, CA2...second axis, CA3...third axis, CA4...fourth axis, CA5...fifth axis, CA 6...6th axis, J...joint, J1...1st joint, J2...2nd joint, J3...3rd joint, J4...4th joint, J5...5th joint, J6...6th joint, L...link, L1...1st link, L2...2nd link, L3...3rd link, L4...4th link, L5...5th link, ML...model link, Siga...notification signal, Sigb...displacement signal, TP...tracking point, TP1...1st tracking point, TP2...2nd tracking point, TP3...3rd tracking point, TP4...4th tracking point, TP5...5th tracking point, θMn...model angle, θIn...reference angle, θa...allowable angle, θn...estimated angle

Claims

1. A robot monitoring system for a robot having a joint that rotatably connects links, a robot control unit that controls the rotation of the links, and a braking mechanism that brakes the rotation of the links, a plurality of imaging units for imaging the robot in a state in which the braking mechanism is activated; an estimation unit that estimates an angle of the joint using image data captured by each of the plurality of imaging units; a determination unit that determines whether or not the link has rotated beyond a predetermined angle range based on the estimated angle estimated by the estimation unit; a setting unit that sets the estimated angle estimated by the estimation unit using the image data at a start point when the braking mechanism is activated to start braking the rotation of the link as a reference angle, and sets a predetermined range centered on the reference angle as the angle range; a second output unit that outputs a displacement signal to the robot control unit to displace a position of the link to a target braking position when the determination unit determines that the angle is not within the angle range after the braking mechanism is activated.

2. 2. The robot monitoring system of claim 1, A robot monitoring system, wherein the target braking position is an initial braking position at the time when operation of the braking mechanism is initiated.

3. 2. The robot monitoring system of claim 1, A robot monitoring system, wherein the target braking position is a preset standard position that is different from an initial braking position at the time when operation of the braking mechanism is initiated.

4. The robot monitoring system according to any one of claims 1 to 3, further comprising: A robot monitoring system comprising: a first output unit that outputs an alarm signal to an alarm device to notify the alarm device that the link has been displaced when the determination unit determines that the angle is not within the angle range.

5. The robot monitoring system according to any one of claims 1 to 4, further comprising: a storage unit that stores a skeletal model configured by the joints and the links, A robot monitoring system, wherein the estimation unit estimates a position of the joint from the image data, and estimates the angle using the estimated position of the joint and the skeletal model.

6. A robot monitoring system according to any one of claims 1 to 5, The robot includes a plurality of the joints, The determination unit determines whether the estimated angle for each of the plurality of joints is within the angle range.

Images