Robot system and robot control method

Abstract

To provide a robot system that effectively removes floating matter on the surface of a liquid object.SOLUTION: A robot system includes: a robot which has a robot arm and an end effector and is arranged in adjacency to a storage body; an operation device which is arranged away from the storage body, receives input of remote control to the robot, and outputs a command for operating the robot in accordance with the remote control; an imaging device for imaging the surface of liquid object of the storage body; and a presentation device for displaying an image captured by the imaging device; and a control device. The end effector has a structure capable of moving floating matter on the surface of the liquid object. The control device is configured to allow the robot arm to move the end effector so as to move the floating matter outside the storage body according to the command.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to robotic systems. [Background technology]

[0002] Robots have been used to assist or replace tasks previously performed by humans. In particular, the use of robots to assist or replace tasks under harsh working environments is useful. For example, Patent Document 1 discloses a removal device that removes impurities called dross that float on the surface of a high-temperature molten metal coating bath. This removal device is equipped with a robot that automatically collects the dross, lifts it out of the coating bath, and deposits it into a collection box according to a predetermined program. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 5-302157 Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, the robot performs each operation related to dross removal in a predetermined order and at a predetermined time according to a program. However, the distribution of dross on the surface of the plating bath is neither uniform nor constant, and varies depending on, for example, the object to be plated. For this reason, it is difficult for the robot of Patent Document 1, which automatically repeats the same operation, to efficiently remove dross.

[0005] The present disclosure aims to provide a robot system that effectively removes floating matter on the surface of a liquid. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, a robot system according to one aspect of the present disclosure includes a robot having a robot arm and an end effector at the tip of the robot arm and arranged adjacent to a container that contains a liquid material, an operation input device arranged away from the container and accepting remote operation input to the robot and outputting commands for operating the robot in response to the remote operation, an imaging device that images the surface of the liquid material, a presentation device that displays an image captured by the imaging device, and a control device that controls the operation of the robot, wherein the end effector is configured to move floating objects on the surface of the liquid material, and the control device is configured to cause the robot arm to move the end effector so as to move the floating objects from the container to the outside in accordance with the commands output from the operation input device.

[0007] According to the present disclosure, it becomes possible to effectively remove floating matter on the surface of a liquid material. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of a configuration of a robot system according to an embodiment; [Diagram 2] FIG. 1 is a side view showing an example of a configuration of a robot according to an embodiment; [Diagram 3] FIG. 1 is a perspective view showing an example of a configuration of an operating device according to an embodiment; [Figure 4] FIG. 4 is an enlarged perspective view of the operation unit of FIG. 3; [Diagram 5] FIG. 1 is a block diagram showing an example of a configuration of a control device and its peripherals according to an embodiment; [Figure 6] FIG. 1 is a block diagram showing an example of a functional configuration of a control device according to an embodiment; [Figure 7] 1 is a flowchart showing an example of an operation in manual operation of a robot system according to an embodiment. [Figure 8] FIG. 1 is a diagram showing an example of a configuration of a robot system 1A according to a modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] First, each embodiment of the present disclosure will be described. A robot system according to one embodiment of the present disclosure includes a robot having a robot arm and an end effector at the tip of the robot arm and disposed adjacent to a container that contains a liquid material, an operation input device disposed away from the container and receiving an input of a remote operation to the robot and outputting a command for operating the robot in response to the remote operation, an imaging device that captures an image of the surface of the liquid material, a presentation device that displays an image captured by the imaging device, and a control device that controls the operation of the robot, the end effector having a configuration that can move a floating object on the surface of the liquid material, and the control device is configured to cause the robot arm to move the end effector so as to move the floating object from the container to the outside according to the command output from the operation input device.

[0010] According to the above aspect, a user can recognize the position and state of floating objects on the surface of the liquid object by viewing an image displayed on the display device at a remote location away from the robot, and input an appropriate command according to the recognition result to the operation input device to cause the robot to move the floating objects from the container to the outside. For example, when the floating objects are impurities such as slag generated in the molten metal in the iron-making process, steel-making process, or casting process, it is difficult to collect the slag using a collection net or the like because the molten metal is much hotter than the hot-dip metal plating bath. For example, the impurities such as slag are collected by being entangled using a rod-shaped object. The robot, whose operation is controlled in response to the state of the impurities based on the user's judgment, can effectively collect various impurities. Therefore, it is possible to effectively remove floating objects on the surface of the liquid object.

[0011] A robot system according to one aspect of the present disclosure may further include an image processing device that processes an image captured by the imaging device, the image processing device being configured to perform image processing that differentiates between the liquid matter and the floating matter, and the presentation device may display the image processed by the imaging device.

[0012] According to the above aspect, the user can easily visually recognize the position and state of the floating matter on the surface of the liquid matter in the image displayed on the presentation device. Therefore, the user can input an appropriate command to the operation input device according to the state of the floating matter.

[0013] In a robot system according to one aspect of the present disclosure, the image processing device may perform image processing to differentiate between the liquid matter and the floating matter based on the brightness values ​​of pixels included in the image.

[0014] According to the above aspect, for example, in the case of a liquid having a high brightness value, or in the case where the brightness value of the liquid and the brightness value of the floating matter are close to each other, it is difficult to distinguish between the liquid and the floating matter on the image. However, the processed image in which the above image processing based on the brightness value has been performed enables easy visual distinction between the liquid and the floating matter. For example, the image processing device may differentiate between the liquid and the floating matter by performing different image processing on pixels having a brightness value equal to or greater than a threshold value and pixels having a brightness value less than the threshold value. For example, the image processing may be processing that changes the color, brightness, shade, texture, etc. of the pixels.

[0015] A robot system according to one aspect of the present disclosure may further include a sensor for detecting a temperature of a surface of the liquid material, and the image processing device may perform image processing to differentiate between the liquid material and the floating matter based on the temperature distribution on the surface of the liquid material detected by the sensor.

[0016] According to the above aspect, for example, when there is a difference between the temperature of the liquid matter and the temperature of the floating matter, the processed image in which the image processing based on the temperature distribution has been performed allows easy visual discrimination between the liquid matter and the floating matter. For example, the image processing device may differentiate between the liquid matter and the floating matter by performing different image processing for each temperature range on images of various temperature ranges. For example, the image processing may be processing that changes the color, brightness, shading, texture, etc. of pixels.

[0017] A robot system according to one aspect of the present disclosure may further include a force detector that detects a force acting on the end effector, and the operation input device may have an operation unit that receives force from a user's hand to manually operate the robot, and a force applying device that applies force to the operation unit, and the control device may receive a detection result from the force detector, and output a command to the operation input device to apply a force to the operation unit corresponding to the force detected by the force detector.

[0018] According to the above aspect, the user can perceive the state of the force received by the end effector through the force applied to the operation unit. For example, the user can perceive whether the end effector is in contact with a floating object, whether the end effector is holding a floating object, the weight of the floating object held by the end effector, etc. This enables the user to effectively operate the robot.

[0019] In a robot system according to one aspect of the present disclosure, the control device outputs a command to apply to the operating unit the applied force obtained by reducing the detection force detected by the force detector, and when the detection force is greater than 0 and less than or equal to a first threshold, determines the applied force to be a force obtained by reducing the detection force by a first ratio, and when the detection force is greater than the first threshold, determines the applied force to be a force obtained by reducing the detection force by a second ratio, and the second ratio may be smaller than the first ratio.

[0020] According to the above aspect, a force acting on the end effector that is equal to or smaller than a first threshold is reduced to a larger first ratio and transmitted to the user via the operation unit. This allows the user to perceive even a relatively small force acting on the end effector. For example, the user can perceive whether the end effector is immersed in a liquid material, whether the end effector is in contact with a floating object, and the like.

[0021] In a robot system according to one aspect of the present disclosure, the operation input device includes an operation unit having a grip that can be held by a human hand to manually operate the robot, and an operation arm that movably supports the operation unit and is bendable and rotatable, the operation unit supports the grip rotatably about three axes so that the grip moves along a spherical surface relative to the operation arm and changes its posture, and the operation input device may output information corresponding to the amount of rotation of the grip about the three axes and the amount of bending and rotation of the operation arm as the command that commands the position and posture of the end effector.

[0022] According to the above aspect, the amount of rotation of the grip around the three axes can be associated with the posture of the end effector. The amount of bending and the amount of rotation of the operating arm can be associated with the position of the end effector. As a result, a user can move the posture of the end effector of the robot to a desired posture by inputting an operation to rotate the grip and change the posture, and can move the position of the end effector of the robot to a desired position by inputting an operation to move the grip together with the operating unit. Furthermore, since the grip is configured to move along a spherical surface and change its posture relative to the operating arm, the user can easily control the posture of the grip and easily perceive the posture of the grip.

[0023] A robot system according to one aspect of the present disclosure further includes a plurality of containers, a plurality of robots arranged adjacent to each of the plurality of containers, and a designation device that accepts input designating a robot from among the plurality of robots to be connected to the operation input device and outputs information about a designated robot that is the designated robot, wherein the control device is configured to control operation of the plurality of robots, and may operate the designated robot in accordance with the instructions output from the operation input device based on the information about the designated robot output from the designation device.

[0024] According to the above aspect, a user can operate any one of the multiple robots using one operation input device. For example, it is possible to remove floating objects from each of the containers in a predetermined order or in accordance with the occurrence status of floating objects in the container. In addition, since the multiple robots do not operate simultaneously, interference such as contact between the robots is suppressed.

[0025] A robot system according to one aspect of the present disclosure further includes a plurality of imaging devices that image the surface of the liquid material in each of the plurality of containers, and the control device is configured to control the operation of the plurality of imaging devices, and may cause the imaging device capable of imaging the container adjacent to the designated robot to image the surface of the liquid material in the container based on information of the designated robot.

[0026] According to the above aspect, the user can visually check the images of the liquid matter in each container captured by the multiple image capture devices. This allows the user to determine a container from which floating matter is to be removed based on the visual check results of the liquid matter in each container, and to have the robot placed in the container remove the floating matter.

[0027] (Embodiment) Hereinafter, the embodiments of the present disclosure will be described with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. In addition, among the components in the following embodiments, components that are not described in the independent claims showing the highest concept are described as optional components. In addition, each figure in the attached drawings is a schematic diagram and is not necessarily illustrated precisely. Furthermore, in each figure, substantially the same components are given the same reference numerals, and duplicated descriptions may be omitted or simplified. In addition, in this specification and the claims, "apparatus" may mean not only one apparatus but also a system consisting of multiple apparatuses.

[0028] [Robot system configuration] The configuration of a robot system 1 according to an embodiment will be described. FIG. 1 is a diagram showing an example of the configuration of the robot system 1 according to an embodiment. As shown in FIG. 1, the robot system 1 includes a robot 10, an operation device 20, a control device 30, an imaging device 40, a presentation device 50, and an input device 60. The control device 30 includes a first control device 31 and a second control device 32. In this embodiment, the robot system 1 performs an operation to remove floating matter S floating on the surface of molten metal M, which is a molten metal, in an iron-making process, a steel-making process, a casting process, or the like, and the following description will be given. An example of the floating matter S is impurities such as slag that are generated in the molten metal during the treatment of the molten metal.

[0029] In this embodiment, the robot system 1 is a system using a master-slave robot 10. The operation device 20 constitutes a master machine, and the robot 10 constitutes a slave machine. The operation device 20 is disposed remotely away from the robot 10, and the robot 10 is remotely controlled by the operation device 20. For example, the robot 10 and the imaging device 40 may be disposed in a work place WP where a first container Cm such as a melting furnace and a refining furnace that contains molten metal M is disposed, and the operation device 20, the control device 30, the presentation device 50, and the input device 60 may be disposed in an operation place OP such as an operation room where the influence of heat from the molten metal M in the first container Cm is reduced or blocked. The operation place OP may be a place isolated from the work place WP.

[0030] The robot system 1 is configured to allow the robot 10 to execute at least one of manual operation, automatic operation, and a combination of manual operation and automatic operation. Although not limited thereto, in the present embodiment, the robot system 1 is configured to allow the robot 10 to execute manual operation and a combination of manual operation and automatic operation. The following mainly describes the configuration for causing the robot 10 to operate in manual operation.

[0031] In manual operation, a user P such as an operator operates and operates the operation device 20 to input commands, and the operation device 20 outputs manual commands to the control device 30 for operating the robot 10 in response to the operation of the user P input to the operation device 20, and the control device 30 causes the robot 10 to perform an action corresponding to the operation. The operation device 20 is an example of an operation input device.

[0032] In the automatic operation, the user P provides an input for automatic operation of the robot 10 to the input device 60, the input device 60 outputs an automatic command for operating the robot 10 in response to the input to the control device 30, and the control device 30 causes the robot 10 to automatically operate according to the predetermined information. The predetermined information may be information including the position, posture, state, and order of each part of the robot 10. For example, the predetermined information may be preset fixed data, teaching data acquired through teaching the robot 10, and the like. The input device 60 is an example of an operation input device.

[0033] In a combination of manual operation and automatic operation, the control device 30 causes the robot 10 to execute an automatic operation and a manual operation in sequence. For example, a series of operations executed by the robot 10 for a task may include an automatic operation and a manual operation. For example, switching from automatic operation to manual operation may be performed automatically by the control device 30 in response to the progress of a series of operations. Switching from manual operation to automatic operation may be performed by the control device 30 in response to a command input by the user P to the operation device 20, the input device 60, etc.

[0034] In this embodiment, the robot 10 is an industrial robot. The robot 10 is fixedly disposed on a base 13, but may be disposed on a movable device such as a conveying device and configured to be movable. In this embodiment, the base 13 of the robot 10 is disposed adjacent to the opening Cma on the work floor WF around the opening Cma at the top of the first container Cm in the work place WP. Also, within the operating range of the robot 10, a second container Cs for collecting floating objects S is disposed. That is, the robot 10 is disposed adjacent to the second container Cs. The second container Cs may be disposed directly on the work floor WF, or may be disposed on a conveying device such as a belt conveyor on the work floor WF.

[0035] The robot 10 includes an end effector 11 that applies an action to an object to be processed, and a robot arm 12 that moves the end effector 11 to perform the action. In this embodiment, the end effector 11 is configured to be able to scrape up floating objects S on the surface of the molten metal M and to hold the floating objects S by entangling them, and the following description will be given.

[0036] The robot arm 12 is not particularly limited as long as it has a configuration that can change the position and / or posture of the end effector 11 at the tip thereof, but in this embodiment, it is a vertical articulated robot arm. Note that the robot arm 12 may be configured as, for example, a horizontal articulated type, a polar coordinate type, a cylindrical coordinate type, a Cartesian coordinate type, or any other type of robot arm.

[0037] FIG. 2 is a side view showing an example of the configuration of a robot 10 according to an embodiment. As shown in FIG. 2, a robot arm 12 is fixed to a base 13. The robot arm 12 includes links 12a to 12f arranged in order from the base to the tip, joints JT1 to JT6 that connect the links 12a to 12f in a sequentially rotatable manner, and drive devices DR1 to DR6 that rotate and drive the joints JT1 to JT6, respectively. The joints JT1 to JT6 are rotary joints. The link 12a is attached to the base 13 via the joint JT1. The tip of the link 12f forms a mechanical interface and is connected to the end effector 11. The operation of the drive devices DR1 to DR6 is controlled by a control device 30. The drive devices DR1 to DR6 are each powered by electricity and have servo motors M1 to M6 (see FIG. 5) as electric motors that drive them, but are not limited thereto. Furthermore, the driving devices DR1 to DR6 each have rotation sensors E1 to E6 (see FIG. 5) such as encoders that detect the rotation amount (e.g., rotation angle) of the servo motors M1 to M6 or the joints JT1 to JT6. The driving devices DR1 to DR6 may each have a current sensor (not shown) that detects the current value of the servo motors MA1 to MA6. The number of joints of the robot arm 12 is not limited to six, and may be seven or more or five or less.

[0038] For example, the joint JT1 connects the base end of the link 12a to the top surface 13a of the base 13 so as to be rotatable around a vertical rotation axis perpendicular to the top surface 13a. The joint JT2 connects the base end of the link 12b to the tip end of the link 12a so as to be rotatable around a horizontal rotation axis along the top surface 13a. The joint JT3 connects the base end of the link 12c to the tip end of the link 12b so as to be rotatable around a horizontal rotation axis. The joint JT4 connects the base end of the link 12d to the tip end of the link 12c so as to be rotatable around a rotation axis of the axis center in the longitudinal direction of the link 12c. The joint JT5 connects the base end of the link 12e to the tip end of the link 12d so as to be rotatable around a rotation axis perpendicular to the rotation axis of the link 12d. The joint JT6 connects the base end of the link 12e to the tip end of the link 12f so as to be rotatable about a torsional rotation axis.

[0039] The robot arm 12 as described above can freely move the position of the link 12f to any three-dimensional position within its operable range, and can freely move the attitude of the link 12f to any three-dimensional attitude. The three-dimensional position is a position in a three-dimensional space, and can be represented, for example, by coordinate values ​​of three coordinate axes in a three-dimensional Cartesian coordinate system. The three-dimensional attitude is a attitude in a three-dimensional space, and can be represented, for example, by rotation angles or Euler angles around three coordinate axes in a three-dimensional Cartesian coordinate system.

[0040] The end effector 11 includes an attachment portion 11a, a holding portion 11b, a driving device 11c, and a force detector 11d. The attachment portion 11a is configured to be connected to a mechanical interface of the link 12f, and attaches the end effector 11 to the link 12f. The holding portion 11b is configured as a columnar member, and a base end of the holding portion 11b is rotatably connected to the attachment portion 11a. Thus, the holding portion 11b rotates about its base end. The holding portion 11b has a tip portion 11ba at its tip, which extends in a direction D2 intersecting with a direction D1 in which the holding portion 11b extends. The length of the tip portion 11ba in the direction D2 is significantly shorter than the length of the holding portion 11b in the direction D1. For example, the shape of the tip portion 11ba may be a plate-like, rod-like, or wedge-like hook shape. The tip portion 11ba may be formed to be wide in a direction D3 intersecting with the directions D1 and D2. In FIG. 2, directions D1, D2 and D3 are perpendicular to each other.

[0041] The driving device 11c is disposed on the mounting portion 11a and drives the rotating shaft 11bb of the holding portion 11b to rotate. In this embodiment, the direction of the rotating shaft 11bb of the holding portion 11b is perpendicular to the direction of the torsional rotation axis S6 of the link 12f and along the direction D3, but is not limited thereto, and may be, for example, a direction that obliquely intersects with the direction of the torsional rotation axis S6. The driving device 11c uses electric power as a power source and has a servo motor M7 (see FIG. 5) as an electric motor that drives them, but is not limited thereto. Furthermore, the driving device 11c has a rotation sensor E7 (see FIG. 5) that detects the amount of rotation (rotation angle) of the servo motor M7 or the holding portion 11b. The driving device 11c may have a current sensor (not shown) that detects the current value of the servo motor M7.

[0042] The force detector 11d is disposed between the attachment portion 11a and the link 12f, and detects a force acting between the attachment portion 11a and the link 12f. The force detector 11d can detect a force received by the end effector 11. The force detector 11d outputs a signal indicating a detected value of the force to the first control device 31. In this embodiment, the force detector 11d is a force sensor that detects six-axis forces including forces in three orthogonal axial directions and moments that are rotational forces around the three axes, but is not limited thereto. Note that the force detected by the force detector 11d is not limited to the above six-axis forces, and may be, for example, forces in some of the six axes.

[0043] The first control device 31 causes the drive device 11c to rotate the holding part 11b to change the orientation of the holding part 11b, and causes the robot arm 12 to move the end effector 11, thereby causing the end effector 11 to scrape and hold the floating objects S on the surface of the molten metal M in the first container Cm. Note that the end effector 11 may have any configuration as long as it is capable of moving the floating objects S.

[0044] As shown in FIG. 1, the imaging device 40 is disposed so as to capture at least an image of the opening Cma of the first housing body Cm. In this embodiment, the imaging device 40 includes, but is not limited to, a visible camera capable of capturing a moving image of a digital image, which is a visible image. The imaging device 40 performs an imaging operation according to, for example, a command from the first control device 31, and sends a signal of the captured image to the first control device 31. For example, the imaging device 40 may be disposed so as to project an image similar to an image of the opening Cma seen by the user P when the user P is positioned beside the robot arm 12. In this case, it is preferable that the imaging device 40 is disposed above the opening Cma and directed downward. However, a protective member for heat insulation may be disposed on the robot arm 12, and the protective member may hide a part of the opening Cma from the imaging device 40. For this reason, in FIG. 1, the imaging device 40 is disposed above the robot arm 12 between the opening Cma and the base 13, and directed toward the opening Cma diagonally downward.

[0045] Note that multiple imaging devices 40 may be arranged for one opening Cma. In an image captured by the imaging device 40 shown in Fig. 1, the image of the opening Cma viewed from diagonally above may be distorted. For this reason, an image capturing the image of the opening Cma viewed from the front may be generated by the first control device 31 or the like by processing multiple images captured by the multiple imaging devices 40.

[0046] The imaging device 40 may include a thermal imaging camera. For example, the thermal imaging camera may be an infrared camera that detects radiated infrared rays and generates a thermal image showing the distribution of the infrared rays. The imaging device 40 outputs a signal of the captured thermal image to the first control device 31.

[0047] The presentation device 50 presents images, sounds, and the like for operating the robot system 1 received from the control device 30 to the user P of the robot system 1. Examples of the presentation device 50 include, but are not limited to, a liquid crystal display and an organic or inorganic electroluminescence display (EL display). The presentation device 50 may include a speaker that emits sounds. For example, the presentation device 50 presents an image captured by the imaging device 40 and a processed image of the image to the user P who operates the operation device 20.

[0048] The input device 60 receives input of various information, data, commands, etc., and outputs the received information, etc. to the first control device 31. For example, the input device 60 may include known input means such as a lever, a button, a touch panel, a joystick, a motion capture, etc. For example, the input device 60 may be a teaching pendant, which is one of teaching devices, etc.

[0049] Fig. 3 is a perspective view showing an example of a configuration of the operating device 20 according to the embodiment. Fig. 4 is a perspective view showing an enlarged view of the operating unit 230 of Fig. 3. As shown in Fig. 3, the operating device 20 includes a support base 210, an operating arm 220, and an operating unit 230. The support base 210 is a base that supports the operating device 20 on a surface on which the operating device 20 is placed. The support base 210 supports the operating arm 220 at its upper portion.

[0050] The operating arm 220 receives an operation for commanding the position of the end effector 11. The operating arm 220 is configured as a multi-joint arm. The operating arm 220 includes links 221-223, joints JA1-JA3, and driving devices DRA1-DRA3. The joints JA1-JA3 are rotary joints. The number of joints may be other than three.

[0051] The joint JA1 connects a base end of the link 221 to the upper surface 211 of the support base 210 so as to be rotatable about a rotation axis SA1 extending in a direction perpendicular to the upper surface 211. The joint JA2 connects a base end of the link 222 to the tip end of the link 221 so as to be rotatable about a rotation axis SA2 extending in a direction perpendicular to the rotation axis SA1. The joint JA3 connects a base end of the link 223 to the tip end of the link 222 so as to be rotatable about a rotation axis SA3 extending in a direction parallel to the rotation axis SA2.

[0052] The joint JA1 allows the operating arm 220 to rotate around the rotation axis SA1. The joints JA2 and JA3 allow the operating arm 220 to bend around the rotation axes SA2 and SA3. The operating arm 220 can move the tip of the link 223 to various positions in three-dimensional space. For example, the position of the tip of the link 223 can be represented by an orientation determined by the amount of rotation (e.g., rotation angle) of the joint JA1, and an elevation angle and distance determined by the amount of rotation (rotation angle) of the joints JA2 and JA3.

[0053] The driving devices DRA1 to DRA3 are connected to the joints JA1 to JA3 so as to be able to transmit driving forces. The driving device DRA1 is disposed on the support base 210, and the driving devices DRA2 and DRA3 are disposed adjacent to the joint JA2. The driving devices DRA1 and DRA2 are connected to the joints JA1 and JA2, respectively, via a transmission mechanism (not shown) such as a reducer. The driving device DRA3 is connected to the joint JA3 via a transmission mechanism (not shown) such as a reducer, a belt, and a pulley. The driving devices DRA1 to DRA3 operate together with the joints JA1 to JA3, respectively. When the joints JA1 to JA3 are operated by the user P, the driving devices DRA1 to DRA3 are similarly operated, respectively, and can apply a rotational force to the joints JA1 to JA3 by generating a driving force. For example, the driving devices DRA1 to DRA3 can apply a reaction force to the user P against the operating force of the user P that moves the tip of the operating arm 220.

[0054] The operation unit 230 accepts an operation for commanding the posture of the end effector 11. As shown in FIG. 3 and FIG. 4, the operation unit 230 includes links 231-235 and driving devices DRA4-DRA6. The link 231 includes a grip 231a that the user P can hold, and an axis portion 231b that is connected to the driving device DRA4. The grip 231a protrudes outward at a position away from the axis portion 231b, for example, toward the opposite side of the support base 210 with respect to the operating arm 220. Furthermore, the link 231 includes an input element 231c on the grip 231a. The input element 231c accepts input of a command for operating the driving device 11c of the end effector 11. In this embodiment, the input element 231c includes two push buttons, but the configuration is not limited as long as it has the above function. When one of the push buttons is pressed to the ON state, it outputs a command to the drive unit 11c to rotate the holding unit 11b in one direction, and when it is released to the OFF state, it outputs a command to the drive unit 11c to stop the rotation. When the other push button is ON, it outputs a command to the drive unit 11c to rotate the holding unit 11b in the direction opposite to the one direction, and when it is OFF, it outputs a command to the drive unit 11c to stop the rotation.

[0055] Link 232 has connection parts 232a and 232b that are connected to links 231 and 234, respectively. The first connection part 232a fits into the cylindrical outer circumferential surface of shaft part 231b of link 231 from the outside, and can rotate on the outer circumferential surface of shaft part 231b around a rotation axis SB1. The rotation axis SB1 is also the axis of shaft part 231b.

[0056] The link 233 has connection parts 233a and 233b connected to the links 231 and 235, respectively. The first connection part 233a fits into the outer peripheral surface of the shaft part 231b of the link 231 from the outside, and can rotate around the rotation axis SB1 on the outer peripheral surface of the shaft part 231b. The first connection part 233a is arranged side by side with the first connection part 232a of the link 232 in the axial direction of the rotation axis SB1. The driving device DRA4 is fixed to the first connection part 233a. The links 231 to 233 can rotate relatively around the rotation axis SB1.

[0057] The link 234 has a first connection portion 234a connected to the second connection portion 232b of the link 232, and a second connection portion 234b connected to a tip end of the link 223 of the operating arm 220. The first connection portion 234a is connected to the second connection portion 232b so as to be rotatable about a rotation axis SB2. The second connection portion 234b is formed of a pulley, and is connected to the tip end of the link 223 so as to be rotatable about a rotation axis SB4. The direction of the rotation axis SB2 is a direction that obliquely intersects the directions of the rotation axes SB1 and SB4.

[0058] The link 235 has a first connection part 235a connected to the second connection part 233b of the link 233, and a second connection part 235b connected to the tip of the link 223 of the operating arm 220. The first connection part 235a is connected to the second connection part 233b so as to be rotatable around the rotation axis SB3. The second connection part 235b is formed of a pulley, and is connected to the tip of the link 223 so as to be rotatable around the rotation axis SB5. The direction of the rotation axis SB3 is a direction that obliquely intersects with the direction of the rotation axis SB1 and the direction of the rotation axis SB5. The rotation axis SB5 is parallel to the rotation axis SB4, and is coaxial with it in this embodiment. Furthermore, the rotation axis SB5 is also parallel to the rotation axis SA3 of the operating arm 220. For example, the direction of the rotation axis SB2 and the direction of the rotation axis SB3 intersect obliquely to form an obtuse angle having a vertex in a direction away from the tip end of the link 223 in the axial direction of the rotation axis SB5 and in a direction away from the second connection portions 232b and 233b in the axial direction of the rotation axis SB1.

[0059] Therefore, the grip 231a can move along a circumferential orbit in the circumferential direction CD1 around the rotation axis SB1. For example, the grip 231a can move in the yawing direction around the rotation axis SB1. At this time, the drive unit DRA4 rotates its drive shaft together with the link 231.

[0060] In addition, the grip 231a can move around the rotation axes SB4 and SB5 along a circumferential orbit in a circumferential direction CD2 that intersects with the circumferential direction CD1. For example, the grip 231a can move in a pitching direction around the rotation axes SB4 and SB5. At this time, the second connection parts 234b and 235b of the links 234 and 235 rotate in the same rotation direction by the same rotation amount, that is, rotate synchronously.

[0061] In addition, the grip 231a can rotate around the rotation axes SB2 and SB3 in the circumferential direction CD1 and the circumferential direction CD3 intersecting the circumferential direction CD2 to change its posture. For example, the grip 231a can rotate around the axis of the grip 231a. The grip 231a can move in the rolling direction around the rotation axes SB2 and SB3. At this time, the links 232 and 233 rotate the first connecting parts 232a and 233a in opposite directions to each other while rotating the second connecting parts 232b and 233b in the same direction. As a result, the second connecting parts 234b and 235b of the links 234 and 235 rotate in opposite directions to each other. Therefore, the difference between the phase of the rotation direction of the second connecting part 234b and the phase of the rotation direction of the second connecting part 235b increases or decreases.

[0062] The above-described operating unit 230 can move the grip 231a to various positions along the spherical surface, and can change the attitude of the grip 231a at the position. For example, the attitude of the grip 231a can be expressed by a yawing angle determined by the amount of rotation (rotation angle) around the rotation axis SB1, a pitching angle determined by the amount of rotation (rotation angle) around the rotation axes SB4 and SB5, and a rolling angle determined by the amount of rotation (rotation angle) around the rotation axes SB2 and SB3. In the operating unit 230, the grip 231a is supported by a pair of parallel links, links 232 and 234 and links 233 and 235. Although not limited thereto, in this embodiment, the central axis of the yawing rotation, the central axis of the pitching rotation, and the central axis of the rolling rotation intersect perpendicularly with each other at one point. This allows the grip 231a to move along a spherical surface centered on the point. Such a mechanism of the operating unit 230 constitutes a spherical parallel link mechanism.

[0063] The driving devices DRA4, DRA5, and DRA6 are connected to the links 231, 234, and 235, respectively, so as to be able to transmit driving forces. The driving device DRA4 is disposed on the link 233, and is connected to the shaft portion 231b of the link 231 via a transmission mechanism (not shown) such as a reducer. The driving devices DRA5 and DRA6 are supported by the link 221 of the operation arm 220. The driving devices DRA5 and DRA6 are connected to the second connection portions 234b and 235b of the links 234 and 235, respectively, via a transmission mechanism (not shown) such as a reducer, a pulley, and a belt. The driving devices DRA4 to DRA6 rotate the respective drive shafts together with the links 231, 234, and 235. When the links 231, 234, and 235 are operated by the user P via the grip 231a, the driving devices DRA4 to DRA6 are similarly operated to generate driving forces, thereby applying rotational forces to the links 231, 234, and 235. For example, the driving devices DRA4 to DRA6 can apply to the user P a reaction force against the operating force of the user P who moves and changes the posture of the grip 231a along the spherical surface. The driving devices DRA1 to DRA6 are examples of force applying devices.

[0064] The driving devices DRA1 to DRA6 are each powered by electricity and have servo motors MA1 to MA6 (see FIG. 5) as electric motors for driving them, but are not limited thereto. Furthermore, the driving devices DRA1 to DRA3 each have rotation sensors EA1 to EA3 (see FIG. 5) for detecting the rotation amount (rotation angle) of the joints JA1 to JA3. The driving devices DRA4 to DRA6 each have rotation sensors EA4 to EA6 (see FIG. 5) for detecting the rotation amount (rotation angle) of the links 231, 234, and 235. The driving devices DRA1 to DRA6 may each have a current sensor (not shown) for detecting the current value of the servo motors MA1 to MA6. The rotation sensors EA1 to EA6 and the current sensors each output a signal indicating the detection value to the second control device 32. The servo motors MA1 to MA6 are driven according to a command received from the second control device 32.

[0065] FIG. 5 is a block diagram showing an example of the configuration of the control device 30 and its periphery according to the embodiment. As shown in FIG. 5, the control device 30 includes a first control device 31 and a second control device 32. The first control device 31 controls the operation of the entire robot 10, and the second control device 32 controls the operation of the entire operation device 20. For example, the first control device 31 and the second control device 32 may be computer devices. The first control device 31 is communicably connected to the second control device 32, the imaging device 40, the presentation device 50, and the input device 60. The second control device 32 is communicably connected to the first control device 31 and the operation device 20. The communication format may be any communication format such as wired communication and wireless communication. The first control device 31 operates the robot 10 in response to a command received by the operation device 20 or the input device 60. The first control device 31 transmits information to the second control device 32 in order to give a reaction force corresponding to the force received by the end effector 11 to the grip 231a. The first control device 31 outputs an image captured by the imaging device 40 or a processed image of the image to the presentation device 50. The second control device 32 outputs information, commands, and the like accepted by the operation device 20 to the first control device 31. In addition, the second control device 32 controls the operations of the drive devices DRA1 to DRA6 of the operation device 20 based on the information, commands, and the like received from the first control device 31.

[0066] The first control device 31 is electrically connected to the servo motors M1 to M7 (indicated as "SM" in FIG. 5) of the robot 10 via the drive circuits C1 to C7, respectively. The drive circuits C1 to C7 adjust the current values ​​of the currents supplied to the servo motors M1 to M7, respectively, according to the command of the first control device 31. The first control device 31 is electrically connected to the rotation sensors E1 to E6 (indicated as "EN" in FIG. 5) of the robot 10. The second control device 32 is electrically connected to the servo motors MA1 to MA6 of the operating device 20 via the drive circuits CA1 to CA6, respectively. The drive circuits CA1 to CA6 adjust the current values ​​of the currents supplied to the servo motors MA1 to MA6, respectively, according to the command of the second control device 32. The second control device 32 is electrically connected to the rotation sensors EA1 to EA6 of the operating device 20. The first control device 31 and the drive circuits C1 to C7 constitute a first control unit 310, and the second control device 32 and the drive circuits CA1 to CA6 constitute a second control unit 320.

[0067] The first control device 31 is composed of a computing unit having a processor, memory, etc. The computing unit transmits and receives information, data, commands, etc. to and from other devices including the operation device 20. The computing unit inputs detection signals from various sensors and outputs control signals to each control target. The memory is composed of semiconductor memories such as volatile memory and non-volatile memory, hard disks, SSDs (Solid State Drives), and other storage devices. For example, the memory stores programs executed by the computing unit, various fixed data, etc.

[0068] The functions of the arithmetic unit may be realized by a computer system (not shown) including a processor such as a CPU (Central Processing Unit), a volatile memory such as a RAM (Random Access Memory), and a non-volatile memory such as a ROM (Read-Only Memory). A part or all of the functions of the arithmetic unit may be realized by the CPU using the RAM as a work area to execute a program recorded in the ROM. Note that a part or all of the functions of the arithmetic unit may be realized by the computer system, a dedicated hardware circuit such as an electronic circuit or an integrated circuit, or a combination of the computer system and the hardware circuit.

[0069] Such a first control device 31 may be configured, for example, by a microcontroller, an MPU (Micro Processing Unit), an LSI (Large Scale Integration), a system LSI, a PLC (Programmable Logic Controller), a logic circuit, or the like. The functions of the first control device 31 may be realized by being individually integrated into one chip, or may be realized by being integrated into one chip to include some or all of the functions. In addition, each circuit may be a general-purpose circuit or a dedicated circuit. As the LSI, an FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, a reconfigurable processor that can reconfigure the connection and / or setting of the circuit cells inside the LSI, or an ASIC (Application Specific Integrated Circuit) in which circuits of multiple functions are integrated into one for a specific purpose, or the like may be used.

[0070] Some or all of the functions of the second control device 32 may be realized by a computer system (not shown) consisting of a CPU, RAM, ROM, etc., or may be realized by a dedicated hardware circuit such as an electronic circuit or integrated circuit, or may be realized by a combination of the above-mentioned computer system and hardware circuit.

[0071] Each of the first control device 31 and the second control device 32 may execute each process under centralized control by a single computer device, or may execute each process under distributed control by cooperation of multiple computer devices. The first control device 31 and the second control device 32 may be included in a single computer device.

[0072] [Functional configuration of the first control device and the second control device] FIG. 6 is a block diagram showing an example of a functional configuration of the control device 30 according to the embodiment. As shown in FIG. 6, the first control device 31 includes an image processing unit 311, an information processing unit 312, operation command units 313 and 314, an operation control unit 315, and a storage unit 316 as functional components. The second control device 32 includes an input processing unit 321, an operation command unit 322, an operation command unit 323, and an operation control unit 324 as functional components. The functions of the above functional components except for the storage unit 316 are realized by a computer system and / or a hardware circuit. The storage unit 316 is realized by a memory. Not all of the above functional components are essential.

[0073] An input processing unit 321 of the second control device 32 receives and processes information, commands, etc. from the operating device 20, and outputs them to the first control device 31 and / or the operation command unit 322. For example, the input processing unit 321 detects information corresponding to the position, attitude, position movement speed, attitude movement speed, etc. of the grip 231a from the detection values ​​of the rotation sensors EA1 to EA6 of the operating device 20, and outputs it to the operation command unit 322. In addition, the input processing unit 321 detects information indicating drive, drive stop, and drive direction to be commanded to the drive device 11c of the end effector 11 from the ON and OFF signals of the two push buttons of the input element 231c, and outputs it to the operation command unit 322.

[0074] For example, the input processing unit 321 detects the azimuth, elevation angle, and distance of the grip 231a relative to a reference point (not shown) of the joint JA1 of the operating arm 220, as well as the speed of change thereof, from the detection values ​​of the rotation sensors EA1 to EA3, as position and orientation information corresponding to the position of the grip 231a and its moving speed. The input processing unit 321 detects the yawing angle, pitching angle, and rolling angle of the grip 231a, as well as the speed of change thereof, from the detection values ​​of the rotation sensors EA4 to EA5, as position and orientation information corresponding to the attitude of the grip 231a and its moving speed.

[0075] The operation command unit 322 generates an operation command for commanding the position, posture, moving speed of the position, moving speed of the posture, etc. of the end effector 11 based on the position and posture information detected by the input processing unit 321, and outputs the operation command to the first control device 31. The operation command is an example of a manual command. For example, the operation command unit 322 may convert the azimuth, elevation angle, and distance of the grip 231a relative to the reference point of the joint JA1 of the operation arm 220, as well as the change speeds thereof, into coordinate values ​​of a three-dimensional Cartesian coordinate system (hereinafter also referred to as the "operation coordinate system") having the above-mentioned reference point as the origin. The operation command unit 322 may convert the yawing angle, pitching angle, and rolling angle of the grip 231a, as well as the change speeds thereof, into rotation angles around the coordinate axes of a three-dimensional Cartesian coordinate system (hereinafter also referred to as the "tip coordinate system") having the center (not shown) of the spherical surface on which the grip 231a moves as the origin, and the change speeds thereof. The tip coordinate system is a coordinate system based on the tip of the link 223. The operation command unit 322 may generate an operation command including the converted command value. Also, the operation command unit 322 generates an operation command including commands for driving, stopping, and driving direction for the driving device 11c of the end effector 11 based on information on the input element 231c of the grip 231a detected by the input processing unit 321.

[0076] The position of the grip 231a and the speed of movement of the position correspond to the position of the end effector 11. The posture of the grip 231a and the speed of movement of the posture correspond to the posture of the end effector 11. For example, the operation coordinate system can be associated with the robot coordinate system, and the tip coordinate system can be associated with the mechanical interface coordinate system (hereinafter, also referred to as the "I / F coordinate system"). The robot coordinate system is a coordinate system that has a reference point (not shown) set on the base 13 of the robot arm 12 as its origin and has the robot 10 as its reference. The I / F coordinate system is a coordinate system that has a reference point (not shown) set on the mechanical interface surface of the link 12f of the robot arm 12 as its origin and has the mechanical interface surface as its reference. The three-dimensional positions of the end effector 11 and the grip 231a can be expressed using the robot coordinate system and the operation coordinate system, respectively. The three-dimensional postures of the end effector 11 and the grip 231a can be expressed using the I / F coordinate system and the tip coordinate system, respectively.

[0077] The operation command unit 323 receives a force command from the first control device 31, generates an operation command for applying a force to the grip 231a according to the force command, and outputs the operation command to the operation control unit 324. The operation command is a command for driving the driving devices DRA1 to DRA6 so as to apply a force to the grip 231a.

[0078] The force command is a command generated by the first control device 31 based on a detection value of the force detector 11d of the end effector 11. The force command is a command to apply to the grip 231a a force of a magnitude and direction corresponding to the magnitude and direction of the force detected by the force detector 11d.

[0079] For example, the force command may include a command value of the magnitude and direction of the force in the robot coordinate system and a command value of the magnitude and direction of the force in the I / F coordinate system. The motion command unit 323 detects target values ​​of the magnitude and direction of the force in the operation coordinate system and the tip coordinate system by performing calculation processing on the command values ​​of the magnitude and direction of the force in the robot coordinate system and the I / F coordinate system included in the force command. Furthermore, the motion command unit 323 calculates the rotation direction, rotation amount, rotation torque, etc. of the servo motors MA1 to MA3 of the driving devices DRA1 to DRA3 for generating the force of the target value in the operation coordinate system, and the rotation direction, rotation amount, rotation torque, etc. of the servo motors MA4 to MA6 of the driving devices DRA4 to DRA6 for generating the force of the target value in the tip coordinate system. The motion command unit 323 generates a motion command including a position command indicating the rotation direction and rotation amount of the servo motors MA1 to MA6 and a force command indicating the rotation torque of the servo motors MA1 to MA6. In generating the above commands, the operation command unit 323 may use detection values ​​of the rotation sensors EA1 to EA6 of the servo motors MA1 to MA6, current sensors (not shown), and the like, as feedback information.

[0080] The command values ​​of the magnitude and direction of the force in the robot coordinate system correspond to the magnitude and direction of the linear force received by the end effector 11. The target values ​​of the magnitude and direction of the force in the operation coordinate system correspond to the magnitude and direction of the linear force applied to the grip 231a. The command values ​​of the magnitude and direction of the force in the I / F coordinate system correspond to the rotational force received by the end effector 11, i.e., the magnitude and direction of the moment. The target values ​​of the magnitude and direction of the force in the tip coordinate system correspond to the magnitude and direction of the rotational force (moment) applied to the grip 231a.

[0081] The motion control unit 324 determines the current value to be supplied to each of the servo motors MA1 to MA6 so as to drive each of the servo motors MA1 to MA6 in accordance with the motion command received from the motion command unit 323, and controls the supply of current. Furthermore, in the above control, the motion control unit 324 may use detection values ​​of the rotation sensors EA1 to EA6 and current sensors (not shown) of the servo motors MA1 to MA6 as feedback information. In this manner, the motion control unit 324 performs servo control. As a result, the user P operating the grip 231a perceives a force corresponding to the force received by the end effector 11 as a reaction force against the operating force.

[0082] The image processing unit 311 of the first control device 31 receives captured image data from the imaging device 40, and outputs the image data to the presentation device 50 for display. The image processing unit 311 may apply image processing to the image data and output it to the presentation device 50. The image processing unit 311 is an example of an image processing device.

[0083] For example, the image processing unit 311 may perform image processing on the image data to visually or otherwise differentiate between the image of the molten metal M and the image of the floating matter S based on the luminance value, which is the pixel value of each pixel of the image data. For example, the image processing unit 311 may perform processing to change the color, brightness, shading, texture, or the like of the pixels. For example, the image processing unit 311 may perform different image processing on pixels with a luminance value equal to or greater than a threshold value and pixels with a luminance value less than the threshold value. For example, the above processing may be processing to make the pixels with a luminance value less than the threshold value more visually or otherwise noticeable than the pixels with a luminance value equal to or greater than the threshold value when the image of the molten metal M corresponds to pixels with a luminance value equal to or greater than a threshold value and the image of the floating matter S corresponds to pixels with a luminance value less than the threshold value.

[0084] A temperature sensor (not shown) for detecting the temperature of the surface of the molten metal M in the first container Cm may be provided in the work area WP. The temperature sensor is preferably capable of detecting the temperature of the surface of the molten metal M at a position away from the surface of the molten metal M, and may be, for example, an infrared sensor. If the imaging device 40 includes a thermal imaging camera, the temperature sensor may be replaced by the thermal imaging camera. The image processing unit 311 may receive temperature data indicating the temperature distribution of the surface of the molten metal M from the temperature sensor or thermal imaging camera, and perform image processing on the image data of the imaging device 40 using the temperature data. The image processing unit 311 may perform image processing for visually differentiating the molten metal M from the floating matter S based on the temperature distribution indicated in the temperature data.

[0085] For example, the image processing unit 311 may perform a process of generating image data representing a temperature distribution based on the temperature data, and superimposing the generated image data on the image data of the imaging device 40. Alternatively, the image processing unit 311 may perform a process of displaying the image data representing the temperature distribution and the image data of the imaging device 40 side by side. Alternatively, the image processing unit 311 may perform different image processing on pixels depicting an area with a temperature equal to or higher than a threshold value and pixels depicting an area with a temperature lower than the threshold value. For example, the above processing may be a process in which, when the image of the molten metal M corresponds to pixels with a temperature equal to or higher than a threshold value and the image of the floating matter S corresponds to pixels with a temperature lower than the threshold value, the pixel color, brightness, shading, texture, etc. are changed to make the pixels with a temperature lower than the threshold value more visually noticeable than the pixels with a temperature higher than the threshold value.

[0086] For example, if the imaging device 40 includes a stereo camera that captures an image capable of detecting the distance to a subject, the image processing unit 311 may generate a distance image and a three-dimensional image of the image captured by the imaging device 40 by detecting the distance to the subject that is displayed in each pixel.

[0087] The storage unit 316 can store, that is, memorize, various information, and enables reading of the stored information. For example, the storage unit 316 stores a threshold value used by the image processing unit 311 and a threshold value used by the second operation command unit 314 described later. The storage unit 316 may store image data captured by the imaging device 40 and image data processed by the image processing unit 311. The storage unit 316 may store a program for realizing each function of the first control device 31. The storage unit 316 may store automatic driving information, which is information used by the program when the robot 10 is made to operate automatically. The automatic driving information may be preset fixed data, teaching data, and the like.

[0088] The information processing unit 312 receives various information and commands from the second control device 32 and the input device 60, and outputs them to the corresponding functional components. For example, the information processing unit 312 receives an operation operation command from the second control device 32, and outputs them to the operation command units 313 and 314. The information processing unit 312 receives commands instructing the robot 10 to perform an automatic driving operation and commands instructing the execution of automatic driving from the input device 60, and outputs them to the first operation command unit 313. The above commands are examples of automatic commands.

[0089] The first operation command unit 313 generates an operation command for the robot 10 according to the operation command and the automatic command, and outputs the operation command to the operation control unit 315. For example, the first operation command unit 313 converts a command value in the operation coordinate system included in the operation command into a command value in the robot coordinate system. Note that the operable range of the grip 231a is different from the operable range of the end effector 11, and the scale is different between the movement amount of the grip 231a and the movement amount of the end effector 11. For this reason, the first operation command unit 313 may convert a command value in the operation coordinate system that is increased by a predetermined ratio into a command value in the robot coordinate system. Also, the first operation command unit 313 converts a command value in the tip coordinate system included in the operation command into a command value in the I / F coordinate system. At this time, the first operation command unit 313 may convert a command value in the tip coordinate system that is increased by a predetermined ratio into a command value in the I / F coordinate system.

[0090] The first operation command unit 313 calculates the rotation direction, rotation amount, rotation torque, etc. of the servo motors M1 to M6 of the driving devices DR1 to DR6 for moving the end effector 11 according to the command value in the robot coordinate system and the command value in the I / F coordinate system. The first operation command unit 313 also calculates the rotation direction, rotation amount, rotation torque, etc. of the servo motor M7 for driving the driving device 11c according to the command to the end effector 11 included in the manipulation operation command. The first operation command unit 313 also reads out automatic driving information corresponding to the automatic command from the storage unit 316, and calculates the rotation direction, rotation amount, rotation torque, etc. of the servo motors M1 to M7 for driving the driving devices DR1 to DR6 and 11c according to the automatic driving information. The first operation command unit 313 generates an operation command including the rotation direction, rotation amount, etc. of the servo motors M1 to M7 as a position command. When generating the above command, the first operation command unit 313 may use detection values ​​of the rotation sensors E1 to E7 of the servo motors M1 to M7 and the like as feedback information.

[0091] Here, the operation command includes at least one of a position command and a force command. The position command includes at least a position target value among target values ​​of the position, attitude, and velocity of the position and attitude of the control target, and includes all target values ​​in this embodiment. The position and its velocity may represent a position and velocity in a three-dimensional space, and the attitude and its velocity may represent an attitude and velocity in a three-dimensional space. Furthermore, the position command may include an execution time of the position command. Furthermore, the force command includes at least a target value of the force magnitude among target values ​​of the magnitude and direction of the force applied by the control target, and includes both target values ​​in this embodiment. The direction of the force may represent a direction in a three-dimensional space. The force command may include an execution time of the force command. In this specification and the claims, "position" may mean including at least a position in a three-dimensional space among a position in a three-dimensional space, a velocity of the position, an attitude, and a velocity of the attitude.

[0092] The operation control unit 315 determines the current value to be supplied to each of the servo motors M1 to M7 so as to drive each of the servo motors M1 to M7 in accordance with the operation command received from the first operation command unit 313, and controls the supply of the current. Furthermore, in the above control, the operation control unit 315 may use the detection values ​​of the rotation sensors E1 to E7 and current sensors (not shown) of the servo motors M1 to M7 and the force detector 11d of the end effector 11, etc., as feedback information. Such an operation control unit 315 performs servo control. As a result, for example, the robot arm 12 operates the end effector 11 in response to the movement of the grip 231a of the operating device 20.

[0093] The second operation command unit 314 receives operation information of the robot 10 from the operation control unit 315, generates an operation command for the operating device 20 using the operation information, and outputs it to the second control device 32. The operation command includes a force command, but may also include a position command.

[0094] The motion information of the robot 10 includes at least one of the motion data of the robot arm 12 and the end effector 11. The motion data of the robot arm 12 includes position data representing the positions of the interface surface of the link 12f and / or the joints JT1 to JT6 during motion. The motion data of the robot arm 12 may also include force data representing the force applied by the end effector 11, i.e., the interface surface, to an object. The force applied by the interface surface to an object is also the force received by the interface surface from the object. The motion data of the end effector 11 includes data representing the rotation position of the holder 11b. The position data may include a position in a three-dimensional space and a posture in the three-dimensional space. The force data may include the magnitude of the force and the direction of the force in the three-dimensional space. The position data and the force data may be time-series data associated with each position and the time when each force occurs.

[0095] Furthermore, the operation information may include, as information other than the operation data, vibration data, impact data, light data, sound data, temperature data, humidity data, pressure data such as atmospheric pressure, etc., generated on the interface surface of the link 12f. The operation information handled in this embodiment includes at least the operation data.

[0096] The second motion command unit 314 calculates the three-dimensional position and three-dimensional orientation of the interface surface of the link 12f in the robot coordinate system using the rotation amounts (rotation angles) of the joints JT1 to JT6 included in the motion data. Furthermore, the second motion command unit 314 determines the I / F coordinate system from the three-dimensional position and three-dimensional orientation of the interface surface. The second motion command unit 314 generates a motion command (force command) including force command values ​​in the robot coordinate system and the I / F coordinate system by processing the six-axis detection values ​​of the force detector 11d included in the motion data. For example, the second motion command unit 314 detects the magnitude and direction of a linear force in the robot coordinate system from the detection values ​​of the force detector 11d, and detects the magnitude and direction of a rotational force in the I / F coordinate system. The second motion command unit 314 determines the force command values ​​in the robot coordinate system and the I / F coordinate system by reducing at least one of the detection values ​​in the robot coordinate system and the detection values ​​in the I / F coordinate system to a predetermined ratio.

[0097] When reducing the command value in the robot coordinate system and the I / F coordinate system, the second operation command unit 314 may vary the ratio according to the magnitude of the command value. For example, when the magnitude of the force of the command value in the robot coordinate system is greater than 0 and equal to or less than a first threshold, the second operation command unit 314 may determine the command value by reducing the magnitude of the force of the command value by a first ratio, and when the magnitude of the force of the command value in the robot coordinate system is greater than the first threshold, the second operation command unit 314 may determine the command value by reducing the magnitude of the force of the command value by a second ratio. The second ratio may be smaller than the first ratio.

[0098] Furthermore, the second operation command unit 314 may determine, as a command value, a value obtained by reducing the magnitude of the force of the command value by a first ratio when the magnitude of the force of the command value in the I / F coordinate system is greater than 0 and equal to or less than a first threshold, and may determine, as a command value, a value obtained by reducing the magnitude of the force of the command value by a second ratio when the magnitude of the force of the command value in the I / F coordinate system is greater than the first threshold. Note that the first threshold, the first ratio, and the second ratio do not have to be the same values ​​between the I / F coordinate system and the robot coordinate system.

[0099] For example, the first threshold may be set to a value corresponding to the magnitude of the force when the holding part 11b of the end effector 11 contacts the floating object S. A force greater than 0 and equal to or less than the first threshold corresponds to the force when the holding part 11b contacts and does not contact the floating object S, and a force greater than the first threshold corresponds to the force when the holding part 11b moves, pulls up, holds, etc. the floating object S. Since the rate of reduction is smaller when the value is greater than 0 and equal to or less than the first threshold, the determined command value can more clearly indicate the presence or absence of contact between the holding part 11b and the floating object S. Since the rate of reduction is larger when the value is greater than the first threshold, the determined command value suppresses the application of excessive force to the grip 231a. Note that the first rate may be a rate at which the command value is increased.

[0100] Moreover, when the second threshold value is greater than the first threshold value, the second action command unit 314 may be configured to reduce the command value to a third rate less than the second rate, thereby preventing excessive force from being applied to the grip 231a.

[0101] [Robot system operation] The operation of the robot system 1 according to the embodiment will be described. Fig. 7 is a flow chart showing an example of the operation of the robot system 1 according to the embodiment in manual operation. As shown in Fig. 7, in step S1, a user P inputs a command to the input device 60 of the robot system 1 to start the operation of removing floating objects S using the robot 10 by manual operation, and the first control device 31 accepts the command.

[0102] Next, in step S2, the first control device 31 activates the imaging device 40 to capture an image of the surface of the molten metal M in the first container Cm. The first control device 31 causes the presentation device 50 to display the image of the molten metal M received from the imaging device 40.

[0103] Next, in step S3, the user P grasps the grip 231a of the operating device 20 while visually checking the image on the presentation device 50, and moves the grip 231a to an intended direction and posture. For example, the user P operates the grip 231a with the intention of the following action. That is, the robot arm 12 scrapes up the floating objects S on the surface of the molten metal M with the holding portion 11b of the end effector 11, then penetrates the floating objects S and rotates them about the axis, and further moves the holding portion 11b up and moves it, thereby moving and throwing the floating objects S entangled and held by the holding portion 11b into the second container Cs.

[0104] Next, in step S4, the second control device 32 generates an operation action command based on the detection values ​​of the rotation sensors EA1 to EA6 of the driving devices DRA1 to DRA6 that operate in conjunction with the movement of the grip 231a, and outputs the operation action command to the first control device 31.

[0105] Next, in step S5, the first control device 31 processes the manipulation operation command to detect command values ​​for the position and posture of the end effector 11. Furthermore, the first control device 31 generates operation commands including command values ​​for the operations of the servo motors M1-M6 of the drive devices DR1-DR6 to operate the end effector 11 in accordance with the command values.

[0106] Next, in step S6, the first control device 31 acquires operation information of the robot arm 12. Furthermore, the first control device 31 uses the operation information as feedback information to operate the servo motors M1 to M6 according to the operation commands. That is, the first control device 31 operates each of the joints JT1 to JT6 of the robot arm 12.

[0107] Next, in step S7, the first control device 31 generates a force command (operation command) for the operating device 20 using the operation information of the robot arm 12, and outputs it to the second control device 32. The force command is a command for applying a force corresponding to a detection value of the force detector 11d of the end effector 11 to the grip 231a.

[0108] Next, in step S8, the second control device 32 generates and outputs operation commands to the drive devices DRA1 to DRA6 for applying forces to the grip 231a in accordance with the force command.

[0109] Next, in step S9, the second control device 32 acquires operation information of the driving devices DRA1 to DRA6. Furthermore, the second control device 32 uses the operation information as feedback information to operate the servo motors MA1 to MA6 according to the operation command. In other words, the second control device 32 causes the servo motors MA1 to MA6 to generate a reaction force against the operating force applied to the grip 231a.

[0110] Next, in step S10, the second control device 32 determines whether or not a command to end the work has been input into the input device 60 by the user P, and if a command has been input (Yes in step S10), ends the series of processes, and if no command has been input (No in step S10), returns to step S3.

[0111] By performing steps S1 to S10, the user P can operate the grip 231a to cause the robot 10 to remove the floating object S while perceiving a force corresponding to the force that the end effector 11 receives from the floating object S through the grip 231a.

[0112] (Modification) The robot system 1 according to the embodiment includes one robot 10, and the robot 10 is connected to the operation device 20 via the control device 30. However, the present invention is not limited to this. The robot system 1 may include multiple robots 10, and one of the multiple robots 10 may be selectively connected to the operation device 20 via the control device 30.

[0113] Fig. 8 is a diagram showing an example of the configuration of a robot system 1A according to a modified example. As shown in Fig. 8, the robot system 1A includes ten robots 10, one operation device 20, one control device 30, ten imaging devices 40, one presentation device 50, one input device 60, and a relay device 70. The robots 10 and the imaging devices 40 are each disposed adjacent to one of the ten first housing units Cm disposed in the work area WP.

[0114] The relay device 70 connects the 10 robots 10 and the imaging devices 40 of the 10 to the first control device 31. The relay device 70 communicatively connects one robot 10 to the first control device 31 according to a command from the first control device 31. The relay device 70 communicatively connects one imaging device 40 to the first control device 31 according to a command from the first control device 31. The robots 10 and imaging devices 40 adjacent to the first housing body Cm may be associated with each other, and the relay device 70 may communicatively connect the associated one robot and one imaging device 40 to the first control device 31 according to a command from the first control device 31. The relay device 70 may include a circuit capable of switching communication connections, or may include the circuit and a computer system.

[0115] When the user P determines the first container Cm from which the floating object S is to be removed from among the 10 first containers Cm, the user P inputs a command to the input device 60 to designate the robot 10 corresponding to the first container Cm. When the first control device 31 receives the command from the input device 60, the first control device 31 outputs a command to the relay device 70 to connect the robot 10 and the imaging device 40 corresponding to the robot 10 to the first control device 31, and the relay device 70 executes the connection. As a result, the robot 10 is connected to the operation device 20 via the relay device 70, the first control device 31, and the second control device 32, and the imaging device 40 is connected to the presentation device 50 via the first control device 31. Thus, the user P can select any robot 10 from the 10 robots 10 and operate it using the operation device 20. The input device 60 is an example of a designation device.

[0116] The robot system 1A may further include a plurality of operation devices 20, a plurality of sets of control devices 31 and 32, a plurality of presentation devices 50, and a plurality of input devices 60. The relay device 70 may be configured to connect the robot 10 and the imaging device 40 corresponding to the robot 10 to the first control device 31 specified in an arbitrary input device 60 according to a command specifying the robot 10 input to the arbitrary input device 60. This enables connection between an arbitrary robot 10 and an arbitrary operation device 20.

[0117] Also, the imaging device 40 may not be provided for each robot 10, and may be configured so that one imaging device 40 can image two or more robots 10. For example, the imaging device 40 may be installed on a gimbal or the like, and the imaging direction may be changed. The first control device 31 may control the gimbal or the like to cause the imaging device 40 to image the robot to be operated.

[0118] (Other embodiments) Although the examples of the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-mentioned embodiments. In other words, various modifications and improvements are possible within the scope of the present disclosure. For example, the scope of the present disclosure also includes various modifications of the embodiments and forms constructed by combining components in different embodiments.

[0119] For example, in the embodiment, the first control device 31 is controlled so that the robot 10 manually performs all of the series of operations of moving and depositing the floating objects S on the surface of the molten metal M into the second container Cs, but this is not limiting. For example, the first control device 31 may be configured to automatically perform some or all of the series of operations described above.

[0120] For example, since the state of the floating objects S on the surface of the molten metal M is not constant, it is effective for the end effector 11 to scrape up and hold the floating objects S in a flexible manner according to the judgment of the user P, rather than a predetermined, fixed operation. The first control device 31 may be configured to manually operate the robot 10 to perform the above-mentioned flexible operation, and to automatically operate the robot 10 to perform a routine operation, such as the operation of moving the end effector 11 holding the floating objects S to the second container Cs.

[0121] The first control device 31 may be configured to automatically drive the robot 10 to perform the above-mentioned flexible operation or the entire series of operations. In this case, the first control device 31 may be configured to use AI (Artificial Intelligence) for the automatic operation. The first control device 31 may be configured to use AI for the automatic operation of regular operations. For example, since the shape, size, and weight of the floating object S held by the end effector 11 are not constant, flexible operation by AI corresponding to the state of the held floating object S is effective.

[0122] For example, the AI ​​may include a learning model that performs machine learning. For example, the learning model may include a neural network. The learning data of the learning model may include data obtained during a manual operation in which the robot 10 operates in a manual operation, and / or data obtained during a corrective operation in which the robot 10 operates while receiving corrections to its operation according to a manual operation input to the operation device 20 during automatic operation, or data obtained during the manual operation and / or corrective operation of various other robots. The learning model may be a model that performs machine learning using teacher data.

[0123] For example, the learning model may use image data of the surface of the molten metal M captured by the imaging device 40 or image processing data thereof as input data, and information corresponding to an operation command for the robot 10 as output data. The learning model may use the operation data of the robot 10 as teacher data. The learning model may be configured to perform machine learning by adjusting a neural network so that output data, which uses image data of the surface of the molten metal M captured immediately before the robot 10 performs the operation of the operation data or image processing data thereof as input data, matches the teacher data or minimizes an error with the teacher data. The image processing may be image processing for identifying floating objects S, and may be, for example, image processing exemplified for the image processing unit 311, etc.

[0124] This allows the learning model to machine-learn various operations performed by various users as a result of visually checking the image data of the surface of the molten metal M, and to output appropriate output data that reflects the user's skilled operations. Based on the image data of the surface of the molten metal M, the first control device 31 can cause the robot 10 to perform an action appropriate to the state of the floating matter S shown in the image data.

[0125] Alternatively, the learning model may use image data of the surface of the molten metal M captured by the imaging device 40 or image processing data thereof and the detection value of the force detector 11d as input data, and information corresponding to an operation command for the robot 10 as output data. The learning model may use operation data of the robot 10 as teacher data. The learning model may be configured to perform machine learning so as to match the output data, which is output using image data of the surface of the molten metal M captured immediately before the robot 10 performs the operation of the operation data or image processing data thereof and the detection value of the force detector 11d immediately before the operation of the operation data as input data, with the teacher data or minimize the error with the teacher data.

[0126] This allows the learning model to machine-learn various operations performed by various users as a result of visually viewing image data of the surface of the molten metal M and perceiving feedback of the force received by the end effector 11 via the grip 231a, and to output appropriate output data that reflects the skilled and delicate operations of the users. Based on the image data of the surface of the molten metal M and the force received by the end effector 11, the first control device 31 can cause the robot 10 to perform an operation appropriate to the state of the floating matter S and the state of the end effector 11 shown in the image data. [Explanation of symbols]

[0127] 1,1A Robot System 10. Robot 11 End effector 11d Force detector 12 Robot Arm 20 Operating device 30 Control device 31 First control device 32 Second control device 40 Imaging device 50 Presentation device 311 Image processing unit (image processing device)

Claims

1. a robot having a robot arm and an end effector at a tip of the robot arm, the robot arm being disposed adjacent to a container that contains a liquid material; an operation input device that is disposed away from the housing and that receives an input of a remote operation to the robot and outputs a command for operating the robot in response to the remote operation; an imaging device for imaging a surface of the liquid material; a presentation device that displays an image captured by the imaging device; A control device for controlling an operation of the robot, The end effector has a configuration capable of moving a floating object on a surface of the liquid object, The control device is configured to cause the robot arm to move the end effector so as to move the floating object from the container to the outside in accordance with the command output from the operation input device. Robot system.

2. an image processing device that processes an image captured by the imaging device; the image processing device is configured to perform image processing for differentiating the liquid matter from the floating matter, The presentation device displays the image processed by the imaging device. The robot system of claim 1 .

3. The image processing device performs image processing to differentiate between the liquid matter and the floating matter based on the luminance values ​​of pixels included in the image. The robot system according to claim 2 .

4. Further comprising a sensor for detecting a temperature of the surface of the liquid material, The image processing device performs image processing to distinguish between the liquid and the floating matter based on the temperature distribution on the surface of the liquid detected by the sensor. The robot system according to claim 2 or 3.

5. the command includes a manual command for operating the robot in response to a user's operation input to the operation input device for manual operation of the robot, The control device causes the robot to execute at least a part of a series of operations of the robot for moving the floating objects from the container to the outside in accordance with the manual command. The robot system according to any one of claims 1 to 4.

6. the command includes an automatic command for operating the robot in response to an input to the operation input device for automatic operation of the robot, The control device causes the robot to execute at least a part of a series of operations of the robot for moving the floating objects from the container to the outside in accordance with the automatic command. The robot system according to any one of claims 1 to 5.

7. Further comprising a force detector for detecting a force acting on the end effector, the operation input device includes an operation unit that receives a force from a user's hand in order to manually operate the robot, and a force imparting device that imparts a force to the operation unit, The control device receives a detection result from the force detector, and outputs to the operation input device a command to apply a force corresponding to the force detected by the force detector to the operation unit. The robot system according to any one of claims 1 to 6.

8. the operation input device includes an operation unit having a grip that can be held by a human hand in order to manually operate the robot, and an operation arm that movably supports the operation unit and is bendable and rotatable; the operating unit supports the grip rotatably around three axes so that the grip moves along a spherical surface relative to the operating arm and changes its position; The operation input device outputs information corresponding to the amount of rotation of the grip about the three axes and the amount of bending and turning of the operation arm as the command for commanding the position and attitude of the end effector. The robot system according to any one of claims 1 to 7.

9. A plurality of the containers; A plurality of the robots are disposed adjacent to each of the plurality of containers; a designation device that receives an input designating a robot to be connected to the operation input device from among the plurality of robots, and outputs information on a designated robot that is the designated robot; The control device includes: configured to control the movement of the plurality of robots; Operate the designated robot in accordance with the command output from the operation input device based on the information of the designated robot output from the designation device. The robot system according to any one of claims 1 to 8.

10. a plurality of the imaging devices for imaging the surfaces of the liquid objects in the plurality of containers, The control device includes: configured to control operation of the plurality of imaging devices; Based on the information of the designated robot, the imaging device capable of imaging the container adjacent to the designated robot is caused to image the surface of the liquid material in the container. The robot system of claim 9.

Images