Robot system and robot control method

Abstract

To provide a robot system that effectively removes floating matter on the surface of a liquid object.SOLUTION: A robot system includes: a robot which has a robot arm and an end effector and is arranged in adjacency to a storage body; an operation device which is arranged away from the storage body, receives input of remote control to the robot, and outputs a command for operating the robot in accordance with the remote control; an imaging device for imaging the surface of liquid object of the storage body; and a presentation device for displaying an image captured by the imaging device; and a control device. The end effector has a structure capable of moving floating matter on the surface of the liquid object. The control device is configured to allow the robot arm to move the end effector so as to move the floating matter outside the storage body according to the command.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a robot system.

Background Art

[0002] Conventionally, robots have been used to assist or replace human work. In particular, assistance and replacement of work by robots in harsh working environments are useful. For example, Patent Document 1 discloses a removing device that removes impurities called dross floating on the surface of a high-temperature molten metal plating bath. This removing device includes a robot, and the robot automatically performs collection of dross, lifting from the plating bath, and input to a collection box in a predetermined program.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In Patent Document 1, the robot performs each operation related to dross removal in a predetermined order and time according to a program. However, the distribution of dross on the surface of the plating bath is neither uniform nor constant, and for example, it also varies depending on the object to be plated. Therefore, it is difficult to efficiently remove dross with the robot of Patent Document 1 that automatically repeats the same operation.

[0005] An object of the present disclosure is to provide a robot system that effectively removes floating substances on the surface of a liquid material.

Means for Solving the Problems

[0006] To achieve the above objective, a robot system according to one aspect of the present disclosure comprises a robot having a robot arm and an end effector at the tip of the robot arm, and positioned adjacent to a container for containing a liquid; an operation input device positioned away from the container, which receives remote operation input to the robot and outputs commands to operate the robot in response to the remote operation; an imaging device for imaging the surface of the liquid; a display device for displaying the image captured by the imaging device; and a control device for controlling the operation of the robot, wherein the end effector is configured to move floating matter on the surface of the liquid; and the control device is configured to move the end effector to the robot arm in accordance with the commands output from the operation input device, so as to move the floating matter out of the container.

[0007] According to this disclosure, it becomes possible to effectively remove suspended particles from the surface of a liquid. [Brief explanation of the drawing]

[0008] [Figure 1] A diagram showing an example of the configuration of a robot system according to an embodiment. [Figure 2] Side view showing an example of the configuration of a robot according to the embodiment. [Figure 3] Perspective view showing an example of the configuration of the operating device according to the embodiment. [Figure 4] A perspective view showing an enlarged view of the control panel in Figure 3. [Figure 5] Block diagram showing an example of the configuration of a control device and its surroundings according to an embodiment. [Figure 6] Block diagram showing an example of the functional configuration of the control device according to the embodiment. [Figure 7] A flowchart illustrating an example of the operation of the robot system according to the embodiment during manual operation. [Figure 8] A diagram showing an example of the configuration of robot system 1A in a modified form. [Modes for carrying out the invention]

[0009] First, examples of each aspect of the present disclosure will be described. A robot system according to one aspect of the present disclosure comprises a robot having a robot arm and an end effector at the tip of the robot arm, and positioned adjacent to a container for containing a liquid; an operation input device positioned away from the container, which receives remote operation input to the robot and outputs commands to operate the robot in response to the remote operation; an imaging device for imaging the surface of the liquid; a display device for displaying the image captured by the imaging device; and a control device for controlling the operation of the robot, wherein the end effector is configured to move floating matter on the surface of the liquid, and the control device is configured to move the end effector to the robot arm in accordance with the commands output from the operation input device, so as to move the floating matter out of the container.

[0010] According to the above embodiment, the user, at a remote location away from the robot, can view an image displayed on a display device to recognize the position and state of suspended matter on the surface of a liquid, input an appropriate command to the operation input device according to this recognition result, and cause the robot to move the suspended matter from the containment to the outside. For example, if the suspended matter is an impurity such as slag generated in molten metal during the ironmaking, steelmaking, or casting process, it is difficult to collect the slag using a collection net or the like because the molten metal is at a much higher temperature than the molten metal plating bath. For example, impurities such as slag are collected by being entangled using a rod-shaped object. A robot whose operation is controlled according to the state of the impurities based on the user's judgment can effectively collect various impurities. Therefore, it becomes possible to effectively remove suspended matter from the surface of a liquid.

[0011] A robot system according to one aspect of the present disclosure further comprises an image processing device for processing an image captured by the imaging device, wherein the image processing device is configured to perform image processing to differentiate the liquid from the suspended matter, and the display device may display the image processed by the imaging device.

[0012] According to the above embodiment, the user can easily visually confirm the position and state of floating particles on the surface of a liquid in the image displayed on the display device. Therefore, the user can input appropriate commands to the operation input device according to the state of the floating particles.

[0013] In a robot system according to one aspect of this disclosure, the image processing device may perform image processing to differentiate the liquid substance from the floating substance based on the brightness values ​​of the pixels included in the image.

[0014] According to the above embodiment, for example, in the case of a liquid with a high luminance value, or when the luminance value of the liquid and the luminance value of the floating object are close, it is difficult to distinguish between the liquid and the floating object in the image. However, the processed image, which has undergone the above image processing based on luminance values, enables easy visual distinction between the liquid and the floating object. For example, the image processing device may differentiate between the liquid and the floating object by performing different image processing on pixels with luminance values ​​above a threshold and pixels with luminance values ​​below a threshold. For example, the image processing may be a process that changes the color, brightness, density, and texture of the pixels.

[0015] A robot system according to one aspect of the present disclosure further comprises a sensor for detecting the surface temperature of the liquid, and the image processing device may perform image processing to differentiate the liquid from the suspended matter based on the temperature distribution of the surface of the liquid detected by the sensor.

[0016] According to the above embodiment, for example, if there is a difference in temperature between the liquid and the suspended matter, the processed image, which has undergone the above image processing based on the temperature distribution, allows for easy visual distinction between the liquid and the suspended matter. For example, the image processing device may differentiate between the liquid and the suspended matter by performing different image processing on images in various temperature ranges for each temperature range. For example, the image processing may be a process that changes the color, brightness, density, and texture of the pixels.

[0017] A robot system according to one aspect of the present disclosure further includes a force detector that detects a force acting on the end effector, the operation input device includes an operation unit that receives a force from a user's hand to manually operate the robot, and a force applying device that applies a force to the operation unit, and the control device may receive a detection result from the force detector and output a command to the operation input device to apply an applied force corresponding to the force detected by the force detector to the operation unit.

[0018] According to the above aspect, the user can perceive the state of the force received by the end effector through the applied force applied to the operation unit. For example, the user can perceive whether the end effector is in contact with a floating object, whether the end effector is holding a floating object, and the weight of the floating object held by the end effector, etc. Therefore, effective operation of the robot by the user becomes possible.

[0019] In a robot system according to one aspect of the present disclosure, the control device outputs a command to apply the applied force obtained by reducing the detected output detected by the force detector to the operation unit, and when the detected output is greater than 0 and less than or equal to a first threshold, determines the force obtained by reducing the detected output by a first ratio as the applied force, and when the detected output is greater than the first threshold, determines the force obtained by reducing the detected output by a second ratio as the applied force, and the second ratio may be smaller than the first ratio.

[0020] According to the above aspect, a force below the first threshold acting on the end effector is reduced at a larger first ratio and transmitted to the user via the operation unit. Thereby, the user can also perceive a relatively small force acting on the end effector. For example, the user can perceive whether the end effector is immersed in a liquid substance and whether the end effector is in contact with a floating object, etc.

[0021] In a robot system according to one aspect of the present disclosure, the operation input device comprises an operating unit having a grip that can be grasped by a person's hand for manually operating the robot, and an operating arm that movably supports the operating unit and is bendable and rotatable, wherein the operating unit supports the grip so as to be rotatable about three axes relative to the operating arm such that the grip moves along a spherical surface and changes its orientation, and the operation input device may output information corresponding to the amount of rotation of the grip about the three axes and the amount of bending and rotatability of the operating arm as commands for commanding the position and orientation of the end effector.

[0022] According to the above embodiment, the amount of rotation of the grip around the three axes can be associated with the posture of the end effector. The amount of bending and rotation of the operating arm can be associated with the position of the end effector. As a result, the user can move the robot to a desired posture by inputting an operation to rotate the grip and change its posture, and can move the robot to a desired position of the end effector by inputting an operation to move the grip together with the operating unit. Furthermore, since the grip is configured to move along a spherical surface and change its posture relative to the operating arm, the user can easily control the posture of the grip and perceive the posture of the grip.

[0023] A robot system according to one aspect of the present disclosure further comprises a plurality of housings, a plurality of robots arranged adjacent to each of the plurality of housings, and a designation device that receives an input to designate a robot from among the plurality of robots to be connected to the operation input device, and outputs information of the designated robot, wherein the control device is configured to control the operation of the plurality of robots, and may operate the designated robot according to the command output from the operation input device based on the information of the designated robot output from the designation device.

[0024] According to the above embodiment, a user can operate any of multiple robots using a single control input device. For example, it is possible to remove floating objects from each containment in a predetermined order or in accordance with the amount of floating objects generated in the containment. Furthermore, since multiple robots do not operate simultaneously, the occurrence of interference such as contact between robots is suppressed.

[0025] A robot system according to one aspect of the present disclosure further comprises a plurality of imaging devices for imaging the surface of the liquid in each of the plurality of containers, the control device is configured to control the operation of the plurality of imaging devices, and based on information of the designated robot, the imaging device capable of imaging a container adjacent to the designated robot may be made to image the surface of the liquid in the container.

[0026] According to the above embodiment, the user can visually view images of the liquid matter in each container captured by multiple imaging devices. Based on the results of visualizing the liquid matter in each container, the user can determine which containers are to be targeted for removal of floating matter and have a robot placed in that container remove the floating matter.

[0027] (Embodiment) Embodiments of this disclosure will be described below with reference to the drawings. The embodiments described below are either comprehensive or specific examples. Furthermore, among the components in the embodiments described below, those not described in the independent claim indicating the highest-level concept will be described as optional components. Also, the figures in the accompanying drawings are schematic and not necessarily strictly illustrative. Furthermore, in each figure, substantially identical components are denoted by the same reference numerals, and redundant explanations may be omitted or simplified. In addition, in this specification and the claims, "apparatus" may mean not only a single apparatus but also a system consisting of multiple apparatuses.

[0028] [Robot System Configuration] The configuration of the robot system 1 according to this embodiment will be described. Figure 1 is a diagram showing an example of the configuration of the robot system 1 according to this embodiment. As shown in Figure 1, the robot system 1 comprises a robot 10, an operating device 20, a control device 30, an imaging device 40, a display device 50, and an input device 60. The control device 30 includes a first control device 31 and a second control device 32. In this embodiment, the robot system 1 is said to perform the task of removing floating matter S floating on the surface of molten metal M, which is molten metal, in ironmaking processes, steelmaking processes, or casting processes, etc., and the following explanation will be given. An example of floating matter S is impurities such as slag that are generated in the molten metal during the molten metal processing process.

[0029] In this embodiment, the robot system 1 is a system that utilizes a master-slave robot 10. The operating device 20 constitutes the master machine, and the robot 10 constitutes the slave machine. The operating device 20 is located remotely away from the robot 10, and the robot 10 is remotely controlled by the operating device 20. For example, the robot 10 and the imaging device 40 may be located in a work area WP where a first containment Cm, such as a melting furnace and a refining furnace that contain molten metal M, is located, while the operating device 20, control device 30, display device 50, and input device 60 may be located in an operating area OP, such as an operating room, where the influence of the heat of the molten metal M in the first containment Cm is reduced or blocked. The operating area OP may be a location isolated from the work area WP.

[0030] The robot system 1 is configured to allow the robot 10 to perform at least one of the following: manual operation, automatic operation, or a combination of manual and automatic operation. While not limited to this, in this embodiment, the robot system 1 is configured to allow the robot 10 to perform manual operation and a combination of manual and automatic operation. The following description will focus on a configuration in which the robot 10 operates manually.

[0031] In manual operation, the user P, such as an operator, operates the control device 20 to input commands. The control device 20 outputs manual commands to the control device 30 to operate the robot 10 in response to the operations of the user P input to the control device 20. The control device 30 then causes the robot 10 to perform the actions corresponding to the above operations. The control device 20 is an example of an operation input device.

[0032] In autonomous driving, user P provides input for the autonomous driving of robot 10 to input device 60, input device 60 outputs an automatic command to control device 30 to operate robot 10 in response to the input, and control device 30 automatically operates robot 10 according to predetermined information. The predetermined information may include information such as the position, posture, state, and order of each part of robot 10. For example, the predetermined information may be pre-set fixed data and teaching data acquired through teaching robot 10. Input device 60 is an example of an operation input device.

[0033] In a combination of manual and automatic operation, the control device 30 causes the robot 10 to perform actions in automatic operation and actions in manual operation in sequence. For example, a series of actions performed by the robot 10 for a task may include actions in automatic operation and actions in manual operation. For example, the switch from automatic operation to manual operation may be performed automatically by the control device 30 in response to the progress of the series of actions. The switch from manual operation to automatic operation may also be performed by the control device 30 in response to commands input by the user P to the operating device 20 and the input device 60, etc.

[0034] In this embodiment, robot 10 is an industrial robot. Robot 10 is fixedly positioned on base 13, but may be positioned on a movable device such as a conveying device and configured to be movable. In this embodiment, the base 13 of robot 10 is positioned adjacent to the opening Cma at the top of the first containment Cm on the work floor WF in the work area WP. A second containment Cs for collecting floating objects S is also positioned within the operating range of robot 10. In other words, robot 10 is positioned adjacent to the second containment Cs. The second containment Cs may be positioned directly on the work floor WF, or it may be positioned on a conveying device such as a belt conveyor on the work floor WF.

[0035] The robot 10 comprises an end effector 11 that applies an action to the object to be processed, and a robot arm 12 that moves the end effector 11 to perform the action. In this embodiment, the end effector 11 is configured to be able to hold floating matter S on the surface of the molten metal M by scraping it and entangling the floating matter S, and the following explanation will be given accordingly.

[0036] The robot arm 12 is not particularly limited as long as it has a configuration that allows it to change the position and / or orientation of the end effector 11 at its tip, but in this embodiment it is a vertical articulated robot arm. The robot arm 12 may be configured as, for example, a horizontal articulated robot arm, a polar coordinate robot arm, a cylindrical coordinate robot arm, a rectangular coordinate robot arm, or any other type of robot arm.

[0037] Figure 2 is a side view showing an example of the configuration of a robot 10 according to an embodiment. As shown in Figure 2, the robot arm 12 is fixed to a base 13. The robot arm 12 comprises links 12a to 12f arranged sequentially from its base to its tip, joints JT1 to JT6 that sequentially connect links 12a to 12f so as to be rotatable, and drive units DR1 to DR6 that rotationally drive each of the joints JT1 to JT6. Joints JT1 to JT6 are rotary joints. Link 12a is attached to the base 13 via joint JT1. The tip of link 12f constitutes a mechanical interface and is connected to an end effector 11. The operation of drive units DR1 to DR6 is controlled by a control device 30. Each of the drive units DR1 to DR6 has a servo motor M1 to M6 (see Figure 5) as an electric motor that uses electricity as a power source to drive them, but is not limited to this. Furthermore, each of the drive units DR1 to DR6 has rotation sensors E1 to E6 (see Figure 5), such as encoders, that detect the amount of rotation (e.g., rotation angle) of the servo motors M1 to M6 or joints JT1 to JT6. Each of the drive units DR1 to DR6 may also have a current sensor (not shown) that detects the current value of the servo motors MA1 to MA6. Note that the number of joints in the robot arm 12 is not limited to six, but may be seven or more, or five or less.

[0038] For example, joint JT1 connects the base end of link 12a to the upper surface 13a of the base 13 so as to be rotatable around a vertical axis of rotation perpendicular to the upper surface 13a. Joint JT2 connects the base end of link 12b to the tip of link 12a so as to be rotatable around a horizontal axis of rotation along the upper surface 13a. Joint JT3 connects the base end of link 12c to the tip of link 12b so as to be rotatable around a horizontal axis of rotation. Joint JT4 connects the base end of link 12d to the tip of link 12c so as to be rotatable around an axis of rotation that is the longitudinal axis of link 12c. Joint JT5 connects the base end of link 12e to the tip of link 12d so as to be rotatable around an axis of rotation perpendicular to the axis of rotation of link 12d. Joint JT6 connects the base end of link 12e to the tip end of link 12f so that it can rotate around a torsional axis.

[0039] The robot arm 12 described above can freely move the position of link 12f to any three-dimensional position within its range of motion, and can freely move the orientation of link 12f to any three-dimensional orientation. The three-dimensional position is the position in three-dimensional space and can be represented, for example, by the coordinate values ​​of three coordinate axes in a three-dimensional Cartesian coordinate system. The three-dimensional orientation is the orientation in three-dimensional space and can be represented, for example, by the rotation angle or Euler angle around three coordinate axes in a three-dimensional Cartesian coordinate system.

[0040] The end effector 11 includes a mounting portion 11a, a holding portion 11b, a drive device 11c, and a force sensor 11d. The mounting portion 11a is configured to connect to the mechanical interface of the link 12f, and the end effector 11 is attached to the link 12f. The holding portion 11b is composed of a columnar member, and the base end of the holding portion 11b is rotatably connected to the mounting portion 11a. Therefore, the holding portion 11b rotates around its base end. The holding portion 11b has a tip portion 11ba at its tip that extends in a direction D2 intersecting the direction D1 in which the holding portion 11b extends. The length of the tip portion 11ba in direction D2 is significantly shorter than the length of the holding portion 11b in direction D1. For example, the shape of the tip portion 11ba may be plate-shaped, rod-shaped, or wedge-shaped hook-shaped. The tip portion 11ba may be formed wider in a direction D3 intersecting directions D1 and D2. In Figure 2, directions D1, D2, and D3 are perpendicular to each other.

[0041] The drive unit 11c is positioned on the mounting portion 11a and rotationally drives the rotation axis 11bb of the holding portion 11b. In this embodiment, the direction of the rotation axis 11bb of the holding portion 11b is perpendicular to the direction of the torsional rotation axis S6 of the link 12f and along direction D3, but is not limited to this, and may be, for example, diagonally intersecting the direction of the torsional rotation axis S6. The drive unit 11c uses electricity as a power source and has a servo motor M7 (see Figure 5) as an electric motor to drive these, but is not limited to this. Furthermore, the drive unit 11c has a rotation sensor E7 (see Figure 5) that detects the amount of rotation (rotation angle) of the servo motor M7 or the holding portion 11b. The drive unit 11c may also have a current sensor (not shown) that detects the current value of the servo motor M7.

[0042] The force detector 11d is positioned between the mounting portion 11a and the link 12f and detects the force acting between the mounting portion 11a and the link 12f. The force detector 11d can detect the force acting on the end effector 11. The force detector 11d outputs a signal indicating the detected force value to the first control device 31. In this embodiment, the force detector 11d is a force sensor that detects six-axis forces, including forces in three orthogonal axial directions and moments, which are rotational forces around the three axes, but is not limited to this. The force detected by the force detector 11d is not limited to the six-axis forces described above, and may be, for example, forces in some of the six axes.

[0043] The first control device 31 rotates the holding part 11b with the drive device 11c to change the orientation of the holding part 11b, and moves the end effector 11 with the robot arm 12, thereby causing the end effector 11 to scrape and hold the floating matter S on the surface of the molten metal M of the first containment Cm. The configuration of the end effector 11 is not limited to anything that can move the floating matter S.

[0044] As shown in Figure 1, the imaging device 40 is positioned to image at least the opening Cma of the first housing Cm. In this embodiment, the imaging device 40 includes, but is not limited to, a visible camera capable of capturing moving images of digital images, which are visible images. The imaging device 40 performs imaging operations, for example, according to commands from the first control device 31, and sends signals of the captured image, etc., to the first control device 31. For example, the imaging device 40 may be positioned to project an image similar to the image of the opening Cma that user P would see if user P were positioned next to the robot arm 12. In this case, it is preferable that the imaging device 40 is positioned above the opening Cma and directed downward. However, the robot arm 12 may have protective members for heat shielding, and these protective members may obstruct part of the opening Cma from the imaging device 40. For this reason, in Figure 1, the imaging device 40 is positioned above the robot arm 12 between the opening Cma and the base 13 and directed diagonally downward towards the opening Cma.

[0045] Multiple imaging devices 40 may be arranged for a single aperture Cma. In the image captured by the imaging device 40 shown in Figure 1, the image of aperture Cma viewed from diagonally above may be distorted. For this reason, the first control device 31 or the like may generate an image that shows the image of aperture Cma viewed from the front by processing multiple images captured by multiple imaging devices 40.

[0046] The imaging device 40 may include a thermal imaging camera. For example, the thermal imaging camera may be an infrared camera that detects emitted infrared radiation and generates a thermal image showing the distribution of infrared radiation. The imaging device 40 outputs the signal of the captured thermal image to the first control device 31.

[0047] The display device 50 presents images and sounds, etc., received from the control device 30, to the user P of the robot system 1, which are necessary for operating the robot system 1. Examples of the display device 50 include, but are not limited to, a liquid crystal display and an organic or inorganic electroluminescent display. The display device 50 may also be equipped with a speaker that emits sound. For example, the display device 50 presents an image captured by the imaging device 40 and a processed image of said image to the user P who operates the operating device 20.

[0048] The input device 60 receives various information, data, and commands, and outputs the received information to the first control device 31. For example, the input device 60 may be equipped with known input means such as levers, buttons, touch panels, joysticks, and motion capture devices. For example, the input device 60 may be a teaching pendant, which is one of the teaching devices.

[0049] Figure 3 is a perspective view showing an example of the configuration of the operating device 20 according to the embodiment. Figure 4 is a perspective view showing an enlarged view of the operating section 230 in Figure 3. As shown in Figure 3, the operating device 20 comprises a support base 210, an operating arm 220, and an operating section 230. The support base 210 is a base that supports the operating device 20 on the surface on which the operating device 20 is placed. The support base 210 supports the operating arm 220 at its upper part.

[0050] The operating arm 220 receives commands to control the position of the end effector 11. The operating arm 220 is composed of a multi-joint arm. The operating arm 220 comprises links 221 to 223, joints JA1 to JA3, and drive units DRA1 to DRA3. Joints JA1 to JA3 are rotary joints. Note that the number of joints may be other than three.

[0051] Joint JA1 connects the base end of link 221 to the upper surface 211 of the support base 210 so as to be rotatable around a rotation axis SA1 that extends perpendicular to the upper surface 211. Joint JA2 connects the base end of link 222 to the tip of link 221 so as to be rotatable around a rotation axis SA2 that extends perpendicular to the rotation axis SA1. Joint JA3 connects the base end of link 223 to the tip of link 222 so as to be rotatable around a rotation axis SA3 that extends parallel to the rotation axis SA2.

[0052] Joint JA1 allows the operating arm 220 to rotate around the rotation axis SA1. Joints JA2 and JA3 allow the operating arm 220 to bend around the rotation axes SA2 and SA3. The operating arm 220 can move the tip of link 223 to various positions in three-dimensional space. For example, the position of the tip of link 223 can be expressed by the orientation determined by the amount of rotation (e.g., rotation angle) of joint JA1, and the elevation angle and distance determined by the amount of rotation (rotation angle) of joints JA2 and JA3.

[0053] Each of the drive units DRA1 to DRA3 is connected to joints JA1 to JA3 in a manner that allows for the transmission of driving force. Drive unit DRA1 is positioned on the support base 210, and drive units DRA2 and DRA3 are positioned adjacent to joint JA2. Drive units DRA1 and DRA2 are connected to joints JA1 and JA2, respectively, via a transmission mechanism (not shown) such as a speed reducer. Drive unit DRA3 is connected to joint JA3 via a transmission mechanism (not shown) such as a speed reducer, belt, and pulley. Each of the drive units DRA1 to DRA3 operates together with joints JA1 to JA3. When joints JA1 to JA3 are operated by user P, each of the drive units DRA1 to DRA3 is operated in the same way, generating driving force and thereby providing rotational force to joints JA1 to JA3. For example, drive units DRA1 to DRA3 can provide a reaction force to user P against the operating force that moves the tip of the operating arm 220.

[0054] The operating unit 230 receives commands to control the posture of the end effector 11. As shown in Figures 3 and 4, the operating unit 230 comprises links 231 to 235 and drive units DRA4 to DRA6. Link 231 has a grip 231a that can be grasped by user P and a shaft portion 231b connected to drive unit DRA4. The grip 231a protrudes outward at a position away from the shaft portion 231b, for example, on the opposite side of the support base 210 relative to the operating arm 220. Furthermore, link 231 is equipped with an input element 231c on the grip 231a. The input element 231c receives input commands to operate the drive unit 11c of the end effector 11. In this embodiment, the input element 231c comprises two push buttons, but its configuration is not limited as long as it has the above functions. One push button, when pressed and in the ON state, outputs a command to the drive unit 11c to rotate the holding part 11b in one direction, and when released and in the OFF state, outputs a command to the drive unit 11c to stop the above rotation. The other push button, when in the ON state, outputs a command to the drive unit 11c to rotate the holding part 11b in the opposite direction to the one direction, and when in the OFF state, outputs a command to the drive unit 11c to stop the above rotation.

[0055] Link 232 has connecting portions 232a and 232b that connect to links 231 and 234, respectively. The first connecting portion 232a fits from the outside onto the cylindrical outer surface of the shaft portion 231b of link 231 and can rotate on the outer surface of the shaft portion 231b about a rotation axis SB1. The rotation axis SB1 is also the axis of the shaft portion 231b.

[0056] Link 233 has connecting parts 233a and 233b that connect to links 231 and 235, respectively. The first connecting part 233a fits from the outside onto the outer circumferential surface of the shaft portion 231b of link 231 and can rotate on the outer circumferential surface of the shaft portion 231b around the rotation axis SB1. The first connecting part 233a is positioned side by side with the first connecting part 232a of link 232 in the axial direction of the rotation axis SB1. The drive unit DRA4 is fixed to the first connecting part 233a. Links 231 to 233 can rotate relative to each other around the rotation axis SB1.

[0057] Link 234 has a first connecting portion 234a that connects to the second connecting portion 232b of link 232, and a second connecting portion 234b that connects to the tip of link 223 of the operating arm 220. The first connecting portion 234a is rotatably connected to the second connecting portion 232b about the rotation axis SB2. The second connecting portion 234b is made up of a pulley and is rotatably connected to the tip of link 223 about the rotation axis SB4. The direction of rotation axis SB2 is obliquely intersecting the directions of rotation axis SB1 and rotation axis SB4.

[0058] Link 235 has a first connecting portion 235a that connects to the second connecting portion 233b of link 233, and a second connecting portion 235b that connects to the tip of link 223 of the operating arm 220. The first connecting portion 235a is rotatably connected to the second connecting portion 233b around the rotation axis SB3. The second connecting portion 235b is made up of a pulley and is rotatably connected to the tip of link 223 around the rotation axis SB5. The direction of rotation axis SB3 is obliquely intersecting the directions of rotation axis SB1 and rotation axis SB5. Rotation axis SB5 is parallel to rotation axis SB4 and, in this embodiment, coaxial. Furthermore, rotation axis SB5 is also parallel to rotation axis SA3 of the operating arm 220. For example, the direction of rotation axis SB2 and the direction of rotation axis SB3 intersect obliquely such that they form an obtuse angle with its apex moving away from the tip of link 223 in the axial direction of rotation axis SB5 and away from the second connecting parts 232b and 233b in the axial direction of rotation axis SB1.

[0059] Therefore, the grip 231a can move along a circumferential orbit CD1 around the rotation axis SB1. For example, the grip 231a can move in the yawing direction around the rotation axis SB1. In this case, the drive unit DRA4 rotates its drive shaft together with the link 231.

[0060] Furthermore, the grip 231a can move along a circumferential orbit in the circumferential direction CD2, which intersects the circumferential direction CD1, around the rotation axes SB4 and SB5. For example, the grip 231a can move in the pitching direction around the rotation axes SB4 and SB5. In this case, the second connecting parts 234b and 235b of links 234 and 235 rotate in the same rotational direction with the same amount of rotation, that is, they rotate synchronously.

[0061] Furthermore, the grip 231a can change its orientation by rotating around the rotation axes SB2 and SB3 in the circumferential direction CD1 and the circumferential direction CD3 which intersects the circumferential direction CD2. For example, the grip 231a can rotate around its axis. The grip 231a can move in the rolling direction around the rotation axes SB2 and SB3. At this time, links 232 and 233 rotate their second connecting parts 232b and 233b in the same direction, while their first connecting parts 232a and 233a rotate in opposite directions. As a result, the second connecting parts 234b and 235b of links 234 and 235 rotate in opposite directions. Therefore, the difference between the rotational phase of the second connecting part 234b and the rotational phase of the second connecting part 235b increases or decreases.

[0062] The operating unit 230 described above can move the grip 231a to various positions along a spherical surface, and change the orientation of the grip 231a at those positions. For example, the orientation of the grip 231a can be represented by a yawing angle determined by the amount of rotation (rotation angle) around the rotation axis SB1, a pitching angle determined by the amount of rotation (rotation angle) around the rotation axes SB4 and SB5, and a rolling angle determined by the amount of rotation (rotation angle) around the rotation axes SB2 and SB3. In the operating unit 230, the grip 231a is supported by pairs of parallel links, namely links 232 and 234 and links 233 and 235. In this embodiment, however, the central axis of yawing rotation, the central axis of pitching rotation, and the central axis of rolling rotation intersect perpendicularly at a single point. This allows the grip 231a to move along a spherical surface centered at that point. The mechanism of such an operating unit 230 constitutes a spherical parallel link mechanism.

[0063] The drive units DRA4, DRA5, and DRA6 are connected to links 231, 234, and 235, respectively, in a manner that enables the transmission of driving force. Drive unit DRA4 is positioned on link 233 and is connected to the shaft portion 231b of link 231 via a transmission mechanism (not shown), such as a gearbox. Drive units DRA5 and DRA6 are supported by link 221 of the operating arm 220. Drive units DRA5 and DRA6 are connected to the second connection portions 234b and 235b of links 234 and 235, respectively, via a transmission mechanism (not shown), such as a gearbox, pulley, and belt. Drive units DRA4 to DRA6, together with links 231, 234, and 235, rotate their respective drive shafts. Each of the drive units DRA4 to DRA6 is operated in the same way as when links 231, 234, and 235 are operated by user P via grip 231a, and generates a driving force that can impart rotational force to links 231, 234, and 235. For example, drive units DRA4 to DRA6 can provide a reaction force to user P to the operating force that causes movement along the spherical surface and changes in the posture of grip 231a. Drive units DRA1 to DRA6 are examples of force-applying devices.

[0064] Each of the drive units DRA1 to DRA6 has a servo motor MA1 to MA6 (see Figure 5) as an electric motor that uses electricity as a power source to drive them, but is not limited to this. Furthermore, each of the drive units DRA1 to DRA3 has a rotation sensor EA1 to EA3 (see Figure 5) that detects the amount of rotation (rotation angle) of joints JA1 to JA3. Each of the drive units DRA4 to DRA6 has a rotation sensor EA4 to EA6 (see Figure 5) that detects the amount of rotation (rotation angle) of links 231, 234, and 235. Each of the drive units DRA1 to DRA6 may also have a current sensor (not shown) that detects the current value of the servo motors MA1 to MA6. Each rotation sensor EA1 to EA6 and each current sensor outputs a signal indicating the detected value to the second control device 32. The servo motors MA1 to MA6 are driven according to the commands received from the second control device 32.

[0065] Figure 5 is a block diagram showing an example of the configuration of a control device 30 and its surroundings according to an embodiment. As shown in Figure 5, the control device 30 includes a first control device 31 and a second control device 32. The first control device 31 controls the operation of the entire robot 10, and the second control device 32 controls the operation of the entire operating device 20. For example, the first control device 31 and the second control device 32 may be computer devices. The first control device 31 is communicatively connected to the second control device 32, the imaging device 40, the display device 50, and the input device 60. The second control device 32 is communicatively connected to the first control device 31 and the operating device 20. The form of communication may be any form of communication, such as wired communication or wireless communication. The first control device 31 operates the robot 10 in response to commands received by the operating device 20 or the input device 60. The first control device 31 transmits information to the second control device 32, for example, to apply a reaction force to the grip 231a corresponding to the force received by the end effector 11. The first control device 31 outputs the image captured by the imaging device 40 or a processed image of said image to the display device 50. The second control device 32 outputs the information and commands received by the operating device 20 to the first control device 31. The second control device 32 also controls the operation of the drive units DRA1 to DRA6 of the operating device 20 based on the information and commands received from the first control device 31.

[0066] The first control device 31 is electrically connected to the servo motors M1 to M7 (labeled "SM" in Figure 5) of the robot 10 via drive circuits C1 to C7, respectively. Each drive circuit C1 to C7 adjusts the current value supplied to the servo motors M1 to M7 according to the command of the first control device 31. The first control device 31 is electrically connected to the rotation sensors E1 to E6 (labeled "EN" in Figure 5) of the robot 10. The second control device 32 is electrically connected to the servo motors MA1 to MA6 of the operating device 20 via drive circuits CA1 to CA6, respectively. Each drive circuit CA1 to CA6 adjusts the current value supplied to the servo motors MA1 to MA6 according to the command of the second control device 32. The second control device 32 is electrically connected to the rotation sensors EA1 to EA6 of the operating device 20. The first control device 31 and drive circuits C1 to C7 constitute the first control unit 310, and the second control device 32 and drive circuits CA1 to CA6 constitute the second control unit 320.

[0067] The first control unit 31 is composed of an arithmetic unit having a processor and memory, etc. The arithmetic unit transmits and receives information, data, and commands with other devices, including the operating device 20. The arithmetic unit receives detection signals from various sensors and outputs control signals to each controlled object. The memory is composed of semiconductor memory such as volatile memory and non-volatile memory, storage devices such as hard disks and SSDs (Solid State Drives). For example, the memory stores programs executed by the arithmetic unit and various fixed data.

[0068] The functions of the arithmetic unit may be implemented by a computer system (not shown) consisting of a processor such as a CPU (Central Processing Unit), volatile memory such as RAM (Random Access Memory), and non-volatile memory such as ROM (Read-Only Memory). Some or all of the functions of the arithmetic unit may be implemented by the CPU using RAM as a work area to execute a program recorded in ROM. Some or all of the functions of the arithmetic unit may be implemented by the above-mentioned computer system, by dedicated hardware circuits such as electronic circuits or integrated circuits, or by a combination of the above-mentioned computer system and hardware circuits.

[0069] Such a first control device 31 may consist of, for example, a microcontroller, an MPU (Micro Processing Unit), an LSI (Large Scale Integration), a system LSI, a PLC (Programmable Logic Controller), a logic circuit, etc. Multiple functions of the first control device 31 may be realized by integrating each function onto a single chip, or by integrating some or all of them onto a single chip. Furthermore, each circuit may be a general-purpose circuit or a dedicated circuit. As the LSI, an FPGA (Field Programmable Gate Array) that can be programmed after LSI manufacturing, a reconfigurable processor that allows the connection and / or configuration of circuit cells inside the LSI to be reconfigured, or an ASIC (Application Specific Integrated Circuit) that combines multiple functional circuits into one for a specific application may be used.

[0070] Some or all of the functions of the second control device 32 may be realized by a computer system (not shown) consisting of a CPU, RAM, ROM, etc., by a dedicated hardware circuit such as an electronic circuit or integrated circuit, or by a combination of the above computer system and hardware circuit.

[0071] The first control device 31 and the second control device 32 may each perform their respective processes through centralized control by a single computer device, or they may each perform their respective processes through distributed control by the cooperation of multiple computer devices. The first control device 31 and the second control device 32 may be included in a single computer device.

[0072] [Functional configuration of the first and second control devices] Figure 6 is a block diagram showing an example of the functional configuration of a control device 30 according to an embodiment. As shown in Figure 6, the first control device 31 includes an image processing unit 311, an information processing unit 312, operation command units 313 and 314, an operation control unit 315, and a storage unit 316 as functional components. The second control device 32 includes an input processing unit 321, an operation command unit 322, an operation command unit 323, and an operation control unit 324 as functional components. The functions of the above functional components, excluding the storage unit 316, are realized by a computer system and / or hardware circuit. The storage unit 316 is realized by memory. Not all of the above functional components are essential.

[0073] The input processing unit 321 of the second control device 32 receives and processes information and commands from the operating device 20 and outputs them to the first control device 31 and / or the operation command unit 322. For example, the input processing unit 321 detects information corresponding to the position, orientation, positional movement speed, and orientational movement speed of the grip 231a from the detection values ​​of the rotation sensors EA1 to EA6 of the operating device 20 and outputs it to the operation command unit 322. In addition, the input processing unit 321 detects information representing the drive, drive stop, and drive direction commands to the drive device 11c of the end effector 11 from the ON and OFF signals of the two push buttons of the input element 231c and outputs it to the operation command unit 322.

[0074] For example, the input processing unit 321 detects the orientation, elevation angle, and distance of the grip 231a relative to the reference point (not shown) of the joint JA1 of the operating arm 220, as well as the rate of change of these, from the values ​​detected by the rotation sensors EA1 to EA3, as position and attitude information corresponding to the position and speed of movement of the grip 231a. The input processing unit 321 also detects the yawing angle, pitching angle, and rolling angle of the grip 231a, as well as the rate of change of these, from the values ​​detected by the rotation sensors EA4 to EA5, as position and attitude information corresponding to the attitude and speed of movement of the grip 231a.

[0075] The operation command unit 322 generates operation commands to control the position, orientation, positional movement speed, and orientational movement speed of the end effector 11 based on the positional orientation information detected by the input processing unit 321, and outputs them to the first control device 31. The operation command is an example of a manual command. For example, the operation command unit 322 may convert the azimuth, elevation angle, and distance of the grip 231a with respect to the reference point of the joint JA1 of the operating arm 220, as well as the rate of change thereof, into coordinate values ​​in a three-dimensional Cartesian coordinate system (hereinafter also called the "operational coordinate system") with the above reference point as the origin. The operation command unit 322 may also convert the yawing angle, pitching angle, and rolling angle of the grip 231a, as well as the rate of change thereof, into rotation angles and their rate of change around the coordinate axes of a three-dimensional Cartesian coordinate system (hereinafter also called the "tip coordinate system") with the center of the spherical surface (not shown) on which the grip 231a moves as the origin. The tip coordinate system is a coordinate system based on the tip of the link 223. The operation command unit 322 may generate an operation command that includes the converted command value. The operation command unit 322 also generates an operation command that includes commands for driving, stopping, and driving direction for the drive device 11c of the end effector 11, based on the information of the input element 231c of the grip 231a detected by the input processing unit 321.

[0076] The position and movement speed of the grip 231a correspond to the position and movement speed of the end effector 11. The posture and movement speed of the posture of the grip 231a correspond to the posture and movement speed of the posture of the end effector 11. For example, the operating coordinate system can be associated with the robot coordinate system, and the tip coordinate system can be associated with the mechanical interface coordinate system (hereinafter also referred to as the "I / F coordinate system"). The robot coordinate system is a coordinate system with the robot 10 as the reference point, with the robot 10 as the origin, and a reference point (not shown) set on the base 13 of the robot arm 12. The I / F coordinate system is a coordinate system with the mechanical interface surface of the link 12f of the robot arm 12 as the origin, with the mechanical interface surface as the reference point. The three-dimensional positions of the end effector 11 and the grip 231a can be represented using the robot coordinate system and the operating coordinate system, respectively. The three-dimensional orientations of the end effector 11 and grip 231a can be represented using the I / F coordinate system and the tip coordinate system, respectively.

[0077] The motion command unit 323 receives a force command from the first control device 31, generates an motion command to apply force to the grip 231a according to the force command, and outputs it to the motion control unit 324. The motion command is a command to drive the drive devices DRA1 to DRA6 to apply force to the grip 231a.

[0078] The force command is a command generated by the first control device 31 based on the detection value of the force detector 11d of the end effector 11. The force command is a command to apply a force to the grip 231a with a magnitude and direction corresponding to the magnitude and direction of the force detected by the force detector 11d.

[0079] For example, the force command may include command values ​​for the magnitude and direction of the force in the robot coordinate system and command values ​​for the magnitude and direction of the force in the I / F coordinate system. The motion command unit 323 detects target values ​​for the magnitude and direction of the force in the operating coordinate system and the end-effector coordinate system by performing calculations on the command values ​​for the magnitude and direction of the force in the robot coordinate system and the I / F coordinate system, respectively, included in the force command. Furthermore, the motion command unit 323 calculates the rotation direction, amount of rotation, and rotational torque of the servo motors MA1 to MA3 of the drive devices DRA1 to DRA3 for generating the target force in the operating coordinate system, and the rotation direction, amount of rotation, and rotational torque of the servo motors MA4 to MA6 of the drive devices DRA4 to DRA6 for generating the target force in the end-effector coordinate system. The motion command unit 323 generates a motion command that includes a position command indicating the rotation direction and amount of rotation of the servo motors MA1 to MA6, and a force command indicating the rotational torque of the servo motors MA1 to MA6. The operation command unit 323 may use the detected values ​​of the rotation sensors EA1 to EA6 of the servo motors MA1 to MA6 and the current sensor (not shown) as feedback information when generating the above commands.

[0080] The command values ​​for the magnitude and direction of the force in the robot coordinate system correspond to the magnitude and direction of the linear force acting on the end effector 11. The target values ​​for the magnitude and direction of the force in the operating coordinate system correspond to the magnitude and direction of the linear force applied to the grip 231a. The command values ​​for the magnitude and direction of the force in the I / F coordinate system correspond to the magnitude and direction of the rotational force, i.e., moment, acting on the end effector 11. The target values ​​for the magnitude and direction of the force in the tip coordinate system correspond to the magnitude and direction of the rotational force (moment) applied to the grip 231a.

[0081] The motion control unit 324 determines the current value to supply to each servo motor MA1 to MA6 in accordance with the motion command received from the motion command unit 323, and controls the supply of current. Furthermore, in the above control, the motion control unit 324 may use the detected values ​​of the rotation sensors EA1 to EA6 and current sensors (not shown) of the servo motors MA1 to MA6 as feedback information. Such a motion control unit 324 performs servo control. As a result, the user P operating the grip 231a perceives the force corresponding to the force received by the end effector 11 as a reaction force to the operating force.

[0082] The image processing unit 311 of the first control device 31 receives captured image data from the imaging device 40 and outputs the image data to the display device 50 for display. The image processing unit 311 may also perform image processing on the image data before outputting it to the display device 50. The image processing unit 311 is an example of an image processing device.

[0083] For example, the image processing unit 311 may perform image processing on the image data to visually differentiate the image of the molten metal M from the image of the floating matter S based on the brightness value, which is the pixel value of each pixel in the image data. For example, the image processing unit 311 may perform processing to change the color, brightness, intensity, and texture of the pixels. For example, the image processing unit 311 may perform different image processing on pixels with brightness values ​​above a threshold and pixels with brightness values ​​below a threshold. For example, if the image of the molten metal M corresponds to pixels with brightness values ​​above a threshold and the image of the floating matter S corresponds to pixels with brightness values ​​below a threshold, the processing may be performed to make the pixels with brightness values ​​below a threshold more visually prominent than the pixels with brightness values ​​above a threshold.

[0084] Furthermore, a temperature sensor (not shown) for detecting the surface temperature of the molten metal M in the first containment Cm may be provided at the work area WP. Preferably, the temperature sensor can detect the surface temperature of the molten metal M at a position away from the surface, and may be, for example, an infrared sensor. If the imaging device 40 includes a thermal imaging camera, the temperature sensor may be replaced by the thermal imaging camera. The image processing unit 311 may receive temperature data showing the temperature distribution of the surface of the molten metal M from the temperature sensor or thermal imaging camera, and perform image processing on the image data of the imaging device 40 using this temperature data. Based on the temperature distribution shown in the temperature data, the image processing unit 311 may perform image processing to visually or otherwise differentiate the molten metal M from the suspended matter S.

[0085] For example, the image processing unit 311 may generate image data representing the temperature distribution based on temperature data and superimpose it on the image data from the imaging device 40. Alternatively, the image processing unit 311 may display the image data representing the temperature distribution and the image data from the imaging device 40 side by side. Alternatively, the image processing unit 311 may perform different image processing on pixels that capture regions with temperatures above a threshold and pixels that capture regions with temperatures below a threshold. For example, if the image of the molten metal M corresponds to pixels with temperatures above a threshold and the image of the suspended matter S corresponds to pixels with temperatures below a threshold, the above processing may involve changing the color, brightness, density, and texture of the pixels to make the pixels with temperatures below a threshold more visually prominent than the pixels with temperatures above a threshold.

[0086] For example, if the imaging device 40 includes a stereo camera that captures images capable of detecting the distance to a subject, the image processing unit 311 may generate a distance image and a 3D image of the image captured by the imaging device 40 by detecting the distance to the subject captured by each pixel.

[0087] The storage unit 316 can store various types of information and allows for the retrieval of stored information. For example, the storage unit 316 stores thresholds used by the image processing unit 311 and thresholds used by the second operation command unit 314, which will be described later. The storage unit 316 may also store image data captured by the imaging device 40 and image data processed by the image processing unit 311. The storage unit 316 may also store programs for realizing each function of the first control device 31. The storage unit 316 may also store automatic driving information, which is information used by the program when the robot 10 is made to operate automatically. The automatic driving information may include pre-set fixed data and teaching data.

[0088] The information processing unit 312 receives various information and commands from the second control device 32 and the input device 60, and outputs them to the corresponding functional components. For example, the information processing unit 312 receives operation commands from the second control device 32 and outputs them to the operation command units 313 and 314. The information processing unit 312 receives commands from the input device 60 instructing the robot 10 to perform automatic driving operations, and commands instructing the execution of automatic driving, and outputs them to the first operation command unit 313. The above commands are examples of automatic commands.

[0089] The first motion command unit 313 generates motion commands for the robot 10 according to the operation motion commands and automatic commands, and outputs them to the motion control unit 315. For example, the first motion command unit 313 converts the command values ​​in the operation coordinate system included in the operation motion command into command values ​​in the robot coordinate system. Note that the movable range of the grip 231a and the movable range of the end effector 11 are different, and the scales of movement between the grip 231a and the end effector 11 are different. For this reason, the first motion command unit 313 may convert the command values ​​in the robot coordinate system into command values ​​obtained by increasing the command values ​​in the operation coordinate system by a predetermined ratio. In addition, the first motion command unit 313 converts the command values ​​in the tip coordinate system included in the operation motion command into command values ​​in the I / F coordinate system. In this case, the first motion command unit 313 may convert the command values ​​in the I / F coordinate system into command values ​​obtained by increasing the command values ​​in the tip coordinate system by a predetermined ratio.

[0090] The first motion command unit 313 calculates the rotation direction, rotation amount, and rotational torque of the servo motors M1 to M6 of the drive units DR1 to DR6 in order to move the end effector 11 according to the command values ​​in the robot coordinate system and the command values ​​in the I / F coordinate system. The first motion command unit 313 also calculates the rotation direction, rotation amount, and rotational torque of the servo motor M7 to be driven by the drive unit 11c according to the command to the end effector 11 included in the operation motion command. The first motion command unit 313 also reads automatic operation information corresponding to the automatic command from the storage unit 316 and calculates the rotation direction, rotation amount, and rotational torque of the servo motors M1 to M7 to be driven by the drive units DR1 to DR6 and 11c according to the automatic operation information. The first motion command unit 313 generates an operation command that includes the rotation direction and rotation amount of the servo motors M1 to M7 as a position command. The first operation command unit 313 may use detected values ​​from the rotation sensors E1 to E7 of the servo motors M1 to M7 as feedback information when generating the above command.

[0091] Here, the motion command includes at least one of a position command and a force command. The position command includes at least a target value for position among the target values ​​for position, orientation, and velocity of position and orientation of the controlled object, and in this embodiment, all target values ​​are included. The position and velocity may represent position and velocity in three-dimensional space, and the orientation and velocity may represent orientation and velocity in three-dimensional space. Furthermore, the position command may include the execution time of the position command. The force command includes at least a target value for the magnitude of force among the target values ​​for the magnitude and direction of force applied by the controlled object, and in this embodiment, both target values ​​are included. The direction of force may represent direction in three-dimensional space. The force command may include the execution time of the force command. In this specification and in the claims, “position” may mean at least a position in three-dimensional space among position in three-dimensional space, velocity of position, orientation, and velocity of orientation.

[0092] The motion control unit 315 determines the current value to supply to each servo motor M1 to M7 in accordance with the motion command received from the first motion command unit 313, and controls the supply of current. Furthermore, in the above control, the motion control unit 315 may use the detected values ​​of the rotation sensors E1 to E7 and current sensors (not shown) of the servo motors M1 to M7, as well as the force detector 11d of the end effector 11, etc., as feedback information. Such a motion control unit 315 performs servo control. As a result, for example, the robot arm 12 operates the end effector 11 in correspondence with the movement of the grip 231a of the operating device 20.

[0093] The second motion command unit 314 receives motion information of the robot 10 from the motion control unit 315, uses this motion information to generate motion commands for the operating device 20, and outputs them to the second control device 32. The motion commands include force commands, but may also include position commands.

[0094] The motion information of the robot 10 includes at least one of the motion data of the robot arm 12 and the end effector 11. The motion data of the robot arm 12 includes position data representing the interface surface of link 12f and / or the position of joints JT1 to JT6, etc., during operation. The motion data of the robot arm 12 may also include force data representing the force that the end effector 11, i.e., the interface surface, applies to the object. The force that the interface surface applies to the object is also the force that the interface surface receives from the object. The motion data of the end effector 11 includes data representing the rotational position of the holding part 11b. The position data may include the position in three-dimensional space and the orientation in three-dimensional space. The force data may include the magnitude of the force and the direction of the force in three-dimensional space. The position data and force data may be time-series data associated with the time of occurrence of each position and each force.

[0095] Furthermore, the operation information may include information other than operation data, such as vibration data, shock data, optical data, sound data, temperature data, humidity data, and pressure data such as atmospheric pressure, which occur at the interface surface of link 12f. The operation information handled in this embodiment includes at least operation data.

[0096] The second motion command unit 314 uses the rotation amounts (rotation angles) of joints JT1 to JT6 included in the motion data to calculate the three-dimensional position and orientation of the interface surface of link 12f in the robot coordinate system. Furthermore, the second motion command unit 314 determines the I / F coordinate system from the three-dimensional position and orientation of the interface surface. The second motion command unit 314 generates motion commands (force commands) that include force command values ​​in the robot coordinate system and the I / F coordinate system by processing the detection values ​​of the six axes of the force detector 11d included in the motion data. For example, the second motion command unit 314 detects the magnitude and direction of linear forces in the robot coordinate system and the magnitude and direction of rotational forces in the I / F coordinate system from the detection values ​​of the force detector 11d. The second motion command unit 314 determines the force command values ​​in the robot coordinate system and the I / F coordinate system by reducing at least one of the detected values ​​in the robot coordinate system and the detected values ​​in the I / F coordinate system to a predetermined ratio.

[0097] Furthermore, when the second motion command unit 314 reduces the command value in the robot coordinate system and the I / F coordinate system, it may vary the above ratio depending on the magnitude of the command value. For example, if the magnitude of the force of the command value in the robot coordinate system is greater than 0 and less than or equal to the first threshold, the second motion command unit 314 may determine the command value to be the first ratio by reducing the magnitude of the force of the command value, and if the magnitude of the force of the command value in the robot coordinate system is greater than the first threshold, it may determine the command value to be the second ratio by reducing the magnitude of the force of the command value. The second ratio may be smaller than the first ratio.

[0098] Furthermore, the second motion command unit 314 may, if the magnitude of the force of the command value in the I / F coordinate system is greater than 0 and less than or equal to the first threshold, reduce the magnitude of the force of the command value to the first ratio and determine that as the command value, and if the magnitude of the force of the command value in the I / F coordinate system is greater than the first threshold, reduce the magnitude of the force of the command value to the second ratio and determine that as the command value. Note that the first threshold, first ratio, and second ratio do not have to be the same values ​​between the I / F coordinate system and the robot coordinate system.

[0099] For example, the first threshold may be set to a value corresponding to the magnitude of the force when the holding part 11b of the end effector 11 comes into contact with the floating object S. Forces greater than 0 and less than or equal to the first threshold correspond to the forces when the holding part 11b comes into contact with and does not come into contact with the floating object S, while forces greater than the first threshold correspond to the forces when the holding part 11b moves, lifts, and holds the floating object S. Since the reduction rate is smaller when the force is greater than 0 and less than or equal to the first threshold, the determined command value can more clearly indicate whether or not there is contact between the holding part 11b and the floating object S. Since the reduction rate is larger when the force is greater than the first threshold, the determined command value suppresses the application of excessive force to the grip 231a. Note that the first rate may be a rate that increases the command value.

[0100] Furthermore, if the value exceeds a second threshold greater than the first threshold, the second operation command unit 314 may be configured to reduce the command value to a third percentage smaller than the second percentage. This prevents excessive force from being applied to the grip 231a.

[0101] [Robot system operation] The operation of the robot system 1 according to the embodiment will now be described. Figure 7 is a flowchart showing an example of the operation of the robot system 1 according to the embodiment in manual operation. As shown in Figure 7, in step S1, user P inputs a command to the input device 60 of the robot system 1 to start the work of removing floating objects S using the robot 10 in manual operation, and the first control device 31 receives the command.

[0102] Next, in step S2, the first control device 31 activates the imaging device 40 to image the surface of the molten metal M of the first container Cm. The first control device 31 displays the image of the molten metal M received from the imaging device 40 on the display device 50.

[0103] Next, in step S3, user P looks at the image on the display device 50, grasps the grip 231a of the operating device 20, and moves the grip 231a in the intended direction and orientation. For example, user P operates the grip 231a with the intention of performing the following action: the robot arm 12 uses the holding part 11b of the end effector 11 to scrape up the floating matter S on the surface of the molten metal M, then penetrates the floating matter S with the holding part 11b and rotates it around its axis, and then moves the holding part 11b upwards, thereby moving the floating matter S, which is held entangled in the holding part 11b, to the second container Cs and depositing it.

[0104] Next, in step S4, the second control device 32 generates an operation command based on the detection values ​​of the rotation sensors EA1 to EA6 of the drive devices DRA1 to DRA6, which operate together with the movement of the grip 231a, and outputs it to the first control device 31.

[0105] Next, in step S5, the first control device 31 processes operation commands to detect command values ​​for the position and orientation of the end effector 11. Furthermore, the first control device 31 generates operation commands that include command values ​​for the operation of the servo motors M1 to M6 of the drive units DR1 to DR6, in order to operate the end effector 11 according to the above command values.

[0106] Next, in step S6, the first control device 31 acquires motion information of the robot arm 12. Furthermore, the first control device 31 uses this motion information as feedback information to operate the servo motors M1 to M6 according to the motion command. In other words, the first control device 31 operates each joint JT1 to JT6 of the robot arm 12.

[0107] Next, in step S7, the first control device 31 generates a force command (operation command) for the operating device 20 using the operation information of the robot arm 12 and outputs it to the second control device 32. The force command is a command to apply a force to the grip 231a that corresponds to the detected value of the force detector 11d of the end effector 11.

[0108] Next, in step S8, the second control device 32 generates and outputs operation commands to the drive devices DRA1 to DRA6 to apply force to the grip 231a according to the force command.

[0109] Next, in step S9, the second control device 32 acquires the operation information of the drive units DRA1 to DRA6. Furthermore, the second control device 32 uses this operation information as feedback information to operate the servo motors MA1 to MA6 according to the operation command. In other words, the second control device 32 generates a reaction force in the servo motors MA1 to MA6 in response to the operating force applied to the grip 231a.

[0110] Next, in step S10, the second control device 32 determines whether or not a command to complete the work has been entered into the input device 60 by the user P. If a command has been entered (Yes in step S10), the series of processes is terminated. If no command has been entered (No in step S10), the process returns to step S3.

[0111] Steps S1 to S10 allow user P to perceive the force corresponding to the force that the end effector 11 receives from the floating object S via the grip 231a, and to operate the grip 231a to cause the robot 10 to perform the task of removing the floating object S.

[0112] (modified version) The robot system 1 according to this embodiment comprises one robot 10, which is connected to the operating device 20 via a control device 30, but is not limited to this. The robot system 1 may comprise a plurality of robots 10, and one of the plurality of robots 10 may be selectively connected to the operating device 20 via the control device 30.

[0113] Figure 8 shows an example of the configuration of a modified robot system 1A. As shown in Figure 8, the robot system 1A comprises 10 robots 10, one operating device 20, one control device 30, 10 imaging devices 40, one display device 50, one input device 60, and a relay device 70. The robots 10 and imaging devices 40 are each positioned adjacent to each of the 10 first housings Cm located in the work area WP.

[0114] The relay device 70 connects the 10 robots 10 and the 10 imaging devices 40 to the first control device 31. The relay device 70 connects one robot 10 to the first control device 31 in a communicative manner according to a command from the first control device 31. The relay device 70 connects one imaging device 40 to the first control device 31 in a communicative manner according to a command from the first control device 31. Robots 10 and imaging devices 40 adjacent to each other in the first housing Cm may be associated, and the relay device 70 may connect the associated robot and imaging device 40 to the first control device 31 in a communicative manner according to a command from the first control device 31. The relay device 70 may include a circuit that can switch the communication connections, and may include such circuit and a computer system.

[0115] When user P determines which of the 10 first containment bodies Cm will remove the floating object S, user P inputs a command to the input device 60 specifying the robot 10 corresponding to that first containment body Cm. Upon receiving the command from the input device 60, the first control device 31 outputs a command to the relay device 70 to connect the robot 10, the imaging device 40 corresponding to the robot 10, and the first control device 31, and the relay device 70 executes the connection. As a result, the robot 10 is connected to the operating device 20 via the relay device 70, the first control device 31, and the second control device 32, and the imaging device 40 is connected to the display device 50 via the first control device 31. Therefore, user P can select any robot 10 from the 10 robots 10 and operate it using the operating device 20. The input device 60 is an example of a designation device.

[0116] The robot system 1A may further include a plurality of operating devices 20, a plurality of sets of control devices 31 and 32, a plurality of display devices 50, and a plurality of input devices 60. The relay device 70 may be configured to connect the robot 10 and the imaging device 40 corresponding to the robot 10, and the first control device 31 specified in the input device 60, according to a command that specifies the robot 10 to be input to the input device 60. This makes it possible to connect any robot 10 to any operating device 20.

[0117] Furthermore, the imaging device 40 may not be placed on each robot 10, and one imaging device 40 may be configured to image two or more robots 10. For example, the imaging device 40 may be mounted on a gimbal or the like, and the imaging direction may be changed. The first control device 31 may control the gimbal or the like to cause the imaging device 40 to image the robot to be operated.

[0118] (Other embodiments) While examples of embodiments of this disclosure have been described above, this disclosure is not limited to the embodiments described above. That is, various modifications and improvements are possible within the scope of this disclosure. For example, embodiments that have been modified in various ways, and forms constructed by combining components from different embodiments, are also included within the scope of this disclosure.

[0119] For example, in the embodiment, the control of the first control device 31 was exemplified such that the robot 10 manually operates to move and deposit floating matter S on the surface of the molten metal M into the second container Cs, but it is not limited to this. For example, the first control device 31 may be configured to have the robot 10 perform some or all of the above-mentioned series of operations by automatic operation.

[0120] For example, since the state of suspended matter S on the surface of the molten metal M is not constant, the operation of the end effector 11 to gather and hold the suspended matter S is more effective when it is a flexible operation that follows the judgment of the user P, rather than a predetermined fixed operation. The first control device 31 may be configured to have the robot 10 perform such flexible operations manually, and to have the robot 10 perform steady operations, such as moving the end effector 11 that holds the suspended matter S to the second containment Cs, automatically.

[0121] Furthermore, the first control device 31 may be configured to allow the robot 10 to perform flexible operations as described above, or an entire series of operations, through automatic operation. In this case, the first control device 31 may be configured to use AI (Artificial Intelligence) for automatic operation. The first control device 31 may also be configured to use AI for automatic operation of steady-state operations. For example, since the shape, dimensions, and weight of the floating object S held by the end effector 11 are not constant, flexible operation by AI that corresponds to the state of the floating object S held is effective.

[0122] For example, the AI ​​may include a learning model that performs machine learning. For example, the learning model may include a neural network. The learning data for the learning model may include data obtained during the manual operation of the robot 10 in manual operation, and / or data obtained during the corrective operation of the robot 10 in automatic operation, where the robot's operation is modified according to manual operations input to the control device 20, and may also include data obtained during the manual operation and / or corrective operation of various other robots. The learning model may also be a model that performs machine learning using training data.

[0123] For example, the learning model may use image data of the surface of the molten metal M captured by the imaging device 40 or processed image data thereof as input data, and information corresponding to the operation commands of the robot 10 as output data. The learning model may also use the operation data of the robot 10 as training data. The learning model may be configured to perform machine learning by adjusting the neural network so that the output data, which is output using image data of the surface of the molten metal M captured immediately before the robot 10 performs the operation of the operation data or processed image data thereof as input data, matches the training data or minimizes the error with the training data. The image processing may be image processing for identifying floating objects S, for example, the image processing exemplified with respect to the image processing unit 311.

[0124] As a result, the learning model can learn from various operations performed by different users as a result of visually inspecting image data of the surface of the molten metal M, and output appropriate output data that reflects the user's skilled operation. The first control device 31 can cause the robot 10 to perform an appropriate action based on the image data of the surface of the molten metal M, according to the state of the suspended matter S shown in the image data.

[0125] Alternatively, the learning model may use image data of the surface of the molten metal M captured by the imaging device 40 or its image processing data and the detected value of the force detector 11d as input data, and output information corresponding to the operation commands of the robot 10 as output data. The learning model may also use the operation data of the robot 10 as training data. The learning model may be configured to use machine learning to match the output data, which is output using image data of the surface of the molten metal M captured immediately before the robot 10 performs the operation of the operation data or its image processing data and the detected value of the force detector 11d immediately before the operation of the operation data, with the training data or minimize the error with the training data.

[0126] As a result, the learning model can learn from various operations performed by different users as a result of visualizing image data of the molten metal M surface and perceiving the force feedback on the end effector 11 via the grip 231a, and output appropriate output data that reflects the user's skilled and delicate operation. Based on the image data of the molten metal M surface and the force on the end effector 11, the first control device 31 can cause the robot 10 to perform actions appropriate to the state of the floating matter S and the state of the end effector 11 shown in the image data. [Explanation of Symbols]

[0127] 1.1A Robot System 10 Robots 11 End Effectors 11d Force detector 12 Robot Arms 20 Operating device 30 Control device 31 First Control Device 32 Second control device 40 Imaging device 50 Presentation device 311 Image Processing Unit (Image Processing Device)

Claims

1. A robot having a robotic arm and an end effector at the tip of the robotic arm, and positioned adjacent to a container for holding liquid, A force detector for detecting the force acting on the end effector, An imaging device for imaging the surface of the liquid, The robot comprises a control device that controls the movement of the robot, The end effector has a configuration that can move floating particles on the surface of the liquid, The control device includes a learning model and is configured to cause the robot to act based on the output data of the learning model. The learning model takes image data of the surface of the liquid captured by the imaging device or image processing data obtained by applying image processing to the image data and the detected force value detected by the force detector as input data, and outputs information corresponding to the robot's operation commands as output data. The robot's actions corresponding to the output data of the learning model include at least one of the following actions: moving floating objects on the surface of the liquid; removing floating objects from the surface of the liquid; and depositing the removed floating objects into a collection point. Robot system.

2. The system further includes an image processing device for processing images captured by the aforementioned imaging device, The image processing apparatus is configured to perform image processing to differentiate between the liquid and the suspended matter. The learning model takes image processing data of the surface of the liquid object processed by the image processing device and the force detected by the force detector as input data, and outputs information corresponding to the robot's movement commands as output data. The robot system according to claim 1.

3. The system further includes an operation input device positioned away from the aforementioned housing, which receives input for operations to the robot and outputs commands to operate the robot in response to such operations, The commands output by the operation input device include manual commands for operating the robot in response to user operations input to the operation input device for manual operation of the robot. The control device causes the robot to perform at least a portion of the series of operations of the robot that move the floating object from the containment to the outside, in accordance with the manual command. The robot system according to claim 1 or 2.

4. It is provided with an operation input device that is positioned away from the housing and receives input for operations to the robot and outputs commands to operate the robot in response to said operations, The commands output by the aforementioned operation input device include automatic commands for operating the robot in response to inputs to the aforementioned operation input device for the automatic operation of the robot. The control device causes the robot to perform at least a portion of the series of operations of the robot that move the floating object from the containment to the outside, in accordance with the automatic command. The robot system according to any one of claims 1 to 3.

5. The aforementioned learning model is configured to perform machine learning, The learning data used by the learning model for machine learning includes: first motion data of the robot obtained during manual operation when the robot is operating in manual operation; second motion data of the robot obtained during corrective operation when the robot is operating while being corrected according to manual operations input to the operation input device during automatic operation; or both the first motion data and the second motion data. The robot system according to claim 3 or 4.

6. The aforementioned learning model is configured to perform machine learning, The learning model uses motion data, including the position data of the end effector and the force data of the end effector during the robot's operation, as training data. The control device is configured to use the image data of the surface of the liquid captured by the imaging device when the robot performs the operation described in the motion data of the training data, or the image processing data of said image data, and the detected force values ​​detected by the force detector when the robot performs the operation described in the motion data of the training data as input data, to perform machine learning on the learning model so that the output data output by the learning model matches the training data or minimizes the error between the training data and the output data output by the learning model. The robot system according to any one of claims 1 to 5.

7. The method involves obtaining a force detection value from a force detector that detects the force acting on the end effector of a robot, wherein the robot has a robot arm and an end effector at the tip of the robot arm and is positioned adjacent to a container that holds a liquid, and the end effector has a configuration that can move floating objects on the surface of the liquid, To acquire an image of the surface of the liquid substance captured by the imaging device, The method involves causing the robot to operate based on the output data of a learning model, wherein the learning model takes image data of the surface of the liquid or image processing data obtained by applying image processing to said image data and the detected force as input data, and outputs information corresponding to the robot's operation commands as output data. The robot's actions corresponding to the output data of the learning model include at least one of the following actions: moving floating objects on the surface of the liquid; removing floating objects from the surface of the liquid; and depositing the removed floating objects into a collection point. Robot control methods.

Images