Robot system, robot processing method and processing program

Abstract

To process workpiece without teaching at a site.SOLUTION: A robot system 100 includes a robot 1 for processing a processed part B of an object W, a storage part 32 for storing an image of the object W and three-dimensional information, a specification device 9 for specifying the processed part B from the image, and a control device 3 for controlling the robot 1. The control device 3 derives the processed part B in the three-dimensional information on the basis of the part specified by the specification device 9 in the image and the three-dimensional information, and allows the robot 1 to process the processed part B by operating the robot 1 on the basis of the three-dimensional information of the processed part B.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present disclosure relates to a robot system, a robot processing method, and a processing program. [Background technology]

[0002] Systems for machining workpieces using robots have been known for some time. For example, Patent Document 1 discloses a robot system in which a robot holding a workpiece is moved according to rough teaching points, and the robot presses the workpiece against a tool in a desired pressing direction. That is, in this robot system, the workpiece moves roughly along the rough teaching points while the tool is pressed against the workpiece with a desired force. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 06-289923 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the robot system of Patent Document 1, it is necessary to provide teaching points according to the workpiece on-site, even if the teaching points are only roughly accurate. If the on-site work is reduced, processing by the robot becomes easier.

[0005] The present disclosure has been made in consideration of the above points, and its purpose is to enable workpiece machining without on-site teaching. [Means for solving the problem]

[0006] The robot system of the present disclosure comprises a robot that processes a processing portion of an object, a memory unit that stores an image and three-dimensional information of the object, a designation device for designating the processing portion from within the image, and a control device that controls the robot, wherein the control device derives the processing portion in the three-dimensional information based on the portion in the image designated by the designation device and the three-dimensional information, and causes the robot to process the processing portion by operating the robot based on the three-dimensional information of the processing portion.

[0007] The robot processing method disclosed herein includes specifying a processing portion of an object in an image of the object, deriving the processing portion in the three-dimensional information based on the specified portion in the image and three-dimensional information of the object, and operating the robot based on the three-dimensional information of the processing portion, thereby causing the robot to process the processing portion.

[0008] The processing program disclosed herein causes a computer to receive a specification of a processing portion in an image of the object to have a robot process the processing portion, derive the processing portion in the three-dimensional information based on the specified portion in the image and three-dimensional information of the object, and operate the robot based on the three-dimensional information of the processing portion to have the robot process the processing portion. [Effects of the Invention]

[0009] According to the robot system, the workpiece can be automatically machined without on-site teaching.

[0010] According to the robot machining method, the workpiece can be machined automatically without on-site teaching.

[0011] According to the machining program, the workpiece can be machined automatically without on-site instruction. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram showing the configuration of a robot system. [Figure 2] FIG. 2 is a diagram illustrating a schematic hardware configuration of the robot control device. [Figure 3] FIG. 3 is a diagram illustrating a schematic hardware configuration of the operation control device. [Figure 4] FIG. 4 is a diagram illustrating a schematic hardware configuration of the control device. [Figure 5] FIG. 5 is a block diagram showing the configuration of a control system for manual control of the robot system. [Figure 6] FIG. 6 is a block diagram showing the configuration of a control system for automatic control of a robot system. [Figure 7] FIG. 7 is a schematic diagram of the processed portion and the target trajectory. [Figure 8] FIG. 8 is a flowchart of the automatic control of the robot system. [Figure 9] FIG. 9 shows the first pattern of the target trajectory. [Figure 10] FIG. 10 shows the second pattern of the target trajectory. [Figure 11] FIG. 11 is an example of an image of an object. [Figure 12] FIG. 12 is an example of three-dimensional information of an object. [Figure 13] FIG. 13 is a schematic diagram of the trajectory of the grinding device in the removal process. DETAILED DESCRIPTION OF THE INVENTION

[0013] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Exemplary embodiments will now be described in detail with reference to the accompanying drawings. Fig. 1 is a schematic diagram showing the configuration of a robot system 100 according to an embodiment.

[0014] The robot system 100 includes a robot 1 that processes a processing portion B of an object W, and a control device 3 that controls the robot 1. The control device 3 controls the robot 1 to cause the robot 1 to process the processing portion B of the object W. In this example, the object W is a casting, and the processing portion B is a burr of the object W. Burrs include casting burrs, cutting burrs, grinding burrs, shear burrs, plastic deformation burrs, gate burrs, welding burrs, etc. The object W also has a reference plane R. The reference plane R is a surface on which the processing portion B exists. In other words, the processing portion B is located on the reference plane R.

[0015] The robot 1 is, for example, an industrial robot. The processing performed by the robot 1 is removal processing. The removal processing performed by the robot 1 is, for example, grinding. Note that the removal processing may also be cutting or polishing.

[0016] The robot system 100 includes a storage unit 32 that stores an image and three-dimensional information of the object W. The storage unit 32 is built into the control device 3. The image of the object W is, for example, a two-dimensional image of the object W. The three-dimensional information of the object W is, for example, point cloud data of the object W.

[0017] The robot system 100 may further include an imaging device 81 that acquires an image of the object W and a three-dimensional scanner 82 that acquires three-dimensional information of the object W. The three-dimensional scanner 82 is an example of a three-dimensional information acquisition device. The storage unit 32 stores the image of the object W acquired by the imaging device 81 and the three-dimensional information of the object W acquired by the three-dimensional scanner 82.

[0018] The robot system 100 includes a designation device 9 for designating a processing portion B from within an image of the object W. Furthermore, the designation device 9 is configured to be able to designate a reference plane R in addition to the processing portion B from within the image of the object W. The designation device 9 is a device operated by an operator. The designation device 9 has a display 91 and an input device 92. The input device 92 is, for example, a mouse. The designation device 9 is capable of communicating with the control device 3, and causes the image of the object W stored in the memory unit 32 to be displayed on the display 91. The operator operates the input device 92 while viewing the display 91 to designate the processing portion B and the reference plane R from within the image of the object W. In other words, the designation device 9 receives designation of the processing portion B and the reference plane R in the image of the object W from the operator via the input device 92.

[0019] The control device 3 derives the processing portion B in the three-dimensional information based on the portion specified by the specifying device 9 in the image of the object W and the three-dimensional information of the object W. The control device 3 causes the robot 1 to remove the processing portion B by operating the robot 1 based on the three-dimensional information of the processing portion B.

[0020] The robot system 100 may further include an operating device 2 that is operated by a user. The control device 3 also controls the operating device 2. The control device 3 controls the operation of the robot 1 in accordance with the operation of the operating device 2, and can also process the target object W. In other words, the robot system 100 can perform both automatic control by the robot 1 without using the operating device 2 and manual control by the robot 1 via the operating device 2.

[0021] [robot] The robot 1 has a base 10, a robot arm 12 supported by the base 10, an end effector 11 connected to the robot arm 12, and a robot control device 14 that controls the entire robot 1. The robot 1 operates, i.e., moves, the end effector 11 using the robot arm 12, and processes an object W using the end effector 11.

[0022] A robot coordinate system is defined for the robot 1. For example, in the robot coordinate system, three orthogonal axes are set: a Z axis extending in the vertical direction, an X axis perpendicular to the Z axis, and a Y axis perpendicular to both the Z axis and the X axis.

[0023] The end effector 11 has a grinding device 11a and applies grinding to the object W as an action. For example, the grinding device 11a is a grinder. The grinder may be a type that rotates a disk-shaped grinding wheel, a type that rotates a conical or cylindrical grinding wheel, or the like. The grinding device 11a may be an orbital sander, a random orbit sander, a delta sander, a belt sander, or the like. Here, the grinding device 11a is an example of a tool.

[0024] The robot arm 12 is a vertical articulated robot arm. The robot arm 12 has a plurality of links 12a, joints 12b connecting the plurality of links 12a, and a servo motor 15 (see FIG. 2) that rotates and drives the plurality of joints 12b. The robot arm 12 changes the position of the grinding device 11a. Furthermore, the robot arm 12 may change the posture of the grinding device 11a. The robot arm 12 may be a horizontal articulated robot arm, a parallel link robot arm, a rectangular coordinate robot arm, a polar coordinate robot arm, or the like.

[0025] The robot 1 has a force sensor. In this example, the robot 1 further has a contact force sensor 13 as a force sensor that detects a reaction force (hereinafter referred to as a "contact force") received from an object W. The contact force sensor 13 is provided between the robot arm 12 and the end effector 11 (specifically, at the connection between the robot arm 12 and the end effector 11). The contact force sensor 13 detects the contact force that the end effector 11 receives from the object W. The contact force sensor 13 detects forces in three orthogonal axial directions and moments around the three axes.

[0026] The force sensor is not limited to the contact force sensor 13. For example, the contact force sensor 13 may detect forces only in one, two, or three axis directions. Alternatively, the force sensor may be a current sensor that detects the current of the servo motor 15 of the robot arm 12, a torque sensor that detects the torque of the servo motor 15, or the like.

[0027] The imaging device 81 is attached to the robot arm 12. Specifically, the imaging device 81 is attached to the link 12a at the most distal end of the robot arm 12. The imaging device 81 captures RGB images. The captured image of the imaging device 81 is input as an image signal from the robot control device 14 to the control device 3.

[0028] The three-dimensional scanner 82 is attached to the robot arm 12. Specifically, the three-dimensional scanner 82 is attached to the link 12a at the tip end of the robot arm 12. The three-dimensional scanner 82 acquires point cloud data of the object W as three-dimensional information. In other words, the three-dimensional scanner 82 outputs the three-dimensional coordinates of a large number of point clouds on the surface of the object W. The point cloud data of the three-dimensional scanner 82 is input from the robot control device 14 to the control device 3.

[0029] 2 is a diagram showing a schematic hardware configuration of the robot control device 14. The robot control device 14 controls the servo motor 15 of the robot arm 12 and the grinding device 11a. The robot control device 14 receives a detection signal from the contact force sensor 13. The robot control device 14 transmits and receives information, commands, data, etc. to and from the control device 3. The robot control device 14 has a control unit 16, a storage unit 17, and a memory 18.

[0030] The control unit 16 controls the entire robot control device 14. The control unit 16 performs various types of arithmetic processing. For example, the control unit 16 is formed of a processor such as a CPU (Central Processing Unit). The control unit 16 may also be formed of an MCU (Micro Controller Unit), an MPU (Micro Processor Unit), an FPGA (Field Programmable Gate Array), a PLC (Programmable Logic Controller), a system LSI, or the like.

[0031] The storage unit 17 stores various data and programs executed by the control unit 16. The storage unit 17 is formed of a nonvolatile memory, a hard disc drive (HDD), a solid state drive (SSD), or the like.

[0032] The memory 18 temporarily stores data, etc. For example, the memory 18 is formed of a volatile memory.

[0033] [Operation device] 1, the operation device 2 has an operation unit 21 operated by a user, and an operation force sensor 23 that detects the operation force applied by the user to the operation unit 21. The operation device 2 accepts input for manually operating the robot 1, and outputs operation information, which is the input information, to the control device 3. Specifically, the user grips the operation unit 21 to operate the operation device 2. The force applied to the operation unit 21 at that time is detected by the operation force sensor 23. The operation force detected by the operation force sensor 23 is output to the control device 3 as operation information.

[0034] The operation device 2 may further include a base 20, a support mechanism 22 provided on the base 20 and supporting the operation unit 21, and an operation control device 24 that controls the entire operation device 2. The operation device 2 presents the user with a reaction force against the operation force under the control of the control device 3. Specifically, the operation control device 24 receives a command from the control device 3 and controls the support mechanism 22, thereby causing the user to sense the reaction force.

[0035] An operation coordinate system with three orthogonal axes is defined for the operation device 2. The operation coordinate system corresponds to the robot coordinate system. That is, the Z axis is set in the vertical direction, and the X and Y axes, which are orthogonal to each other, are set in the horizontal direction.

[0036] The support mechanism 22 has a plurality of links 22a, joints 22b connecting the plurality of links 22a, and a servo motor 25 (see FIG. 3) that rotationally drives the plurality of joints 22b. The support mechanism 22 supports the operation unit 21 so that the operation unit 21 can assume any position and posture in three-dimensional space. The servo motor 25 rotates in accordance with the position and posture of the operation unit 21. The amount of rotation of the servo motor 25, i.e., the rotation angle, is uniquely determined.

[0037] In this example, the operating force sensor 23 is provided between the operating unit 21 and the support mechanism 22 (specifically, at the connection between the operating unit 21 and the support mechanism 22). The operating force sensor 23 detects forces in three orthogonal axial directions and moments around the three axes.

[0038] The detection unit for the operating force is not limited to the operating force sensor 23. For example, the operating force sensor 23 may detect only one-axis, two-axis, or three-axis force. Alternatively, the detection unit may be a current sensor that detects the current of the servo motor 25 of the support mechanism 22, a torque sensor that detects the torque of the servo motor 25, or the like.

[0039] 3 is a diagram showing a schematic hardware configuration of the operation control device 24. The operation control device 24 operates the support mechanism 22 by controlling the servo motor 25. The operation control device 24 receives a detection signal from the operation force sensor 23. The operation control device 24 transmits and receives information, commands, data, etc. to and from the control device 3. The operation control device 24 has a control unit 26, a storage unit 27, and a memory 28.

[0040] The control unit 26 controls the entire operation control device 24. The control unit 26 performs various types of arithmetic processing. For example, the control unit 26 is formed of a processor such as a CPU (Central Processing Unit). The control unit 26 may also be formed of an MCU (Micro Controller Unit), an MPU (Micro Processor Unit), an FPGA (Field Programmable Gate Array), a PLC (Programmable Logic Controller), a system LSI, or the like.

[0041] The storage unit 27 stores various data and programs executed by the control unit 26. The storage unit 27 is formed of a nonvolatile memory, a hard disc drive (HDD), a solid state drive (SSD), or the like.

[0042] The memory 28 temporarily stores data, etc. For example, the memory 28 is formed of a volatile memory.

[0043] [Control device] The control device 3 controls the entire robot system 100 and controls the operations of the robot 1 and the operation device 2. Specifically, the control device 3 performs manual control of the robot system 100 in response to user operation and automatic control of the robot system 100. In manual control, the control device 3 performs master-slave control, specifically bilateral control, between the robot 1 and the operation device 2. The operation device 2 functions as the master device, and the robot 1 functions as the slave device. The control device 3 controls the operation of the robot 1 in response to the operation of the operation device 2 operated by the user, and also controls the operation of the operation device 2 so as to present to the user a reaction force in response to the detection result of the contact force sensor 13. In other words, the grinding device 11a processes the target object W in response to the user's operation, and the reaction force during processing is presented to the user via the operation device 2. In automatic control, the control device 3 receives a designation of a processing area B in an image of the target object W from the user, and automatically removes and processes the designated processing area B using the grinding device 11a.

[0044] 4 is a diagram showing a schematic hardware configuration of the control device 3. The control device 3 transmits and receives information, commands, data, etc. to and from the robot control device 14 and the operation control device 24. Furthermore, the control device 3 transmits and receives information, commands, data, etc. to and from the designation device 9. The control device 3 has a control unit 31, a storage unit 32, and a memory 33. The control device 3 may further have an input operation unit operated by the user to set the operation control of the robot 1 and the operation device 2, and a display that displays the setting contents.

[0045] The control unit 31 controls the entire control device 3. The control unit 31 performs various types of arithmetic processing. For example, the control unit 31 is formed of a processor such as a CPU (Central Processing Unit). The control unit 31 may also be formed of an MCU (Micro Controller Unit), an MPU (Micro Processor Unit), an FPGA (Field Programmable Gate Array), a PLC (Programmable Logic Controller), a system LSI, or the like.

[0046] The memory unit 32 stores various data and programs executed by the control unit 31. For example, the memory unit 32 stores a program for controlling the robot system 100. The memory unit 32 is formed of a non-volatile memory, a hard disk drive (HDD), a solid state drive (SSD), or the like. For example, the program stored in the memory unit 32 is a processing program 32a that causes a computer to execute a predetermined procedure to remove and process the processing portion B of the target object W.

[0047] The memory 33 temporarily stores data, etc. For example, the memory 33 is formed of a volatile memory.

[0048] <Robot system control> In the robot system 100 configured in this manner, the control device 3 controls the operation of the robot 1 in response to the operation of the operation device 2 operated by the user, and also performs manual control to control the operation of the operation device 2 so as to present to the user a reaction force in response to the detection result of the contact force sensor 13. Furthermore, the control device 3 identifies the processing portion B based on the image and three-dimensional information of the target object W, and performs automatic control to remove the identified processing portion B by the robot 1.

[0049] First, a description will be given of the manual control of the robot system 100. Fig. 5 is a block diagram showing the configuration of a control system for manual control of the robot system 100.

[0050] The control unit 16 of the robot control device 14 realizes various functions by reading out a program from the storage unit 17 into the memory 18 and developing it. Specifically, the control unit 16 functions as an input processing unit 41 and an operation control unit 42.

[0051] The input processing unit 41 outputs information, data, commands, etc. received from the contact force sensor 13 and the servo motor 15 to the control device 3. Specifically, the input processing unit 41 receives six-axis force detection signals (hereinafter referred to as "sensor signals") from the contact force sensor 13 and outputs the sensor signals to the control device 3. The input processing unit 41 also receives detection signals from a rotation sensor (e.g., an encoder) and a current sensor from the servo motor 15. The input processing unit 41 outputs the detection signals to the operation control unit 42 for feedback control of the robot arm 12 by the operation control unit 42. The input processing unit 41 also outputs the detection signals to the control device 3 as position information of the robot arm 12.

[0052] The operation control unit 42 receives a command position xds from the control device 3 and generates a control command for operating the robot arm 12 in accordance with the command position xds. The operation control unit 42 applies a current corresponding to the control command to the servo motor 15, thereby operating the robot arm 12 and moving the grinding device 11a to a position corresponding to the command position xds. At this time, the operation control unit 42 feedback-controls the operation of the robot arm 12 based on a detection signal from the rotation sensor or current sensor of the servo motor 15 received from the input processing unit 41. The operation control unit 42 also outputs a control command to the grinding device 11a to operate it. As a result, the grinding device 11a grinds the object W.

[0053] The control unit 26 of the operation control device 24 realizes various functions by reading out a program from the storage unit 27 into the memory 28 and developing it. Specifically, the control unit 26 functions as an input processing unit 51 and an operation control unit 52.

[0054] The input processing unit 51 outputs information, data, commands, etc. received from the operating force sensor 23 to the control device 3. Specifically, the input processing unit 51 receives six-axis force detection signals from the operating force sensor 23 and outputs the detection signals to the control device 3. The input processing unit 51 also receives detection signals from a rotation sensor (e.g., an encoder) and a current sensor from the servo motor 25. The input processing unit 51 outputs the detection signals to the operation control unit 52 for feedback control of the support mechanism 22 by the operation control unit 52.

[0055] The operation control unit 52 receives the command position xdm from the control device 3 and generates a control command for operating the support mechanism 22 in accordance with the command position xdm. The operation control unit 52 applies a current corresponding to the control command to the servo motor 25, thereby operating the support mechanism 22 and moving the operation unit 21 to a position corresponding to the command position xdm. At this time, the operation control unit 52 feedback-controls the operation of the support mechanism 22 based on a detection signal from the rotation sensor or current sensor of the servo motor 25 received from the input processing unit 51. This applies a reaction force to the operating force applied by the user to the operation unit 21. As a result, the user can operate the operation unit 21 while virtually feeling a reaction force from the object W from the operation unit 21.

[0056] The control unit 31 of the control device 3 realizes various functions by reading out a program from the storage unit 32 into the memory 33 and expanding the program. Specifically, the control unit 31 functions as an operation command unit 60 that outputs operation commands to the robot control device 14 and the operation control device 24. More specifically, the control unit 31 functions as an operation force acquisition unit 61, a contact force acquisition unit 62, an addition unit 63, a force / velocity conversion unit 64, a first velocity / position conversion unit 65, and a second velocity / position conversion unit 66.

[0057] The operating force acquiring unit 61 receives the detection signal of the operating force sensor 23 via the input processing unit 51, and acquires the operating force fm based on the detection signal. The operating force acquiring unit 61 inputs the operating force fm to the adding unit 63.

[0058] The contact force acquiring unit 62 receives the sensor signal of the contact force sensor 13 via the input processing unit 41, and acquires the contact force fs based on the sensor signal. The contact force acquiring unit 62 inputs the contact force fs to the adding unit 63.

[0059] The adder 63 calculates the sum of the operation force fm input from the operation force acquisition unit 61 and the contact force fs input from the contact force acquisition unit 62. Here, the operation force fm and the contact force fs are forces in opposite directions, and therefore the operation force fm and the contact force fs have opposite signs. In other words, by adding the operation force fm and the contact force fs, the absolute value of the resultant force fm+fs, which is the sum of the operation force fm and the contact force fs, becomes smaller than the absolute value of the operation force fm. The adder 63 outputs the resultant force fm+fs.

[0060] The force / velocity conversion unit 64 converts the input resultant force fm+fs into a command velocity xd'. The force / velocity conversion unit 64 calculates the command velocity xd' using a motion model based on a motion equation including an inertia coefficient, a viscosity coefficient (damper coefficient), and a stiffness coefficient (spring coefficient). Specifically, the force / velocity conversion unit 64 calculates the command velocity xd' based on the following motion equation:

[0061]

number

[0062] Equation (1) is a linear differential equation, and solving equation (1) for xd' gives equation (2).

[0063]

number

[0064] Equation (2) is stored in the memory unit 32. The force / velocity conversion unit 64 reads out equation (2) from the memory unit 32 to determine the command velocity xd′, and outputs the determined command velocity xd′ to the first velocity / position conversion unit 65 and the second velocity / position conversion unit 66.

[0065] The first speed / position conversion unit 65 converts the coordinate-converted command speed xd' into a command position xds for the robot 1, based on the robot coordinate system. For example, if the ratio of the movement amount of the robot 1 to the movement amount of the operating device 2 is set, the first speed / position conversion unit 65 multiplies the command position xd calculated from the command speed xd' in accordance with the movement ratio to calculate the command position xds. The first speed / position conversion unit 65 outputs the calculated command position xds to the robot control device 14, specifically, to the movement control unit 42. As described above, the movement control unit 42 operates the robot arm 12 based on the command position xds.

[0066] The second velocity / position conversion unit 66 converts the command velocity xd' into a command position xdm for the operation device 2, based on the operation coordinate system. The second velocity / position conversion unit 66 outputs the obtained command position xdm to the operation control device 24, specifically to the operation control unit 52. As described above, the operation control unit 52 operates the support mechanism 22 based on the command position xdm.

[0067] Next, a description will be given of the automatic control of the robot system 100. Fig. 6 is a block diagram showing the configuration of a control system for the automatic control of the robot system 100.

[0068] The control unit 31 of the control device 3 realizes various functions by reading out a program (e.g., a machining program 32a) from the storage unit 32 and expanding it in the memory 33. Specifically, the control unit 31 functions as an operation command unit 60, an imaging unit 67, a three-dimensional information acquisition unit 68, a derivation unit 69, and a trajectory generation unit 610.

[0069] The operation command unit 60 generates a command position xds for the robot arm 12 and outputs the generated command position xds to the robot control device 14. The robot control device 14 generates a control command for the servo motor 15 based on the command position xds from the operation command unit 60. The robot control device 14 applies a supply current corresponding to the control command to the servo motor 15. At this time, the robot control device 14 feedback controls the supply current to the servo motor 15 based on the detection result of the encoder.

[0070] For example, the operation command unit 60 generates a command position xds to move the imaging device 81 and the three-dimensional scanner 82 to a predetermined position, or to cause the grinding device 11a to perform grinding processing, and operates the robot arm 12.

[0071] The imaging section 67 controls the imaging device 81 to cause the imaging device 81 to capture an image of the object W. The imaging section 67 causes the image acquired by the imaging device 81 to be stored in the storage section 32.

[0072] The three-dimensional information acquisition unit 68 controls the three-dimensional scanner 82 to cause the three-dimensional scanner 82 to acquire point cloud data of the object W. The three-dimensional information acquisition unit 68 stores the point cloud data acquired by the three-dimensional scanner 82 in the storage unit 32. Note that if the coordinates of each point included in the point cloud data output from the three-dimensional scanner 82 are not in the robot coordinate system, the three-dimensional information acquisition unit 68 converts the coordinates of each point included in the point cloud data into the robot coordinate system.

[0073] The derivation unit 69 derives the processing portion B in the three-dimensional information based on the designation of the processing portion B in the image of the object W by the designation device 9. In addition, the derivation unit 69 derives the reference plane R in the three-dimensional information of the object W based on the designation of the reference plane R in the image of the object W by the designation device 9.

[0074] In detail, in response to a request from the designation device 9, the derivation unit 69 reads out an image of the object W from the storage unit 32 and provides it to the designation device 9. The provided image of the object W is displayed on the display 91 of the designation device 9. The operator operates the input device 92 to designate the processing portion B in the image of the object W. In addition, the operator operates the input device 92 to designate the reference plane R in the image of the object W. The derivation unit 69 receives the designation of the processing portion B and the reference plane R in the image of the object W from the designation device 9.

[0075] The derivation unit 69 compares the image of the object W, in which the processing portion B and reference surface R are specified, with the point cloud data of the object W stored in the memory unit 32, and derives the processing portion B and reference surface R in the point cloud data.

[0076] Specifically, since the position of the imaging device 81 when the image of the object W is acquired and the position of the three-dimensional scanner 82 when the point cloud data of the object W is acquired are known, it is generally possible to determine which part of the point cloud data of the object W corresponds to which part in the image of the object W. The derivation unit 69 identifies a part of the point cloud data of the object W that corresponds to the processing part B specified in the image of the object W, and designates the part of the identified part that protrudes compared to the surrounding area as the processing part B. The derivation unit 69 also identifies a part of the point cloud data of the object W that corresponds to the reference plane R specified in the image of the object W, and designates the surface including the identified part as the reference plane R. For example, the reference plane R is a smooth surface with few irregularities, and may be a flat or curved surface. In this way, the derivation unit 69 derives the processing part B and the reference plane R in the point cloud data of the object W.

[0077] The trajectory generating unit 610 generates a target trajectory for the grinding device 11a, i.e., a target trajectory for the robot arm 12, based on the point cloud data of the target object W. The target trajectory is a trajectory along the reference surface R, more specifically, a trajectory that is approximately parallel to the reference surface R. The target trajectories can be generated in multiple layers. The multiple target trajectories are arranged at intervals in the normal direction of the reference surface R. The multiple target trajectories can include a final target trajectory that passes over the reference surface R.

[0078] FIG. 7 is a schematic diagram of the machining portion B and the target trajectory. Specifically, the trajectory generating unit 610 determines the start position S of the grinding device 11a in the removal process based on point cloud data of the machining portion B. The trajectory generating unit 610 determines the highest point M of the machining portion B in the point cloud data that is farthest from the reference plane R, and then determines a point that is closer to the reference plane R from the highest point M by a predetermined cutting depth C in the normal direction of the reference plane R. The trajectory generating unit 610 determines a virtual first target machining plane that passes through the point that is closer to the reference plane R and is substantially parallel to the reference plane R, and determines a point on the first target machining plane that is not part of the machining portion B (i.e., a point away from the machining portion B) as the start position S. The trajectory generating unit 610 generates a target trajectory for the grinding device 11a as the first target trajectory T1, which starts from the start position S, passes through the first target machining plane, and passes through substantially the entire portion of the machining portion B that intersects with the first target machining plane. Next, the trajectory generating unit 610 sets a second target machining surface by moving the first target machining surface closer to the reference surface R by the depth of cut C in the normal direction of the reference surface R, and generates a target trajectory of the grinding device 11a as a second target trajectory T2, which passes over the second target machining surface and passes through almost the entire portion of the machining portion B that intersects with the second target machining surface. In this way, the trajectory generating unit 610 sequentially generates target trajectories at positions approaching the reference surface R by the depth of cut C in the normal direction of the reference surface R from the highest point M. When the target trajectory coincides with the reference surface R or is below the reference surface R, the trajectory generating unit 610 generates a target trajectory of the grinding device 11a as a final target trajectory Tf, which passes over the reference surface R and passes through almost the entire portion of the machining portion B that intersects with the reference surface R.

[0079] The number of target trajectories to be generated depends on the reference plane R, the highest point M, and the cutting depth C. The number of target trajectories is the quotient obtained by dividing the distance from the reference plane R to the highest point M by the cutting depth C and adding 1 to that quotient. If the distance from the reference plane R to the highest point is equal to or less than the cutting depth C, the number of target trajectories to be generated is one. In other words, the number of target trajectories is not limited to multiple.

[0080] The operation command unit 60 operates the robot 1 so that the grinding device 11a removes the processing portion B until it reaches the reference surface R. The operation command unit 60 operates the robot 1 so that the processing portion B is removed in multiple steps from the start position S toward the reference surface R. Specifically, the operation command unit 60 operates the robot 1 so that the grinding device 11a moves along the target trajectories, sequentially using a first target trajectory T1 that is farthest from the reference surface R to a final target trajectory Tf. For example, the operation command unit 60 removes the processing portion B in layers in multiple steps using the grinding device 11a. At this time, the operation command unit 60 performs position control to operate the robot 1 so that the grinding device 11a moves along the target trajectory, while also performing elasticity control to operate the robot 1 so that the grinding device 11a deviates from the target trajectory in response to a reaction force from the workpiece W and so that the pressing force of the grinding device 11a against the workpiece W increases in accordance with the distance from the target trajectory.

[0081] More specifically, the operation command unit 60 functions as a contact force acquisition unit 62, a force / velocity conversion unit 64, and a first velocity / position conversion unit 65. The functions of the contact force acquisition unit 62, the force / velocity conversion unit 64, and the first velocity / position conversion unit 65 are basically the same as those in the case of manual control. In automatic control, position control based on a target trajectory is the basis, so the operation command unit 60 does not function as the operating force acquisition unit 61, the addition unit 63, or the second velocity / position conversion unit 66.

[0082] The contact force acquisition unit 62 receives a sensor signal from the contact force sensor 13 via the input processing unit 41 and acquires the contact force fs based on the sensor signal. The contact force acquisition unit 62 inputs the contact force fs to the force / velocity conversion unit 64. The contact force acquisition unit 62 also stores the contact force fs in the memory unit 32 during grinding.

[0083] The force / velocity conversion unit 64 converts the input contact force fs into a command velocity xd'. The force / velocity conversion unit 64 calculates the command velocity xd' using a motion model based on a motion equation including an inertia coefficient, a viscosity coefficient (damper coefficient), and a stiffness coefficient (spring coefficient). Specifically, the force / velocity conversion unit 64 calculates the command velocity xd' based on the motion equation of equation (1). In equation (1), e = xd - xu, where xd is the command position and xu is the target trajectory generated by the trajectory generation unit 610. The force / velocity conversion unit 64 converts the target trajectory xu into a target velocity xu' and substitutes it into equation (2) to obtain the command velocity xd'.

[0084] The first speed / position conversion unit 65 converts the coordinate-converted command speed xd' into a command position xds for the robot 1, based on the robot coordinate system. The first speed / position conversion unit 65 outputs the determined command position xds to the robot control device 14, specifically to the operation control unit 42. As described above, the operation control unit 42 operates the robot arm 12 based on the command position xds. The first speed / position conversion unit 65 stores the command position xds in the memory unit 32 during grinding.

[0085] Since the motion model of equation (1) includes a viscosity coefficient cd and a stiffness coefficient kd, the grinding device 11a is basically controlled to move along the target trajectory xu. However, if resistance exists on the target trajectory xu, the grinding device 11a moves along a trajectory in which the elastic force and the damping force cooperate to avoid the resistance and apply a pressing force to the resistance. As a result, the grinding device 11a grinds the portion of the processing area B that is located on the target trajectory. At this time, the grinding device 11a, and ultimately the robot arm 12, are prevented from receiving excessive reaction force from the target object W.

[0086] When multiple target trajectories have been generated, the operation command unit 60 moves the grinding device 11a along the target trajectories, using them in order from the target trajectory farthest from the reference plane R. In other words, the grinding device 11a performs grinding along target trajectories that gradually come closer to the reference plane R, and finally performs grinding along a final target trajectory Tf that coincides with the reference plane R.

[0087] In the automatic control, the control device 3 does not generate or output a command position xdm for the operation device 2. In other words, the operation device 2 does not control the position of the operation unit 21.

[0088] [Robot system operation] Next, the operation of the robot system 100 configured as above will be described.

[0089] <Manual Control> In manual control, the user operates the operation device 2 to cause the robot 1 to perform an actual task on the object W. For example, the user operates the operation device 2 to cause the robot 1 to perform a grinding process on the object W. The operation force applied by the user to the operation unit 21 as an operation via the operation device 2 is detected by the operation force sensor 23. The robot arm 12 is controlled in accordance with the operation force.

[0090] Specifically, when the user operates the operation device 2, the operation force sensor 23 detects the operation force applied by the user via the operation unit 21. At this time, the contact force sensor 13 of the robot 1 detects the contact force.

[0091] The operating force detected by the operating force sensor 23 is input as a detection signal to the control device 3 by the input processing unit 51. In the control device 3, the operating force acquisition unit 61 inputs the operating force fm based on the detection signal to the adder 63.

[0092] At this time, the contact force detected by the contact force sensor 13 is input as a sensor signal to the input processing unit 41. The sensor signal input to the input processing unit 41 is input to the contact force acquisition unit 62. The contact force acquisition unit 62 inputs the contact force fs based on the sensor signal to the addition unit 63.

[0093] The adder 63 inputs the resultant force fm+fs to the force / speed converter 64. The force / speed converter 64 uses the resultant force fm+fs to determine the command speed xd' based on equation (2).

[0094] For the robot 1, the first speed / position conversion unit 65 calculates a command position xds from the command speed xd'. The operation control unit 42 of the robot control device 14 operates the robot arm 12 in accordance with the command position xds, and controls the position of the grinding device 11a. As a result, a pressing force corresponding to the operating force fm is applied to the object W, and the object W is ground by the grinding device 11a.

[0095] Meanwhile, for the operation device 2, the second velocity / position conversion unit 66 calculates a command position xdm from the command velocity xd'. The operation control unit 52 of the operation control device 24 operates the support mechanism 22 in accordance with the command position xdm, and controls the position of the operation unit 21. As a result, the user senses a reaction force corresponding to the contact force fs.

[0096] When the user operates the operation device 2 in this manner, the robot 1 processes the object W.

[0097] Automatic Control Next, a description will be given of the automatic control operation of the robot system 100. FIG.

[0098] First, in step S1, initialization is performed. The operator performs initialization for automatic control via the designation device 9. The initialization is input from the designation device 9 to the control device 3. For example, the initialization includes input of the cutting depth C of the grinding device 11a and selection of a target trajectory pattern. The cutting depth C refers to the cutting depth. Regarding the target trajectory pattern, there are multiple possible patterns for the movement of the grinding device 11a on the target machining surface to form the target machining surface. The control device 3 has multiple target trajectory patterns. FIG. 9 shows a first target trajectory pattern, and FIG. 10 shows a second target trajectory pattern. The first pattern is a trajectory formed by repeatedly moving the grinding device 11a back and forth along one path (e.g., a path extending in the Y direction), then shifting the path in a direction intersecting the path (e.g., in the X direction), and then moving the grinding device 11a back and forth along the shifted path. The second pattern is a trajectory formed by the grinding device 11a repeatedly moving along one path (e.g., a path extending in the Y direction), then shifting the path in a direction intersecting the path (e.g., the X direction), and then moving back and forth along the shifted path. That is, in the first pattern, the grinding device 11a passes through one path twice, whereas in the second pattern, the grinding device 11a passes through one path once. The target machining surface may be a flat surface or a curved surface. The target path pattern is not limited to these, and may also be a trajectory in which the grinding device 11a moves spirally on the target machining surface.

[0099] After inputting the initial settings, the operator outputs an instruction to capture an image of the object W to the control device 3 via the designation device 9. Upon receiving the image capture instruction, the control device 3 acquires an image of the object W and also acquires point cloud data of the object W in step S2. Specifically, the operation command unit 60 moves the robot arm 12 so that the imaging device 81 and the three-dimensional scanner 82 are positioned at predetermined positions. Since the object W is placed at a predetermined position on the support table, the predetermined positions of the imaging device 81 and the three-dimensional scanner 82 are also determined in advance.

[0100] Thereafter, the imaging unit 67 causes the imaging device 81 to capture an image of the object W. The imaging unit 67 stores the image of the object W captured by the imaging device 81 in the storage unit 32. The three-dimensional information acquisition unit 68 causes the three-dimensional scanner 82 to acquire point cloud data of the object W. The three-dimensional scanner 82 acquires the point cloud data of the object W at approximately the same angle of view as the imaging device 81. The three-dimensional information acquisition unit 68 stores the point cloud data acquired by the three-dimensional scanner 82 in the storage unit 32.

[0101] Furthermore, if the position of the robot arm 12 when the imaging device 81 is located at a predetermined position is different from the position of the robot arm 12 when the three-dimensional scanner 82 is located at a predetermined position, the operation command unit 60 may move the robot arm 12 when imaging by the imaging device 81 and when acquiring point cloud data by the three-dimensional scanner 82.

[0102] Next, in step S3, the control device 3 receives designation of the processing portion B and the reference plane R in the image of the object W from the designation device 9. Step S3 corresponds to designating the processing portion B of the object W in the image of the object W. FIG. 11 is an example of an image of the object W.

[0103] Specifically, the derivation unit 69 reads out an image of the object W from the storage unit 32 and provides it to the designation device 9. The provided image of the object W is displayed on the display 91. The derivation unit 69 displays a frame F for designating the processing portion B and a point P for designating the reference plane R on the image of the object W. The operator operates the input device 92 to adjust the position and shape of the frame F so that the processing portion B in the image of the object W is included within the frame F. The operator designates the processing portion B in the image of the object W by confirming the position and shape of the frame F. The derivation unit 69 identifies the portion of the image of the object W within the frame F confirmed by the designation device 9 as a portion that includes at least the processing portion B.

[0104] The operator also operates the input device 92 to adjust the position of point P so that point P is located on the reference plane R in the image of the object W. By determining the position of point P, the operator specifies the reference plane R in the image of the object W. The derivation unit 69 identifies the portion of the image of the object W where point P determined by the designation device 9 is located as being part of the reference plane R.

[0105] Next, in step S4, the derivation unit 69 reads out the point cloud data of the object W from the storage unit 32, compares the image of the object W with the point cloud data, and derives the portion of the point cloud data that corresponds to the processing portion B specified in the image of the object W and the reference plane R. Step S4 corresponds to deriving the processing portion B in the three-dimensional information based on the specified portion in the image and the three-dimensional information of the object W. FIG. 12 is an example of the three-dimensional information of the object W.

[0106] In detail, the derivation unit 69 identifies a portion corresponding to a portion surrounded by a frame F in the image of the object W from the point cloud data of the object W, and sets a portion that protrudes compared to the surrounding area within a predetermined region including the identified portion as the processing portion B. The derivation unit 69 also identifies a portion corresponding to a point P in the image of the object W from the point cloud data of the object W, and sets a surface including the identified portion as the reference surface R. If the surface including the identified portion is flat, the reference surface R will be flat, and if the surface including the identified portion is curved, the reference surface R will be curved. In this way, the derivation unit 69 derives the processing portion B and the reference surface R in the point cloud data of the object W.

[0107] Next, in step S5, the trajectory generating unit 610 derives the start position S of the removal processing. As described above, the trajectory generating unit 610 obtains the highest point M of the processing portion B in the point cloud data, obtains a first target processing plane that passes through a point that is closer to the reference plane R from the highest point M by the cutting depth C in the normal direction of the reference plane R, and obtains a point that exists on the first target processing plane and is outside the processing portion B as the start position S.

[0108] Thereafter, the trajectory generating unit 610 generates a target trajectory in step S6. Step S6 corresponds to generating a target trajectory of the robot tool that passes through the processing portion of the object. The trajectory generating unit 610 generates, as a first target trajectory T1, a target trajectory of the grinding device 11a that starts from the start position S, passes over the first target processing surface, and passes through substantially the entire portion of the processing portion B that intersects with the first target processing surface. At this time, the trajectory generating unit 610 generates the target trajectory according to the target trajectory pattern set in the initial setting.

[0109] Next, as described above, the trajectory generating unit 610 sets a second target machining surface by moving the first target machining surface closer to the reference surface R by the cutting depth C in the normal direction of the reference surface R, and generates a second target trajectory that passes through the second target machining surface. The trajectory generating unit 610 repeats this operation until a final target trajectory Tf is generated on the reference surface R.

[0110] In this way, a plurality of target trajectories are generated that are arranged at intervals in the normal direction of the reference plane R and that extend along the reference plane R.

[0111] Next, in step S7, the operation command unit 60 operates the robot 1 to perform grinding. Step S7 corresponds to operating the robot 1 based on three-dimensional information about the processing portion B, thereby causing the robot 1 to remove and process the processing portion B. Step S7 also corresponds to performing position control to operate the robot so that the tool moves along a target trajectory, and, in parallel with the position control, performing elasticity control to operate the robot so that the tool deviates from the target trajectory in response to a reaction force from the object and the pressing force of the tool against the object increases in accordance with the distance from the target trajectory. First, the operation command unit 60 operates the robot arm 12 so that the grinding device 11a moves along the first target trajectory T1. At this time, the operation command unit 60 performs elasticity control in parallel with position control to move the grinding device 11a along the target trajectory. By the elasticity control, the grinding device 11a moves along a trajectory that applies an appropriate pressing force to the object W while deviating from the target trajectory so as to avoid an excessive reaction force from the object W. Note that the operation command unit 60 also executes inertia control and viscosity control of the robot arm 12 in addition to the elasticity control.

[0112] FIG. 13 is a schematic diagram of the trajectory of the grinding device 11a during removal processing. Specifically, as shown in FIG. 13, the grinding device 11a moves along a first target trajectory T1 in areas where the processing portion B does not exist. When the grinding device 11a comes into contact with the processing portion B, the reaction force from the target object W increases, and the grinding device 11a deviates from the first target trajectory T1 in a direction along the surface of the processing portion B under the influence of the viscosity coefficient cd. However, the grinding device 11a is also affected by the rigidity coefficient kd, and the pressing force applied to the processing portion B increases the further it moves away from the first target trajectory T1. In other words, the cutting depth increases in areas of the processing portion B that are farther away from the first target trajectory T1. On the other hand, in areas where the reaction force from the target object W is small, the grinding device 11a moves closer to the first target trajectory T1. As a result, in the area where the processing portion B exists, the grinding device 11a follows the first actual trajectory t1 between the first target trajectory T1 and the surface of the processing portion B, as shown by the dashed line in Figure 13, and grinds the processing portion B with an appropriate pressing force.

[0113] While the grinding device 11a moves along the first target trajectory T1 (including when it deviates from the first target trajectory T1), the operation command unit 60 stores the contact force fs and the command position xds in the memory unit 32. When one grinding cycle by the grinding device 11a along the first target trajectory T1 is completed, the operation command unit 60 reads the contact force fs and the command position xds during grinding from the memory unit 32 and calculates the standard deviation of the contact force fs during grinding and the standard deviation of the command position xds during grinding. In step S8, the operation command unit 60 determines whether a completion condition for the grinding process has been met. For example, the completion condition is that parameters related to the removal process (i.e., grinding) have stabilized. Specifically, the parameters related to the removal processing are at least one of the contact force fs during grinding, the command position xd during grinding, the command velocity xd' during grinding, the acceleration xd'' of the grinding device 11a during grinding, and the current supplied to the servo motor 15 during grinding. In this example, the completion condition is that the standard deviation of the contact force fs during grinding is equal to or less than a predetermined first threshold value α, and the standard deviation of the command position xds during grinding is equal to or less than a predetermined second threshold value β.

[0114] In other words, if the processing portion B includes a portion that is significantly deviated from the first target trajectory T1, the contact force fs increases, and the standard deviation of the contact force fs during grinding also increases. Since the position of the grinding device 11a at that time also deviates significantly from the first target trajectory T1, the standard deviation of the command position xds during grinding also increases. When the contact force fs during grinding is equal to or less than the first threshold value α and the standard deviation of the command position xds during grinding is equal to or less than the second threshold value β, this means that the processing portion B has been ground into a shape that generally follows the first target trajectory T1.

[0115] If the completion condition is not satisfied, the processing portion B has not been ground to a shape corresponding to the first target trajectory T1. In this case, the operation command unit 60 returns to step S7 and again operates the robot arm 12 so that the grinding device 11a moves along the first target trajectory. By the first grinding process, the processing portion B has been ground to a shape that generally follows the first actual trajectory t1. In the second grinding process, for example, in the area where the processing portion B exists, the grinding device 11a follows the second actual trajectory t2 between the first target trajectory T1 and the first actual trajectory t1, as shown by the two-dot chain line in FIG. 13, and grinds the processing portion B with an appropriate pressing force.

[0116] If the completion condition is not satisfied in step S8 during the second grinding process, the operation command unit 60 returns to step S7 and again operates the robot arm 12 so that the grinding device 11a moves along the first target trajectory. By the second grinding process, the processing portion B is ground to a shape that generally follows the second actual trajectory t2. During the third grinding process, for example, in the area where the processing portion B exists, the grinding device 11a follows the third actual trajectory t3, which is approximately identical to the first target trajectory T1 (shown by the dashed-dotted line in FIG. 13), and grinds the processing portion B with an appropriate pressing force. Note that when the reaction force from the workpiece W is small, the effect of elasticity control is small, and position control becomes dominant. Therefore, the grinding device 11a follows a trajectory close to the first target trajectory T1. That is, the grinding device 11a is prevented from grinding the object W beyond the first target trajectory T1, and the object W is machined into a desired shape.

[0117] If the completion condition is satisfied, in step S9, the operation command unit 60 determines whether or not the grinding device 11a has reached the reference plane R. In other words, the operation command unit 60 determines whether or not the target trajectory when the condition of step S8 is satisfied is the final target trajectory Tf.

[0118] If the grinding device 11a has not reached the reference plane R, the operation command unit 60 increases the cutting amount of the grinding device 11a in step S10. In other words, the operation command unit 60 switches the target trajectory to the next target trajectory (i.e., a target trajectory closer to the reference plane R).

[0119] The operation command unit 60 returns to step S7 and executes grinding along the new target trajectory. The operation command unit 60 repeats the movement of the grinding device 11a along the target trajectory until the completion condition is satisfied for the new target trajectory as well.

[0120] In this way, the operation command unit 60 moves the grinding device 11a along one target trajectory to perform removal processing, and then switches to the next target trajectory to perform removal processing if the completion condition is met, but moves the grinding device 11a again along the one target trajectory (i.e., the same target trajectory) to perform removal processing if the completion condition is not met. The operation command unit 60 repeats this process until the completion condition is met in the grinding processing along the final target trajectory Tf.

[0121] When the completion condition is satisfied in the grinding process along the final target trajectory Tf, the operation command unit 60 goes through step S9 and ends the automatic control.

[0122] If there are multiple processing portions B on the target object W, the processing from step S1 may be repeated as many times as the number of processing portions B. Alternatively, multiple processing portions B may be specified in step S2, and the processing from step S3 may be repeated as many times as the number of processing portions B.

[0123] If the processed portion B is not completely removed after the automatic control and remains, the processed portion B may be removed by manual control.

[0124] Thus, according to the automatic control of the robot system 100, the operator specifies the processing portion B and the reference plane R in the image of the object W via the designation device 9, and the control device 3 identifies the processing portion B in the object W, and the processing portion B is automatically removed by the robot 1. The operator does not need to teach the robot 1 on-site. Therefore, processing of the processing portion B by the robot 1 can be easily achieved.

[0125] Furthermore, the control device 3 identifies not only the processed portion B of the object W but also the reference surface R, so that excessive removal of the object W can be prevented.

[0126] At this time, the control device 3 can remove the processed portion B little by little by performing the grinding process of the processed portion B by the grinding device 11a moving along the reference surface R in multiple steps toward the reference surface R. This also makes it possible to prevent the target object W from being removed excessively.

[0127] Furthermore, in the robot system 100, an image and three-dimensional information of the object W are acquired by an imaging device 81 and a three-dimensional scanner 82, respectively, provided on the robot 1. Specifically, the imaging device 81 and the three-dimensional scanner 82 are provided on the robot arm 12. The robot arm 12 is equipped with a grinding device 11a and can move freely around the object W to perform grinding. In other words, the robot arm 12 can freely move the imaging device 81 and the three-dimensional scanner 82 around the object W, making it easy to acquire an image and three-dimensional information of the object W using the imaging device 81 and the three-dimensional scanner 82.

[0128] As described above, the robot system 100 comprises a robot 1 that removes and processes the processing portion B of the object W, a memory unit 32 that stores an image and three-dimensional information of the object W, a designation device 9 for designating the processing portion B from within the image, and a control device 3 that controls the robot 1. The control device 3 derives the processing portion B in the three-dimensional information based on the portion in the image designated by the designation device 9 and the three-dimensional information, and operates the robot 1 based on the three-dimensional information of the processing portion B, thereby causing the robot 1 to remove the processing portion B.

[0129] In other words, the processing method of the robot 1 includes specifying a processing portion B of the object W in an image of the object W, deriving the processing portion B in the three-dimensional information based on the specified portion in the image and the three-dimensional information of the object W, and operating the robot 1 based on the three-dimensional information of the processing portion B, thereby causing the robot 1 to remove and process the processing portion B.

[0130] In addition, the processing program 32a causes the computer to receive a specification of the processing portion B in an image of the object W, derive the processing portion B in the three-dimensional information based on the specified portion in the image and the three-dimensional information of the object W, and operate the robot 1 based on the three-dimensional information of the processing portion B, thereby causing the robot 1 to remove and process the processing portion B, in order to cause the robot 1 to remove and process the processing portion B.

[0131] With these configurations, the operator does not need to teach the robot 1 on-site, as long as he or she specifies the processing portion B in the image of the object W. The control device 3 receives the specification of the processing portion B in the image of the object W, identifies the position of the processing portion B on the object W, and the processing portion B is automatically removed by the robot 1. As a result, the removal processing of the processing portion B by the robot 1 is simplified.

[0132] In addition, based on the part in the image specified by the specifying device 9 and the three-dimensional information, the control device 3 derives the processing part B in the three-dimensional information and the reference plane R of the object W where the processing part B exists, and causes the robot 1 to remove the processing part B until it reaches the reference plane R.

[0133] According to this configuration, not only the processing portion B but also the reference plane R is specified in the image of the object W. The control device 3 derives the processing portion B and the reference plane R in the three-dimensional information based on the specification in the image of the object W and the three-dimensional information of the object W. By deriving the reference plane R, the control device 3 can achieve removal processing of the processing portion B up to the reference plane R. This makes it possible to prevent excessive removal processing of the object W.

[0134] Specifically, the control device 3 receives the designation of the reference plane R in addition to the processing portion B in the image from the designation device 9, and derives the reference plane R in the three-dimensional information based on the designation of the reference plane R in the image by the designation device 9.

[0135] According to this configuration, the control device 3 receives the operator's designation of the reference plane R in the image from the designation device 9. The control device 3 receives the direct designation from the operator and derives the reference plane R in the three-dimensional information, so that the reference plane R can be determined with high accuracy.

[0136] Furthermore, the control device 3 determines the start position S of the removal process based on the three-dimensional information of the part B to be processed.

[0137] According to this configuration, the control device 3 determines the start position S of the removal process based on three-dimensional information of the processing portion B. In other words, the operator only needs to specify the processing portion B in the image of the target object W, and does not need to specify the start position S. Naturally, the operator does not need to teach the robot 1 the start position S of the removal process on-site or the like.

[0138] Furthermore, the control device 3 operates the robot 1 so as to remove the processed portion B toward the reference surface R in multiple steps.

[0139] This configuration can reduce the reaction force acting on the robot 1 from the target object W. Also, by removing the processing portion B little by little, it is possible to prevent removal of portions that should not be removed.

[0140] The robot system 100 further includes an imaging device 81 that acquires an image of the object W and a three-dimensional scanner 82 (three-dimensional information acquisition device) that acquires three-dimensional information of the object W, and the memory unit 32 stores the image of the object W acquired by the imaging device 81 and the three-dimensional information of the object W acquired by the three-dimensional scanner 82.

[0141] According to this configuration, the image and three-dimensional information of the object W can be acquired by the imaging device 81 and the three-dimensional scanner 82 without the need to separately prepare the image and three-dimensional information of the object W.

[0142] The removal processing performed by the robot 1 is grinding, cutting, or polishing.

[0143] According to this configuration, the processed portion B of the object W is removed by grinding, cutting, or polishing.

[0144] Other Embodiments As described above, the above embodiment has been described as an example of the technology disclosed in this application. However, the technology of the present disclosure is not limited to this and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made as appropriate. Furthermore, the components described in the above embodiment can be combined to create new embodiments. Furthermore, the components described in the accompanying drawings and detailed description may include not only components essential for solving the problem, but also components that are not essential for solving the problem in order to exemplify the technology. Therefore, the fact that these non-essential components are described in the accompanying drawings or detailed description should not be interpreted as immediately determining that these non-essential components are essential.

[0145] For example, the robot 1 is not limited to one that can realize bilateral control. For example, the operation device 2 may be omitted.

[0146] The processing performed by the robot 1 is not limited to removal processing, but may be, for example, sealing processing.

[0147] The target object is not limited to a casting. Any workpiece can be the target object as long as it includes a processed portion. The processed portion is not limited to a burr. Any portion that needs to be processed can be the target object.

[0148] The imaging device 81 does not have to be provided on the robot arm 12. For example, the imaging device 81 may be fixed at a location separate from the robot 1. For example, the imaging device 81 may be separated from the robot 1 and placed above the target object W.

[0149] The three-dimensional scanner 82 does not have to be provided on the robot arm 12. For example, the three-dimensional scanner 82 may be fixed at a location separate from the robot 1. For example, the three-dimensional scanner 82 may be separated from the robot 1 and placed above the object W.

[0150] The three-dimensional information of the object is not limited to point cloud data. The three-dimensional information may be any information that represents the three-dimensional shape of the object. For example, the three-dimensional information may be a depth image.

[0151] The image and three-dimensional information of the object W are not limited to those acquired by the imaging device 81 and the three-dimensional scanner 82 provided in the robot 1. The image and three-dimensional information of the object W may be acquired in advance and stored in the storage unit 32 in advance.

[0152] The method of specifying the processing portion B and the reference plane R in the image of the object W is not limited to the above-mentioned method. The processing portion B in the image may be specified by a point P instead of a frame F. The control device 3 may obtain a portion in the three-dimensional information that corresponds to the point P in the image, and derive a portion that includes this portion and protrudes from the surrounding area as the processing portion B. Furthermore, the portion surrounding the processing portion B may be derived as the reference plane R.

[0153] Furthermore, the control device 3 may only receive designation of the processing portion B in the image via the designation device 9, without receiving direct designation of the reference plane R. In other words, the control device 3 may derive the processing portion B in the three-dimensional information based on the portion in the image of the object W designated by the designation device 9 and the three-dimensional information of the object W, and may also derive the surface surrounding the processing portion B as the reference plane R. In this way, the control device 3 derives the reference plane R in addition to the processing portion B by receiving designation of the processing portion B, even without receiving direct designation of the reference plane R.

[0154] The removal processing method is not limited to the above description. The control device 3 executes position control and elasticity control of the grinding device 11a, but elasticity control may not be executed. Furthermore, the control device 3 removes the processing portion B toward the reference surface R in multiple steps, but this is not limited to this. The control device 3 may generate only the final target trajectory Tf and perform the grinding processing along the final target trajectory Tf from the beginning.

[0155] Furthermore, the operation command unit 60 determines whether or not the grinding completion condition is satisfied when transitioning from one target trajectory to the next target trajectory, but this is not limiting. In other words, when grinding along one target trajectory is completed, the operation command unit 60 may transition to grinding along the next target trajectory without checking whether or not the completion condition is satisfied.

[0156] The completion condition is not limited to the above. For example, the completion condition may be that the standard deviation of the contact force fs during grinding is equal to or less than a predetermined first threshold value α. The completion condition may be that the standard deviation of the command position xds during grinding is equal to or less than a predetermined second threshold value β. The completion condition may be that at least one of the following is satisfied: the standard deviation of the contact force fs during grinding is equal to or less than the predetermined first threshold value α, and the standard deviation of the command position xds during grinding is equal to or less than a predetermined second threshold value β.

[0157] The control device 3 performs position control and elasticity control using the motion model expressed by Equation (1), but the position control and elasticity control are not limited to this. Position control and elasticity control using any model can be adopted as long as the position of the tool is controlled so as to move the tool along a target trajectory, and if the reaction force from the object to the tool is large, the tool deviates from the target trajectory and applies a pressing force to the tool against the object according to the distance from the target trajectory. [Explanation of symbols]

[0158] 100 Robot Systems 1. Robot 11a Grinding equipment (tools) 3. Control device 32 Storage section 32a Machining Program 60 Operation command section 610 Trajectory generator 81 Imaging device 82 Three-dimensional scanner (three-dimensional information acquisition device) 9 Designated device B Machining part R reference plane W Object

Claims

1. a robot that removes and processes the processed portion of the object; a storage unit for storing an image and three-dimensional information of the object; a designation device for receiving designation of the processing portion in the image from an operator and designating the processing portion from within the image; a control device for controlling the robot, The control device deriving the processed portion in the three-dimensional information and a reference plane of the object on which the processed portion exists, based on the portion in the image designated by the designation device and the three-dimensional information; A robot system that causes the robot to remove the processed portion until the robot reaches the reference surface by operating the robot based on the three-dimensional information of the processed portion.

2. 2. The robot system according to claim 1, The control device receiving designation of the reference surface in addition to the processed portion in the image from the designation device; A robot system that derives the reference plane in the three-dimensional information based on the designation of the reference plane in the image by the designation device.

3. 2. The robot system according to claim 1, The control device derives the processing portion in the three-dimensional information based on the portion in the image specified by the designation device and the three-dimensional information, and derives the surface surrounding the processing portion as the reference surface.

4. 4. The robot system according to claim 1, The control device is a robot system that determines a start position of the removal processing based on the three-dimensional information of the processing portion.

5. The robot system according to claim 4, The control device is a robot system that operates the robot so as to remove the processed portion toward the reference surface in multiple steps.

6. 6. The robot system according to claim 1, an imaging device for acquiring an image of the object; further comprising a three-dimensional information acquisition device for acquiring three-dimensional information of the object; The storage unit stores the image of the object acquired by the imaging device and the three-dimensional information of the object acquired by the three-dimensional information acquisition device.

7. 7. The robot system according to claim 1, A robot system in which the removal processing by the robot is grinding, cutting, or polishing.

8. designating a portion of the object to be processed in an image of the object based on an instruction from an operator; deriving the processed portion in the three-dimensional information and a reference surface of the object on which the processed portion exists, based on the specified portion in the image and three-dimensional information of the object; and causing the robot to remove the processed portion until the reference surface is reached by operating the robot based on the three-dimensional information of the processed portion.

9. To have the robot remove the processed part of the object, the computer receiving, from an operator, a designation of the processing portion in the image of the object; deriving the processed portion in the three-dimensional information and a reference surface of the object on which the processed portion exists, based on the specified portion in the image and three-dimensional information of the object; a machining program that causes the robot to remove the processed portion until it reaches the reference surface by operating the robot based on the three-dimensional information of the processed portion.

Images