Robot system, processing method of robot, and processing program

Abstract

A robot system is provided. The robot system (100) includes a robot (1) that processes a processing portion (B) of an object (W), a storage section (32) that holds an image of the object (W) and three-dimensional information, a specifying device (9) for specifying the processing portion (B) from the image, and a control device (3) that controls the robot (1). The control device (3) causes the robot (1) to process the processing portion (B) by deriving the processing portion (B) in the three-dimensional information based on the portion specified by the specifying device (9) in the image and the three-dimensional information, and causing the robot (1) to act based on the three-dimensional information of the processing portion (B).

Description

Technical Field

[0001] This invention relates to robot systems, robot machining methods, and machining procedures. Background Technology

[0002] Systems for machining workpieces using robots are well known to date. For example, Patent Document 1 discloses a robot system in which a robot holding a workpiece moves along a coarser teach point and then presses the workpiece relative to the tool in a desired pressing direction. That is, in this robot system, the workpiece moves along a generally coarser teach point while the tool is pressed against the workpiece with the desired force.

[0003] Patent Document 1: Japanese Patent Application Publication No. 06-289923 Summary of the Invention

[0004] However, in the robot system of Patent Document 1, even for coarser precision, a teaching point in response to the workpiece needs to be provided on-site. If on-site work is reduced, the machining performed by the robot becomes simpler.

[0005] In view of the above, the purpose of this invention is to process workpieces even without on-site teaching.

[0006] The robot system of the present invention includes a robot, a storage unit, a designating device, and a control device. The robot processes a processing part of an object. The storage unit stores an image and three-dimensional information of the object. The designating device is used to designate the processing part from the image. The control device controls the robot. The control device derives the processing part from the three-dimensional information based on the part designated by the designating device in the image and the three-dimensional information, and causes the robot to move according to the three-dimensional information of the processing part, thereby enabling the robot to process the processing part.

[0007] The robot processing method of the present invention includes the steps of specifying a processing portion of an object in an image of an object, deriving the processing portion in the three-dimensional information based on the specified portion in the image and the three-dimensional information of the object, and processing the processing portion by causing the robot to move according to the three-dimensional information of the processing portion.

[0008] The processing procedure of the present invention enables a computer to perform the following steps in order to enable a robot to process a processing portion of an object: specifying the processing portion in an image of the object; deriving the processing portion in the three-dimensional information based on the specified portion in the image and the three-dimensional information of the object; and enabling the robot to process the processing portion by moving the robot according to the three-dimensional information of the processing portion.

[0009] (Invention Effects)

[0010] According to the robot system, workpieces can be processed automatically even without on-site teaching.

[0011] According to the robot's processing method, it can automatically process workpieces even without on-site teaching.

[0012] According to the processing procedure, the workpiece can be processed automatically even without on-site teaching. Attached Figure Description

[0013] Figure 1 This is a schematic diagram showing the structure of the robot system.

[0014] Figure 2 This is a diagram showing the general hardware structure of the robot control device.

[0015] Figure 3 This is a diagram showing the general hardware structure of the operation control device.

[0016] Figure 4 This is a diagram showing the general hardware structure of the control device.

[0017] Figure 5 This is a block diagram showing the structure of the control system for manual control of a robot system.

[0018] Figure 6 This is a block diagram showing the structure of the automatic control system of the robot system.

[0019] Figure 7 It is a schematic diagram of the processing section and the target trajectory.

[0020] Figure 8 This is a flowchart of the automatic control of a robot system.

[0021] Figure 9 It is the first pattern of the target trajectory.

[0022] Figure 10 It is the second mode of the target trajectory.

[0023] Figure 11It is an example of an image of an object.

[0024] Figure 12 It is an example of the three-dimensional information of an object.

[0025] Figure 13 This is a schematic diagram of the trajectory of the grinding device during the removal process. Detailed Implementation

[0026] Hereinafter, the exemplary embodiments will be described in detail with reference to the accompanying drawings. Figure 1 A schematic diagram of the structure of the robot system 100 is shown.

[0027] The robot system 100 includes a robot 1 and a control device 3. The robot 1 processes the machining portion B of an object W, and the control device 3 controls the robot 1. The control device 3 controls the robot 1 to process the machining portion B of the object W. In this example, the object W is a casting, and the machining portion B is a burr on the object W. Burrs include casting burrs, cutting burrs, grinding burrs, shearing burrs, plastic deformation burrs, gating burrs, and welding burrs. Furthermore, the object W has a reference surface R. The reference surface R is the surface on which the machining portion B exists. That is, the machining portion B is located on the reference surface R.

[0028] Robot 1 is, for example, an industrial robot. The processing performed by robot 1 is a removal process. The removal process performed by robot 1 is, for example, grinding. Alternatively, the removal process can also be cutting or polishing.

[0029] The robot system 100 includes a storage unit 32 that holds images and three-dimensional information of an object W. The storage unit 32 is built into the control device 3. The image of the object W is, for example, a two-dimensional image of the object W. The three-dimensional information of the object W is, for example, point cloud data of the object W.

[0030] The robot system 100 may also include an imaging device 81 and a 3D scanner 82, wherein the imaging device 81 acquires an image of the object W, and the 3D scanner 82 acquires 3D information of the object W. The 3D scanner 82 is an example of a 3D information acquisition device. The storage unit 32 stores the image of the object W acquired by the imaging device 81 and the 3D information of the object W acquired by the 3D scanner 82.

[0031] The robot system 100 includes a designation device 9 that designates a machining portion B from an image of an object W. Furthermore, the designation device 9 is configured to designate not only the machining portion B from the image of the object W, but also a reference plane R. The designation device 9 is operated by an operator. The designation device 9 has a display 91 and an input device 92. The input device 92 is, for example, a mouse. The designation device 9 can communicate with a control device 3 to display an image of the object W stored in a storage unit 32 on the display 91. The operator, while viewing the display 91, operates the input device 92 to designate the machining portion B and the reference plane R from the image of the object W. In other words, the designation device 9 receives designations of the machining portion B and the reference plane R from the image of the object W from the operator via the input device 92.

[0032] The control device 3 derives the machining portion B from the three-dimensional information of the object W based on the portion specified by the designating device 9 in the image of the object W and the three-dimensional information of the object W. The control device 3 then moves the robot 1 according to the three-dimensional information of the machining portion B, thereby causing the robot 1 to remove the machining portion B.

[0033] The robot system 100 may also include an operating device 2, which is operated by a user. A control device 3 also controls the operating device 2. The control device 3 can also control the actions of the robot 1 based on the actions of the operating device 2 to process the object W. That is, the robot system 100 can implement both automatic control by the robot 1 without using the operating device 2 and manual control by the robot 1 via the operating device 2.

[0034] [robot]

[0035] Robot 1 has a base 10, a robot arm 12, an end effector 11, and a robot control device 14. The robot arm 12 is supported by the base 10, the end effector 11 is connected to the robot arm 12, and the robot control device 14 controls the entire robot 1. Robot 1 moves by causing the end effector 11 to move, and the end effector 11 processes the object W.

[0036] A robot coordinate system is defined for robot 1. For example, in the robot coordinate system, three orthogonal axes are defined, namely the Z-axis extending in the vertical direction, the X-axis perpendicular to the Z-axis, and the Y-axis perpendicular to both the Z-axis and the X-axis.

[0037] The end effector 11 has a grinding device 11a that applies grinding as an action to the object W. For example, the grinding device 11a is a grinding machine. The grinding machine can also be of the type that rotates a disc-shaped grinding wheel, a conical or cylindrical grinding wheel, etc. In addition, the grinding device 11a can also be a track sander, a random track sander, a delta sander, or a belt sander, etc. Here, the grinding device 11a is an example of a tool.

[0038] Robotic arm 12 is a vertical multi-joint robotic arm. Robotic arm 12 has multiple links 12a, joints 12b, and servo motors 15 (see reference). Figure 2 The joint 12b connects to multiple links 12a, and the servo motor 15 rotates to drive the multiple joints 12b. The robot arm 12 changes the position of the grinding device 11a. Furthermore, the robot arm 12 can also change the posture of the grinding device 11a. Additionally, the robot arm 12 can be a horizontal multi-joint robot arm, a parallel linkage robot arm, a Cartesian coordinate robot arm, or a polar coordinate robot arm, etc.

[0039] Robot 1 has a force sensor. In this example, robot 1, acting as a force sensor, also has a contact force sensor 13 that detects the reaction force (hereinafter referred to as "contact force") received from the object W. The contact force sensor 13 is disposed between the robot arm 12 and the end effector 11 (specifically, at the connection between the robot arm 12 and the end effector 11). The contact force sensor 13 detects the contact force received by the end effector 11 from the object W. The contact force sensor 13 detects forces in three orthogonal axial directions and moments about these three axes.

[0040] Furthermore, the force sensor is not limited to the contact force sensor 13. For example, the contact force sensor 13 may also detect forces in the direction of a single axis, dual axes, or three axes. Alternatively, the force sensor may be a current sensor that detects the current of the servo motor 15 of the robot arm 12, or a torque sensor that detects the torque of the servo motor 15, etc.

[0041] Imaging device 81 is mounted on robot arm 12. Specifically, imaging device 81 is mounted on link 12a at the foremost end of robot arm 12. Imaging device 81 captures RGB images. The captured images are input as image signals from robot control device 14 to control device 3.

[0042] A 3D scanner 82 is mounted on the robot arm 12. Specifically, the 3D scanner 82 is mounted on the foremost link 12a of the robot arm 12. The 3D scanner 82 acquires point cloud data of the object W as 3D information. That is, the 3D scanner 82 outputs the 3D coordinates of a large number of point clouds on the surface of the object W. The point cloud data of the 3D scanner 82 is input from the robot control device 14 to the control device 3.

[0043] Figure 2 This diagram illustrates the schematic hardware structure of the robot control device 14. The robot control device 14 controls the servo motor 15 of the robot arm 12 and the grinding device 11a. The robot control device 14 receives detection signals from the contact force sensor 13. The robot control device 14 transmits and receives information, commands, and data from the control device 3. The robot control device 14 includes a control unit 16, a storage unit 17, and a memory 18.

[0044] The control unit 16 controls the entire robot control device 14. The control unit 16 performs various calculations. For example, the control unit 16 is formed using a processor such as a CPU (Central Processing Unit). The control unit 16 can also be formed using a MCU (Microcontroller Unit), MPU (Microprocessor Unit), FPGA (Field Programmable Gate Array), PLC (Programmable Logic Controller), system LSI, etc.

[0045] The storage unit 17 stores programs and various data executed by the control unit 16. The storage unit 17 is formed using non-volatile memory, HDD (hard disk drive), or SSD (solid-state drive), etc.

[0046] Memory 18 temporarily stores data, etc. For example, memory 18 is formed using volatile memory.

[0047] [Operating Device]

[0048] like Figure 1 As shown, the operating device 2 includes an operating section 21 and an operating force sensor 23. The operating section 21 is operated by a user, and the operating force sensor 23 detects the operating force applied by the user to the operating section 21. The operating device 2 accepts input for manually controlling the robot 1 and outputs the input information, i.e., the operating information, to the control device 3. Specifically, the user holds the operating section 21 to operate the operating device 2. At that time, the operating force sensor 23 detects the force applied to the operating section 21. The operating force detected by the operating force sensor 23 is output as operating information to the control device 3.

[0049] The operating device 2 may also include a base 20, a support mechanism 22, and an operating control device 24. The support mechanism 22 is mounted on the base 20 and supports the operating part 21. The operating control device 24 controls the entire operating device 2. The operating device 2 provides feedback to the user regarding the reaction force of the operating force through control from the control device 3. Specifically, the operating control device 24 receives commands from the control device 3 and controls the support mechanism 22 to allow the user to perceive the reaction force.

[0050] A three-axis orthogonal operating coordinate system is defined for the operating device 2. The operating coordinate system corresponds to the robot coordinate system. That is, the Z-axis is set in the vertical direction, and the mutually orthogonal X-axis and Y-axis are set in the horizontal direction.

[0051] The support mechanism 22 has multiple links 22a, joints 22b and servo motors 25 (see reference). Figure 3 The joint 22b connects to multiple links 22a, and the servo motor 25 drives the multiple joints 22b to rotate. The support mechanism 22 supports the operating part 21 in a manner that allows the operating part 21 to be in any position and posture in three-dimensional space. The servo motor 25 rotates in response to the position and posture of the operating part 21. The amount of rotation of the servo motor 25, i.e., the rotation angle, is uniquely determined.

[0052] In this example, the operating force sensor 23 is disposed between the operating part 21 and the support mechanism 22 (specifically, at the connection between the operating part 21 and the support mechanism 22). The operating force sensor 23 detects forces in three orthogonal axial directions and torques around these three axes.

[0053] It should be noted that the force detection unit is not limited to the force sensor 23. For example, the force sensor 23 may also detect forces in the direction of a single axis, dual axis, or three axes. Alternatively, the detection unit may be a current sensor that detects the current of the servo motor 25 of the support mechanism 22, or a torque sensor that detects the torque of the servo motor 25, etc.

[0054] Figure 3 This diagram illustrates the schematic hardware structure of the operation control device 24. The operation control device 24 controls the servo motor 25 to move the support mechanism 22. The operation control device 24 receives detection signals from the force sensor 23. The operation control device 24 transmits and receives information, commands, and data from the control device 3. The operation control device 24 includes a control unit 26, a storage unit 27, and a memory 28.

[0055] The control unit 26 controls the entire operation control device 24. The control unit 26 performs various calculations. For example, the control unit 26 is formed using a processor such as a CPU (Central Processing Unit). The control unit 26 can also be formed using a MCU (Microcontroller Unit), MPU (Microprocessor Unit), FPGA (Field Programmable Gate Array), PLC (Programmable Logic Controller), system LSI, etc.

[0056] The storage unit 27 stores programs and various data executed by the control unit 26. The storage unit 27 is formed using non-volatile memory, HDD (hard disk drive), or SSD (solid-state drive), etc.

[0057] Memory 28 temporarily stores data, etc. For example, memory 28 is formed using volatile memory.

[0058] [Control Device]

[0059] Control device 3 controls the entire robot system 100, performing motion control on robot 1 and operating device 2. Specifically, control device 3 performs both manual and automatic control of robot system 100 in response to user operations. In manual control, control device 3 performs master-slave control between robot 1 and operating device 2, specifically, bilateral control. Operating device 2 acts as the master device, and robot 1 acts as the slave device. Control device 3 controls the movement of robot 1 based on the movement of operating device 2 caused by user operations, and simultaneously controls the movement of operating device 2 to prompt the user of the reaction force in response to the detection result of contact force sensor 13. That is, grinding device 11a processes object W according to user operations, and the reaction force during processing is prompted to the user via operating device 2. In automatic control, control device 3 receives the designation of processing part B from the user in the image of object W, and grinding device 11a automatically removes the designated processing part B.

[0060] Figure 4 This diagram illustrates the general hardware structure of the control device 3. The control device 3 exchanges information, commands, and data with the robot control device 14 and the operation control device 24. Furthermore, the control device 3 exchanges information, commands, and data with the designated device 9. The control device 3 includes a control unit 31, a storage unit 32, and a memory 33. Additionally, the control device 3 may also include an input operation unit and a display. The input operation unit is operated by the user to set the motion control of the robot 1 and the operation device 2, and the display shows the settings.

[0061] The control unit 31 controls the entire control device 3. The control unit 31 performs various calculations. For example, the control unit 31 is formed using a processor such as a CPU (Central Processing Unit). The control unit 31 can also be formed using a MCU (Microcontroller Unit), MPU (Microprocessor Unit), FPGA (Field Programmable Gate Array), PLC (Programmable Logic Controller), system LSI, etc.

[0062] Storage unit 32 stores programs and various data executed by control unit 31. For example, storage unit 32 stores programs that control robot system 100. Storage unit 32 is formed using non-volatile memory, HDD (hard disk drive), or SSD (solid-state drive). Storage unit 32 is a non-temporary tangible medium. For example, the program stored in storage unit 32 is a machining program 32a that causes the computer to execute a predetermined sequence to remove the machining part B of the workpiece W.

[0063] Memory 33 temporarily stores data, etc. For example, memory 33 is formed using volatile memory.

[0064] (Control of the robot system)

[0065] In the robot system 100 configured in this way, the control device 3 performs manual control, that is, it controls the movement of the robot 1 based on the movement of the operating device 2 caused by the user's operation, and at the same time controls the movement of the operating device 2 to prompt the user with the reaction force in response to the detection result of the contact force sensor 13. Furthermore, the control device 3 performs automatic control, that is, based on the image and three-dimensional information of the object W, the robot 1 removes the specific processing part B.

[0066] First, the manual control of the robot system 100 will be explained. Figure 5 This is a block diagram showing the structure of the control system for manual control of the robot system 100.

[0067] The control unit 16 of the robot control device 14 performs various functions by reading programs from the storage unit 17 into the memory 18 and expanding them. Specifically, the control unit 16 functions as the input processing unit 41 and the motion control unit 42.

[0068] The input processing unit 41 outputs information, data, and commands received from the contact force sensor 13 and the servo motor 15 to the control device 3. Specifically, the input processing unit 41 receives force detection signals (hereinafter referred to as "sensor signals") from the contact force sensor 13 along six axes and outputs these sensor signals to the control device 3. Furthermore, the input processing unit 41 receives detection signals from a rotation sensor (e.g., an encoder) and a current sensor from the servo motor 15. The input processing unit 41 outputs these detection signals to the motion control unit 42 for feedback control of the robot arm 12 performed by the motion control unit 42. Additionally, the input processing unit 41 outputs these detection signals as position information of the robot arm 12 to the control device 3.

[0069] The motion control unit 42 receives the command position xds from the control device 3 and generates a control command to move the robot arm 12 according to the command position xds. The motion control unit 42 moves the robot arm 12 by applying a current corresponding to the control command to the servo motor 15, causing the grinding device 11a to move towards the position corresponding to the command position xds. At this time, the motion control unit 42 provides feedback control of the robot arm 12's movement based on the detection signals from the rotation sensor or current sensor of the servo motor 15 from the input processing unit 41. Furthermore, the motion control unit 42 outputs a control command to the grinding device 11a, causing the grinding device 11a to move. Therefore, the grinding device 11a grinds the object W.

[0070] The control unit 26 of the operation control device 24 performs various functions by reading the program from the storage unit 27 into the memory 28 and expanding it. Specifically, the control unit 26 functions as the input processing unit 51 and the motion control unit 52.

[0071] The input processing unit 51 outputs information, data, and commands received from the operating force sensor 23 to the control device 3. Specifically, the input processing unit 51 receives force detection signals from the operating force sensor 23 for all six axes and outputs these detection signals to the control device 3. Furthermore, the input processing unit 51 receives detection signals from the servo motor 25 from a rotation sensor (e.g., an encoder) and a current sensor. The input processing unit 51 outputs these detection signals to the motion control unit 52 for feedback control of the support mechanism 22 performed by the motion control unit 52.

[0072] The motion control unit 52 receives a command position xdm from the control device 3 and generates a control command to move the support mechanism 22 according to the command position xdm. The motion control unit 52 moves the support mechanism 22 by applying a current corresponding to the control command to the servo motor 25, causing the operating unit 21 to move to the position corresponding to the command position xdm. At this time, the motion control unit 52 provides feedback control of the support mechanism 22's movement based on the detection signal from the rotation sensor or current sensor of the servo motor 25 from the input processing unit 51. Therefore, a reaction force is provided for the operating force applied to the operating unit 21 by the user. As a result, the user can operate the operating unit 21 while simultaneously sensing a reaction force from the object W.

[0073] The control unit 31 of the control device 3 performs various functions by reading programs from the storage unit 32 into the memory 33 and expanding them. Specifically, the control unit 31 functions as an action command unit 60 that outputs action commands to the robot control device 14 and the operation control device 24. More specifically, the control unit 31 functions as an operation force acquisition unit 61, a contact force acquisition unit 62, an addition unit 63, a force / velocity conversion unit 64, a first velocity / position conversion unit 65, and a second velocity / position conversion unit 66.

[0074] The operating force acquisition unit 61 receives the detection signal from the operating force sensor 23 via the input processing unit 51, and acquires the operating force fm based on the detection signal. The operating force acquisition unit 61 inputs the operating force fm into the addition unit 63.

[0075] The contact force acquisition unit 62 receives the sensor signal from the contact force sensor 13 via the input processing unit 41, and acquires the contact force fs based on the sensor signal. The contact force acquisition unit 62 inputs the contact force fs to the addition unit 63.

[0076] The addition unit 63 calculates the sum of the operating force fm input from the operating force acquisition unit 61 and the contact force fs input from the contact force acquisition unit 62. Here, since the operating force fm and the contact force fs are forces in opposite directions, they have different signs. That is, by adding the operating force fm and the contact force fs, the absolute value of the resultant force fm+fs is made less than the absolute value of the operating force fm. The addition unit 63 outputs the resultant force fm+fs.

[0077] The force / velocity conversion unit 64 converts the input resultant force fm+fs into a command velocity xd'. The force / velocity conversion unit 64 calculates the command velocity xd' using a motion model based on motion equations that include the inertia coefficient, viscosity coefficient (damping coefficient), and stiffness coefficient (spring coefficient). Specifically, the force / velocity conversion unit 64 calculates the command velocity xd' according to the following motion equations.

[0078]

Formula 1

[0079] md·e″+·cd·e′+kd·e=fm+fs…(1)

[0080] Here, e = xd - xu. xd is the command position. xu is the target trajectory, which will be discussed later. In manual control, since there is no target trajectory, e = xdmd, where md is the inertia coefficient, cd is the viscosity coefficient, kd is the stiffness coefficient, fm is the operating force, and fs is the contact force. Additionally, "'" indicates a single derivative, and "" indicates a double derivative.

[0081] Equation (1) is a linear differential equation. When solving for the velocity xd' in equation (1), it becomes equation (2).

[0082]

Formula 2

[0083] xd′=xu′+A…(2)

[0084] Here, A is a term represented by fm, fs, md, cd, kd, etc.

[0085] Equation (2) is stored in storage unit 32. Force / velocity conversion unit 64 reads equation (2) from storage unit 32 to calculate command velocity xd', and outputs the calculated command velocity xd' to first velocity / position conversion unit 65 and second velocity / position conversion unit 66.

[0086] The first velocity / position conversion unit 65 uses the robot coordinate system as a reference to convert the coordinate-transformed command velocity xd' into a command position xds for the robot 1. For example, when there is a ratio between the movement of the robot 1 and the movement of the operating device 2, the first velocity / position conversion unit 65 doubles the command position xd obtained from the command velocity xd' according to the movement ratio to obtain the command position xds. The first velocity / position conversion unit 65 outputs the obtained command position xds to the robot control device 14, specifically, to the motion control unit 42. As described above, the motion control unit 42 causes the robot arm 12 to move according to the command position xds.

[0087] The second speed / position conversion unit 66 uses the operating coordinate system as a reference to convert the command speed xd' into the command position xdm used by the operating device 2. The second speed / position conversion unit 66 outputs the calculated command position xdm to the operating control device 24, specifically, to the motion control unit 52. As described above, the motion control unit 42 causes the support mechanism 22 to operate according to the command position xdm.

[0088] Next, the automatic control of the robot system 100 will be explained. Figure 6This is a block diagram showing the structure of the automatic control system of the robot system 100.

[0089] The control unit 31 of the control device 3 performs various functions by reading a program (e.g., machining program 32a) from the storage unit 32 into the memory 33 and expanding it. Specifically, the control unit 31 functions as an action command unit 60, an imaging unit 67, a three-dimensional information acquisition unit 68, an export unit 69, and a trajectory generation unit 610.

[0090] The motion command unit 60 generates a command position xds for the robot arm 12 and outputs the generated command position xds to the robot control unit 14. Based on the command position xds from the motion command unit 60, the robot control unit 14 generates a control command for the servo motor 15. The robot control unit 14 applies a supply current corresponding to the control command to the servo motor 15. At this time, the robot control unit 14 controls the supply current to the servo motor 15 based on the encoder's detection results.

[0091] For example, the motion command unit 60 generates a command position xds to move the imaging device 81 and the 3D scanner 82 to a specified position, or to enable the grinding device 11a to perform grinding processing, thereby causing the robot arm 12 to move.

[0092] The imaging unit 67 controls the imaging device 81 to capture images of the target object W. The imaging unit 67 stores the images acquired by the imaging device 81 in the storage unit 32.

[0093] The 3D information acquisition unit 68 controls the 3D scanner 82 to acquire point cloud data of the object W. The 3D information acquisition unit 68 stores the point cloud data acquired by the 3D scanner 82 in the storage unit 32. Furthermore, when the coordinates of each point in the point cloud data output from the 3D scanner 82 are not in the robot coordinate system, the 3D information acquisition unit 68 converts the coordinates of each point in the point cloud data to the robot coordinate system.

[0094] The export unit 69 exports the processed portion B in the three-dimensional information based on the specified processed portion B in the image of the object W processed by the designated device 9. Furthermore, the export unit 69 exports the reference plane R in the three-dimensional information of the object W based on the specified reference plane R in the image of the object W processed by the designated device 9.

[0095] In detail, the export unit 69, in response to a request from the designated device 9, reads an image of the object W from the storage unit 32 and provides it to the designated device 9. The provided image of the object W is displayed on the display 91 of the designated device 9. The operator operates the input device 92 to specify the processing portion B in the image of the object W. Additionally, the operator operates the input device 92 to specify the reference plane R in the image of the object W. The export unit 69 receives the specifications of the processing portion and the reference plane R from the image of the object W from the designated device 9.

[0096] The export unit 69 compares the image of the object W to which the processing part B and the reference plane R are specified with the point cloud data of the object W stored in the storage unit 32, and exports the processing part B and the reference plane R in the point cloud data.

[0097] In detail, since the positions of the imaging device 81 when acquiring the image of the object W and the 3D scanner 82 when acquiring the point cloud data of the object W are known, it is roughly possible to determine which part of the point cloud data of the object W corresponds to a certain part of the image of the object W. The extraction unit 69 identifies the part in the image of the object W that corresponds to the designated processing part B from the point cloud data of the object W, and designates the part that protrudes from the surrounding area in the designated part as the processing part B. Furthermore, the extraction unit 69 identifies the part in the image of the object W that corresponds to the designated reference surface R from the point cloud data of the object W, and designates the surface containing the designated part as the reference surface R. For example, the reference surface R is a smooth surface with few bumps, and can be either a plane or a curved surface. In this way, the extraction unit 69 extracts the processing part B and the reference surface R from the point cloud data of the object W.

[0098] The trajectory generation unit 610 generates the target trajectory of the grinding device 11a, i.e., the target trajectory of the robot arm 12, based on the point cloud data of the object W. The target trajectory is a trajectory along the reference plane R; more specifically, it is a trajectory approximately parallel to the reference plane R. The target trajectory can be generated in multiple layers. These multiple target trajectories are arranged at intervals along the normal direction of the reference plane R. The multiple target trajectories may include a final target trajectory passing through the reference plane R.

[0099] Figure 7This is a schematic diagram of the machining section B and the target trajectory. Specifically, the trajectory generation unit 610 determines the starting position S of the grinding device 11a to be removed from the machining section B based on the point cloud data of the machining section B. The trajectory generation unit 610 finds the highest point M in the machining section B that is farthest from the reference surface R in the point cloud data, and finds the point that approaches the cutting amount C specified by the reference surface R in the normal direction from the highest point M towards the reference surface R. The trajectory generation unit 610 finds an imaginary first target machining surface that passes through this point and is approximately parallel to the reference surface R, and finds the point existing on the first target machining surface that is outside the machining section B (i.e., the point away from the machining section B) as the starting position S. The trajectory generation unit 610 generates the first target trajectory T1, which is the target trajectory of the grinding device 11a passing through approximately the entire portion of the machining section B that intersects with the first target machining surface, starting from the starting position S. Next, the trajectory generation unit 610 sets a second target machining surface such that the normal direction of the first target machining surface to the reference surface R approaches the infeed C of the reference surface R. It then generates a second target trajectory T2, which is the target trajectory of the grinding device 11a passing through approximately the entire portion of the machining section B that intersects the second target machining surface. In this manner, the trajectory generation unit 610 sequentially generates target trajectories at positions approaching the infeed C of the reference surface R in the normal direction from the highest point M. When the target trajectory coincides with the reference surface R, or when the target trajectory is below the reference surface R, the trajectory generation unit 610 generates a final target trajectory Tf, which is the target trajectory of the grinding device 11a passing through approximately the entire portion of the machining section B that intersects the reference surface R.

[0100] Furthermore, the number of target trajectories generated depends on the reference plane R, the highest point M, and the cutting depth C. The number of target trajectories is the quotient obtained by dividing the distance from the reference plane R to the highest point M by the cutting depth C, plus 1. When the distance from the reference plane R to the highest point is less than or equal to the cutting depth C, only one target trajectory is generated. In other words, the number of target trajectories is not limited to multiple.

[0101] The motion command unit 60 moves the robot 1 in such a way that the grinding device 11a removes the processed portion B until it reaches the reference surface R. The motion command unit 60 moves the robot 1 in such a way that the processed portion B is removed multiple times from the starting position S toward the reference surface R. Specifically, the motion command unit 60 moves the robot 1 in such a way that the grinding device 11a moves along the target trajectory sequentially from the first target trajectory T1 furthest from the reference surface R to the final target trajectory Tf. For example, the motion command unit 60 removes the processed portion B in layers multiple times using the grinding device 11a. At this time, the motion command unit 60 performs position control and elastic control. In the position control, the robot 1 is moved by the grinding device 11a moving along the target trajectory. In the elastic control, the robot 1 is moved by the grinding device 11a moving away from the target trajectory in response to the reaction force from the object W, and the pressing force of the grinding device 11a toward the object W increases in response to the distance from the target trajectory.

[0102] In detail, the motion command unit 60 functions as the contact force acquisition unit 62, the force / velocity conversion unit 64, and the first velocity / position conversion unit 65. The functions of each of the contact force acquisition unit 62, the force / velocity conversion unit 64, and the first velocity / position conversion unit 65 are essentially the same as in manual control. In automatic control, since position control based on the target trajectory becomes fundamental, the motion command unit 60 does not function as the operating force acquisition unit 61, the addition unit 63, or the second velocity / position conversion unit 66.

[0103] The contact force acquisition unit 62 receives the sensor signal from the contact force sensor 13 via the input processing unit 41 and acquires the contact force fs based on the sensor signal. The contact force acquisition unit 62 inputs the contact force fs to the force / velocity conversion unit 64. Furthermore, the contact force acquisition unit 62 stores the contact force fs in the storage unit 32 during the grinding process.

[0104] The force / velocity conversion unit 64 converts the input contact force fs into a command velocity xd'. The force / velocity conversion unit 64 calculates the command velocity xd' using a motion model based on motion equations that include the inertia coefficient, viscosity coefficient (damping coefficient), and stiffness coefficient (spring coefficient). Specifically, the force / velocity conversion unit 64 calculates the command velocity xd' according to the motion equation of equation (1). In equation (1), e = xd - xu, xd is the command position, and xu is the target trajectory generated by the trajectory generation unit 610. The force / velocity conversion unit 64 calculates the command velocity xd' by converting the target trajectory xu into a target velocity xu' and substituting it into equation (2).

[0105] The first speed / position conversion unit 65, using the robot coordinate system as a reference, converts the command speed xd' of the coordinate transformation into a command position xds for the robot 1. The first speed / position conversion unit 65 outputs the calculated command position xds to the robot control device 14; specifically, it outputs the calculated command position xds to the motion control unit 42. As described above, the motion control unit 42 moves the robot arm 12 according to the command position xds. During grinding, the first speed / position conversion unit 65 stores the command position xds in the storage unit 32.

[0106] Since the motion model of equation (1) includes the viscosity coefficient cd and the stiffness coefficient kd, the basic principle is to control the position of the grinding device 11a along the target trajectory xu. Simultaneously, when resistance exists on the target trajectory xu, the grinding device 11a moves along a trajectory where elastic force and damping force coordinate, avoiding the resistance while applying pressure to it. As a result, the grinding device 11a grinds the portion of the processing part B located on the target trajectory. In this way, it is possible to avoid the grinding device 11a and thus the robot arm 12 receiving excessive reaction force from the object W.

[0107] When multiple target trajectories are generated, the motion command unit 60 uses the target trajectory that is farther from the reference surface R in sequence, causing the grinding device 11a to move along the target trajectory. That is, the grinding device 11a grinds in stages along the target trajectory that is closer to the reference surface R, and finally grinds along the final target trajectory Tf that is consistent with the reference surface R.

[0108] Furthermore, in automatic control, the control device 3 does not generate or output the command position xdm used by the operating device 2. That is to say, the operating device 2 does not perform position control of the operating unit 21.

[0109] [The actions of the robot system]

[0110] Next, the operation of the robot system 100 configured in this way will be explained.

[0111] <Manual Control>

[0112] In manual control, the robot 1 performs actual work on the object W by operating the control device 2. For example, the user operates the control device 2 to have the robot 1 perform grinding on the object W. The force applied by the user to the operating part 21 via the operation device 2 is detected by the force sensor 23. The robot arm 12 is controlled in response to the force applied.

[0113] Specifically, when the user operates the operating device 2, the operating force sensor 23 detects the operating force applied by the user through the operating unit 21. At this time, the contact force sensor 13 of the robot 1 detects the contact force.

[0114] The operating force detected by the operating force sensor 23 is input as a detection signal to the control device 3 by the input processing unit 51. In the control device 3, the operating force acquisition unit 61 inputs the operating force fm based on the detection signal to the addition unit 63.

[0115] At this time, the contact force detected by the contact force sensor 13 is input as a sensor signal to the input processing unit 41. The sensor signal input to the input processing unit 41 is then input to the contact force acquisition unit 62. The contact force acquisition unit 62 inputs the contact force fs based on the sensor signal to the addition unit 63.

[0116] Addition unit 63 inputs the resultant force fm+fs into force / velocity conversion unit 64. Force / velocity conversion unit 64 uses the resultant force fm+fs to calculate the command velocity xd' according to formula (2).

[0117] Regarding robot 1, the first speed / position conversion unit 65 calculates the command position xds from the command speed xd'. The motion control unit 42 of robot control device 14 moves robot arm 12 according to the command position xds, controlling the position of grinding device 11a. Therefore, in response to the pressing force fm applied to object W, object W is simultaneously ground by grinding device 11a.

[0118] On the other hand, regarding the operating device 2, the second speed / position conversion unit 66 calculates the command position xdm from the command speed xd'. The action control unit 52 of the operating control device 24 actuates the support mechanism 22 according to the command position xdm, controlling the position of the operating unit 21. Therefore, the user perceives the reaction force in response to the contact force fs.

[0119] The user operates the device 2 to perform the processing of the object W by the robot 1.

[0120] Automatic Control

[0121] Next, the automatic control actions of the robot system 100 will be explained. Figure 8 This is a flowchart of the automatic control of robot system 100.

[0122] First, in step S1, initial settings are performed. The operator performs initial settings for automatic control via the designated device 9. The initial settings are input from the designated device 9 to the control device 3. For example, the initial settings include inputting the depth of cut C of the grinding device 11a and selecting the target trajectory mode. The depth of cut C refers to the depth of penetration. Regarding the target trajectory mode, various modes can be considered for the movement of the grinding device 11a on the target machining surface to form a target machining surface. The control device 3 has various target trajectory modes. Figure 9 It is the first pattern of the target trajectory. Figure 10 This is the second pattern of the target trajectory. The first pattern is formed by the grinding device 11a moving back and forth along a path (e.g., a path extending in the Y direction), then shifting that path in a direction intersecting the path (e.g., the X direction), and repeating this movement along the shifted path. The second pattern is formed by the grinding device 11a moving along a path (e.g., a path extending in the Y direction), then shifting that path in a direction intersecting the path (e.g., the X direction), and repeating this movement along the shifted path. In other words, in the first pattern, the grinding device 11a traverses a path twice, while in the second pattern, the grinding device 11a traverses a path once. Furthermore, the target machining surface can be either a plane or a curved surface. The target path pattern is not limited to these; it can also be a trajectory of the grinding device 11a moving helically on the target machining surface.

[0123] After the operator inputs the initial settings, the imaging instruction for the object W is output to the control device 3 via the designated device 9. Upon receiving the imaging instruction, the control device 3, in step S2, acquires the image of the object W and its point cloud data. Specifically, the motion command unit 60 moves the robot arm 12 to position the imaging device 81 and the 3D scanner 82 at predetermined locations. Since the object W is placed in a predetermined position on the support platform, the predetermined positions of the imaging device 81 and the 3D scanner 82 are also predetermined.

[0124] Then, the imaging unit 67 causes the imaging device 81 to capture an image of the object W. The imaging unit 67 stores the image of the object W acquired by the imaging device 81 in the storage unit 32. The 3D information acquisition unit 68 causes the 3D scanner 82 to acquire point cloud data of the object W. The 3D scanner 82 acquires the point cloud data of the object W from approximately the same perspective as the imaging device 81. The 3D information acquisition unit 68 stores the point cloud data acquired by the 3D scanner 82 in the storage unit 32.

[0125] Furthermore, when the position of the robot arm 12 when the imaging device 81 is in a specified position is different from the position of the robot arm 12 when the 3D scanner 82 is in a specified position, the motion command unit 60 can also move the robot arm 12 when the imaging device 81 takes a picture and when the 3D scanner 82 acquires point cloud data.

[0126] Next, in step S3, the control device 3 receives the designation of the processing portion B and the reference plane R from the designation device 9 in the image of the object W. Step S3 is equivalent to designating the processing portion B of the object W in the image of the object W. Figure 11 This is an example of an image of object W.

[0127] Specifically, the export unit 69 reads an image of the object W from the storage unit 32 and provides it to the designating device 9. The provided image of the object W is displayed on the display 91. The export unit 69 displays a frame F for designating the machining portion B and a point P for designating the reference plane R on the image of the object W. The operator operates the input device 92 to adjust the position and shape of the frame F so that the machining portion B in the image of the object W is contained within the frame F. The operator designates the machining portion B in the image of the object W by determining the position and shape of the frame F. The export unit 69 specifies the portion within the frame F in the image of the object W determined by the designating device 9 as the portion containing at least the machining portion B.

[0128] Furthermore, the operator operates the input device 92 to adjust the position of point P so that point P lies on the reference plane R in the image of the object W. The operator designates the reference plane R in the image of the object W by determining the position of point P. The export unit 69 designates the portion of the image of the object W where point P, determined by the designation device 9, lies as a part of the reference plane R.

[0129] Next, in step S4, the export unit 69 reads the point cloud data of the object W from the storage unit 32, compares the image of the object W with the point cloud data, and exports the portion in the point cloud data that corresponds to the processing portion B and the reference plane R specified in the image of the object W. Step S4 is equivalent to exporting the processing portion B in the three-dimensional information based on the specified portion in the image and the three-dimensional information of the object W. Figure 12 This is an example of the three-dimensional information of object W.

[0130] In detail, the export unit 69 identifies the portion from the point cloud data of the object W that corresponds to the portion enclosed by the frame F in the image of the object W, and designates the portion protruding from the surrounding area within a defined region containing the identified portion as the processing portion B. Furthermore, the export unit 69 identifies the portion from the point cloud data of the object W that corresponds to point P in the image of the object W, and designates the surface containing the identified portion as the reference surface R. If the surface containing the identified portion is a plane, the reference surface R becomes a plane; if the surface containing the identified portion is a curved surface, the reference surface R becomes a curved surface. In this way, the export unit 69 exports the processing portion B and the reference surface R from the point cloud data of the object W.

[0131] Next, the trajectory generation unit 610 derives the starting position S for the removal process in step S5. As described above, the trajectory generation unit 610 calculates the highest point M of the processing portion B in the point cloud data, calculates the first target processing surface through the point that approaches the reference surface R by the cutting amount C in the normal direction from the highest point M to the reference surface R, and calculates the starting position S as the point that exists on the first target processing surface and is outside the processing portion B.

[0132] Then, the trajectory generation unit 610 generates a target trajectory in step S6. Step S6 is equivalent to generating a target trajectory for the robot's tool that passes through the machining section of the object. The trajectory generation unit 610 generates a first target trajectory T1 as the target trajectory of the grinding device 11a that passes through approximately the entire portion of the machining section B that intersects with the first target machining surface, starting from the starting position S. At this time, the trajectory generation unit 610 generates the target trajectory according to the target trajectory pattern set in the initial settings.

[0133] Next, as described above, the trajectory generation unit 610 sets a second target machining surface such that the normal direction of the first target machining surface to the reference surface R is close to the cutting depth C of the reference surface R, and generates a second target trajectory passing through the second target machining surface. The trajectory generation unit 610 repeats this operation until a final target trajectory Tf is generated on the reference surface R.

[0134] In this way, multiple target trajectories are generated with spaced intervals along the normal direction to the reference plane R.

[0135] Next, in step S7, the motion command unit 60 causes the robot 1 to perform grinding. Step S7 is equivalent to causing the robot 1 to remove the machining part B by moving the robot 1 according to the three-dimensional information of the machining part B. Furthermore, step S7 is equivalent to performing position control and, in parallel with position control, elastic control. In the position control, the robot 1 is moved such that the tool moves along the target trajectory. In the elastic control, the robot 1 is moved such that the tool moves away from the target trajectory in response to the reaction force from the object, and the pressure applied to the object by the tool increases in response to the distance from the target trajectory. First, the motion command unit 60 moves the robot arm 12 such that the grinding device 11a moves along the first target trajectory T1. At this time, the motion command unit 60 performs position control based on the grinding device 11a moving along the target trajectory, while simultaneously performing elastic control in parallel. Through elastic control, the grinding device 11a moves along the trajectory of the object W while moving away from the target trajectory to avoid excessive reaction force from the object W, and applies appropriate pressure to the object W. In addition to elastic control, the motion command unit 60 also performs inertial control and viscous control of the robot arm 12.

[0136] Figure 13 This is a schematic diagram of the trajectory of the grinding device 11a during the removal process. More specifically, as... Figure 13 As shown, the grinding device 11a moves along the first target trajectory T1 in the region where the processing part B is absent. When the grinding device 11a contacts the processing part B, due to the increased reaction force from the object W, it is affected by the viscosity coefficient cd and disengages from the first target trajectory T1 in the direction along the surface of the processing part B. However, due to the stiffness coefficient kd, the grinding device 11a exerts a greater pressing force on the processing part B the further away from the first target trajectory T1. That is, the further away from the first target trajectory T1 the processing part B is, the greater the depth of cut. On the other hand, in the region where the reaction force from the object W is smaller, the grinding device 11a passes near the first target trajectory T1. As a result, in the region where the processing part B exists, the grinding device 11a... Figure 13 As shown by the dashed line, the machining part B is ground with appropriate pressing pressure through the first actual trajectory t1 between the first target trajectory T1 and the surface of the machining part B.

[0137] During the movement of the grinding apparatus 11a along the first target trajectory T1 (including the case of disengagement from the first target trajectory T1), the action command unit 60 stores the contact force fs and the command position xds in the storage unit 32. When a grinding operation performed by the grinding apparatus 11a along the first target trajectory T1 is completed, the action command unit 60 reads the contact force fs and the command position xds during grinding from the storage unit 32, and calculates the standard deviation of the contact force fs and the standard deviation of the command position xds during grinding. In step S8, the action command unit 60 determines whether the completion conditions of the grinding process are met. For example, the completion condition is that the parameters associated with the removal process (i.e., grinding) are stable. Specifically, the parameters associated with the removal process are at least one of the following: the contact force fs during grinding, the command position xd during grinding, the command speed xd' during grinding, the acceleration xd" of the grinding device 11a during grinding, and the current supplied to the servo motor 15 during grinding. In this example, the completion condition is that the standard deviation of the contact force fs during grinding is below a specified first threshold α, and the standard deviation of the command position xds during grinding is below a specified second threshold β.

[0138] In other words, when the portion far from the first target trajectory T1 is included in the machining portion B, the standard deviation of the contact force fs during grinding increases because the contact force fs increases. Since the position of the grinding device 11a is also far from the first target trajectory T1 at this time, the standard deviation of the command position xds during grinding also increases. A contact force fs below the first threshold α and a standard deviation of the command position xds below the second threshold β mean that the machining portion B is ground into a shape approximately along the first target trajectory T1.

[0139] When the completion condition is not met, the machining part B is not ground into a shape corresponding to the first target trajectory T1. At that time, the motion command unit 60 returns to step S7 and again moves the robot arm 12 in such a way that the grinding device 11a moves along the first target trajectory. Through the first grinding process, the machining part B is ground into a shape that approximately follows the first actual trajectory t1. In the second grinding process, for example, the grinding device 11a, in the area where the machining part B exists, uses... Figure 13 The second actual trajectory t2, between the first target trajectory T1 and the first actual trajectory t1 represented by the two dashed lines, is ground with appropriate pressing pressure on part B.

[0140] Even in the second grinding process, if the completion condition in step S8 is not met, the motion command unit 60 returns to step S7 and again moves the robot arm 12 by moving the grinding device 11a along the first target trajectory. Through the second grinding process, the processed portion B is ground into a shape approximately along the second actual trajectory t2. In the third grinding process, for example, the grinding device 11a, in the area where the processed portion B exists, uses... Figure 13 The third actual trajectory t3, represented by the dotted line, is approximately consistent with the first target trajectory T1. Part B is ground using appropriate pressure. Furthermore, when the reaction force from the object W is small, the influence of elastic control diminishes, and position control becomes dominant. Therefore, the grinding device 11a passes through a trajectory close to the first target trajectory T1. In other words, it prevents the grinding device 11a from over-grinding the object W compared to the first target trajectory T1, and the object W is processed into the desired shape.

[0141] When the completion condition is met, the action command unit 60 determines in step S9 whether the grinding device 11a has reached the reference surface R. That is, the action command unit 60 determines whether the target trajectory when the condition of step S8 is met is the final target trajectory Tf.

[0142] When the grinding device 11a fails to reach the reference surface R, the action command unit 60 increases the cutting depth of the grinding device 11a in step S10. That is, the action command unit 60 switches the target trajectory to the next target trajectory (i.e., a target trajectory closer to the reference surface R).

[0143] The motion command unit 60 returns to step S7 and performs grinding with a new target trajectory. In the new target trajectory, the motion command unit 60 also repeats the movement of the grinding device 11a along the target trajectory until the completion condition is met.

[0144] In this way, after the motion command unit 60 moves the grinding device 11a along a target trajectory to perform a removal process, when the completion condition is met, it switches to the next target trajectory to perform the removal process. When the completion condition is not met, it moves the grinding device 11a again along a target trajectory (i.e., the same target trajectory) to perform the removal process. The motion command unit 60 repeats this process until the completion condition is met in the grinding process along the final target trajectory Tf.

[0145] When the conditions for completion of grinding along the final target trajectory Tf are met, the action command unit 60 proceeds to step S9 and ends the automatic control.

[0146] When there are multiple processing parts B in the object W, the process starting from step S1 can be repeated as many times as the number of processing parts B. Alternatively, multiple processing parts B can be specified in step S2, and the process starting from step S3 can be repeated as many times as the number of processing parts B.

[0147] In addition, if there are residues in the processing part B after automatic control, the processing part B can also be removed manually.

[0148] In this way, according to the automatic control of the robot system 100, the operator specifies the machining part B and the reference plane R in the image of the object W via the designated device 9, so that the robot 1 automatically removes the machining part B from the object W specified by the control device 3. The operator does not need to teach the robot 1 on-site. Therefore, the machining of the machining part B performed by the robot 1 can be easily achieved.

[0149] Furthermore, since the control device 3 not only processes the processing part B of the specific object W, but also the specific reference surface R, it can prevent the object W from being over-removed.

[0150] At this time, the control device 3 can remove the processed part B little by little by performing grinding operations on the processed part B, which is carried out by the grinding device 11a moving along the reference surface R, multiple times towards the reference surface R. By doing so, it is also possible to prevent the object W from being over-removed.

[0151] Furthermore, in the robot system 100, the image and three-dimensional information of the object W are acquired by the imaging device 81 and the 3D scanner 82 installed on the robot 1, respectively. Specifically, the imaging device 81 and the 3D scanner 82 are installed on the robot arm 12. The robot arm 12 is equipped with a grinding device 11a, which can move freely around the object W for grinding operations. In other words, since the robot arm 12 allows the imaging device 81 and the 3D scanner 82 to move freely around the object W, it is relatively easy to acquire the image and three-dimensional information of the object W by the imaging device 81 and the 3D scanner 82.

[0152] As described above, the robot system 100 includes a robot 1, a storage unit 32, a designation device 9, and a control device 3. The robot removes the processing portion B of the object W. The storage unit 32 stores an image of the object W and its three-dimensional information. The designation device 9 is used to designate the processing portion B from the image. The control device 3 controls the robot 1. The control device 3 derives the processing portion B from the three-dimensional information based on the portion designated by the designation device 9 in the image and the three-dimensional information. Based on the three-dimensional information of the processing portion B, the control device 3 causes the robot 1 to move, thereby removing the processing portion B.

[0153] In other words, the machining method of robot 1 includes the steps of specifying the machining portion B of object W in an image of object W, deriving the machining portion B in three-dimensional information based on the specified portion in the image and the three-dimensional information of object W, and causing robot 1 to move according to the three-dimensional information of machining portion B to remove machining portion B.

[0154] Furthermore, in order for the robot 1 to remove the processed part B of the object W, the processing program 32a executes the following steps: specifying the processed part B in the image of the object W; deriving the processed part B in the three-dimensional information based on the specified part in the image and the three-dimensional information of the object W; and causing the robot 1 to remove the processed part B by moving the robot 1 according to the three-dimensional information of the processed part B.

[0155] Based on these structures, if the operator specifies the processing part B in the image of the object W, there is no need to teach the robot 1 on-site. The control device 3 receives the specification of the processing part B in the image of the object W, the location of the processing part B on the specific object W, and the robot 1 automatically removes the processing part B. As a result, the removal of the processing part B performed by the robot 1 becomes simple.

[0156] Furthermore, based on the portion specified by the designated device 9 in the image and the three-dimensional information, the control device 3 derives the machining portion B and the reference plane R of the object W in which the machining portion B exists in the three-dimensional information, so that the robot 1 removes the machining portion B until it reaches the reference plane R.

[0157] According to this structure, not only is the machining portion B specified in the image of the object W, but the reference plane R is also specified. The control device 3, based on the specification in the image of the object W and the three-dimensional information of the object W, derives the machining portion B and the reference plane R from the three-dimensional information. By deriving the reference plane R, the control device 3 can perform the removal machining of the machining portion B up to the reference plane R. Therefore, excessive removal machining of the object W can be prevented.

[0158] Specifically, in addition to receiving the designation of the processing portion B in the image from the designating device 9, the control device 3 also receives the designation of the reference plane R, and derives the reference plane R in the three-dimensional information based on the designation of the reference plane R in the image by the designating device 9.

[0159] According to this structure, the control device 3 receives the reference plane R from the image provided by the operator from the designating device 9. Since the control device 3 receives the direct designation from the operator, it derives the reference plane R from the three-dimensional information, thus enabling it to determine the reference plane R with high accuracy.

[0160] Furthermore, the control device 3 determines the starting position S of the removal process based on the three-dimensional information of the processing part B.

[0161] According to this structure, the control device 3 determines the starting position S of the removal process based on the three-dimensional information of the processing part B. In other words, the operator only needs to specify the processing part B in the image of the object W, without needing to specify the starting position S. Similarly, the operator does not need to be on-site to teach the robot 1 the starting position S of the removal process.

[0162] Furthermore, the control device 3 causes the robot 1 to move in a manner that removes the processed part B in multiple steps toward the reference plane R.

[0163] This structure reduces the reaction force exerted on robot 1 by the object W. Furthermore, by removing the processed portion B piece by piece, it prevents even parts that shouldn't be removed from being removed.

[0164] Furthermore, the robot system 100 also includes an imaging device 81 and a 3D scanner 82 (3D information acquisition device). The imaging device 81 acquires an image of the object W, and the 3D scanner 82 acquires the 3D information of the object W. The storage unit 32 stores the image of the object W acquired by the imaging device 81 and the 3D information of the object W acquired by the 3D scanner 82.

[0165] According to this structure, even without prior preparation of images and three-dimensional information of the object W, the imaging device 81 and the three-dimensional scanner 82 can acquire images and three-dimensional information of the object W.

[0166] Furthermore, the removal process performed by robot 1 is grinding, cutting, or lapping.

[0167] According to this structure, the machined portion B of the object W is removed by grinding, cutting or lapping.

[0168] Other implementation methods

[0169] As described above, the embodiments described herein have been presented as examples of the technology disclosed in this application. However, the technology disclosed herein is not limited to this and can be applied to embodiments with appropriate modifications, substitutions, additions, omissions, etc. Furthermore, the various constituent elements described in the embodiments can be combined to form new embodiments. Moreover, the constituent elements described in the drawings and detailed descriptions include not only those necessary to solve the problem, but also, for the purpose of illustrating the technology, constituent elements that are not necessary to solve the problem. Therefore, one should not immediately assume that those non-essential constituent elements are essential simply because they are described in the drawings and detailed descriptions.

[0170] For example, robot 1 is not limited to a robot capable of bilateral control. For example, the operating device 2 may also be omitted.

[0171] The processing performed by robot 1 is not limited to removal processing. For example, the processing performed by robot 1 can also be processing such as sealing.

[0172] The object is not limited to castings. Any workpiece containing machined parts can become an object. The machined parts are not limited to burrs. Any part that should be machined can become an object.

[0173] The imaging device 81 may also be omitted from the robot arm 12. For example, the imaging device 81 may be fixed away from the robot 1. For example, the imaging device 81 may also be detached from the robot 1 and positioned above the object W.

[0174] The 3D scanner 82 may also be omitted from the robot arm 12. For example, the 3D scanner 82 may be fixed away from the robot 1. For example, the 3D scanner 82 may also be detached from the robot 1 and positioned above the object W.

[0175] The 3D information of an object is not limited to point cloud data. 3D information simply refers to information that represents the 3D shape of the object. For example, 3D information can also be a depth image.

[0176] The image and 3D information of the object W are not limited to the information acquired by the imaging device 81 and the 3D scanner 82 installed on the robot 1. The image and 3D information of the object W can also be acquired in advance and stored in the storage unit 32.

[0177] The method for specifying the machining portion B and the reference plane R in the image of the object W is not limited to the method described above. The machining portion B in the image may also be specified by point P instead of by the box F. The control device 3 may also determine the portion in the three-dimensional information corresponding to point P in the image, and export the portion containing that portion that protrudes compared to its surroundings as the machining portion B. Furthermore, the portion surrounding the machining portion B may also be exported as the reference plane R.

[0178] Furthermore, the control device 3 can also receive only the designation of the machining portion B in the image via the designating device 9, without receiving a direct designation of the reference plane R. In other words, the control device 3 can also derive the machining portion B in the three-dimensional information based on the portion designated by the designating device 9 in the image of the object W and the three-dimensional information of the object W, and derive the surface surrounding the machining portion B as the reference plane R. In this way, even without receiving a direct designation of the reference plane R, the control device 3, by receiving the designation of the machining portion B, derives the reference plane R in addition to the machining portion B.

[0179] The method of removal is not limited to the above description. Although the control device 3 performs position control and elastic control of the grinding device 11a, it may also omit elastic control. Furthermore, the control device 3 removes the machining portion B in multiple steps toward the reference surface R, and is not limited to this. The control device 3 may also generate only the final target trajectory Tf and perform grinding along the final target trajectory Tf from the beginning.

[0180] Furthermore, when the motion command unit 60 moves from one target trajectory to the next target trajectory, it determines whether the completion conditions for grinding have been met, but it is not limited to this. That is to say, the motion command unit 60 may also move to grinding along the next target trajectory without confirming whether the completion conditions have been met when the grinding along one target trajectory is completed.

[0181] The completion condition is not limited to the above. For example, the completion condition could also be that the standard deviation of the contact force fs during grinding is below a specified first threshold α. The completion condition could also be that the standard deviation of the command position xds during grinding is below a specified second threshold β. The completion condition could also be that at least one of the following conditions is met: the standard deviation of the contact force fs during grinding is below the specified first threshold α, and the standard deviation of the command position xds during grinding is below the specified second threshold β.

[0182] The control device 3 uses the motion model represented by equation (1) for position control and elastic control, but the position control and elastic control are not limited to this. As long as the position of the tool is controlled so that the tool moves along the target trajectory, and control is performed so that when the reaction force from the object to the tool is large, the tool disengages from the target trajectory and the pressing force on the object is applied to the tool according to the distance from the target trajectory, position control and elastic control using any model can be adopted.

[0183] A flowchart is just one example. Steps in a flowchart can be modified, replaced, added, or omitted as appropriate. Furthermore, the order of steps in the flowchart can be changed, or sequential processes can be handled in parallel.

[0184] The functions implemented by the constituent elements described in this specification can also be implemented in a circuit or processing circuitry that includes a general-purpose processor, a special-purpose processor, an integrated circuit, an ASIC (Application Specific Integrated Circuits), a CPU (a Central Processing Unit), conventional circuitry, and / or combinations thereof, all programmed to perform the described functions. A processor contains transistors and other circuitry and is considered as a circuit or processing circuitry. A processor can also be a programmable processor that executes a program stored in memory.

[0185] In this specification, a circuit, unit, or device is hardware programmed or executing to perform the described function. This hardware may also be any hardware disclosed in this specification or any hardware known to be programmed or executing to perform the described function.

[0186] When the hardware is a processor of the type of circuit, the circuit, device, or unit is a combination of hardware and software used to constitute the hardware and / or the processor.

Claims

1. A robot system, characterized in that: The robot system includes a robot, a storage unit, a designating device, and a control device. The robot processes the processing portion of an object. The storage unit stores an image and three-dimensional information of the object. The designating device is used to designate the processing portion and a reference plane of the object on which the processing portion exists from the image. The control device controls the robot. The control device derives the machining portion and the reference surface from the three-dimensional information based on the portion in the image specified by the designated device and the three-dimensional information. The robot is moved according to the three-dimensional information of the processing part, so that the robot removes the processing part until it reaches the reference surface.

2. The robot system according to claim 1, characterized in that: The control device determines the starting position of the removal process based on the three-dimensional information of the processing section.

3. The robot system according to claim 2, characterized in that: The control device causes the robot to move in a manner that removes the processed portion multiple times toward the reference plane.

4. The robot system according to claim 1 or 2, characterized in that: The robot system also includes an imaging device and a 3D information acquisition device. The imaging device acquires an image of the object, and the 3D information acquisition device acquires 3D information of the object. The storage unit stores the image of the object acquired by the imaging device and the three-dimensional information of the object acquired by the three-dimensional information acquisition device.

5. The robot system according to claim 1 or 2, characterized in that: The removal process performed by the robot is grinding, cutting, or lapping.

6. A machining method for a robot, characterized in that: The robot's processing method includes the steps of specifying the processing portion of the object in an image of the object and the reference plane of the object where the processing portion exists, the steps of deriving the processing portion and the reference plane in the three-dimensional information based on the specified portion in the image and the three-dimensional information of the object, and the steps of causing the robot to move according to the three-dimensional information of the processing portion so that the robot removes the processing portion until it reaches the reference plane.

7. A processing procedure, characterized in that: The machining process involves the computer performing the following steps to enable the robot to process a portion of an object: specifying the portion to be processed in an image of the object and specifying a reference plane of the object in which the portion to be processed exists; deriving the portion to be processed and the reference plane in the three-dimensional information based on the specified portion in the image and the three-dimensional information of the object; and causing the robot to move according to the three-dimensional information of the portion to remove the portion to the reference plane.

Images