Robot control device, robot control system, and robot control method

Abstract

To provide a robot control device, a robot control system, and a robot control method that can improve the calibration accuracy of a robot.SOLUTION: This robot control device comprises a control unit that controls a robot having a sensor and an end effector. The control unit: acquires information of the position of the end effector with respect to a measurement object disposed within an operational space of the robot; and corrects a coordinate system related to robot operations, on the basis of the positional information.SELECTED DRAWING: Figure 1

Description

Cross-reference to related applications

[0001] This application claims priority to Japanese Patent Application No. 2021-127725 (filed August 3, 2021), the entire disclosure of which is incorporated herein by reference. [Technical Field]

[0002] The present disclosure relates to a robot control device, a robot control system, and a robot control method. [Background technology]

[0003] BACKGROUND ART Conventionally, there is known a robot control device that creates teaching data based on image recognition processing of alignment marks on an object to be worked on (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 5-301183 Summary of the Invention

[0005] A robot control device according to an embodiment of the present disclosure includes a control unit that controls a robot having a sensor and an end effector, the control unit acquiring position information of the end effector relative to a measurement target disposed within a motion space of the robot, and correcting a coordinate system relating to the motion of the robot based on the position information.

[0006] A robot control system according to an embodiment of the present disclosure includes the robot control device and the robot.

[0007] A robot control method according to an embodiment of the present disclosure controls a robot having a sensor and an end effector, the method including: acquiring position information of the end effector relative to a measurement target disposed in a motion space of the robot; and correcting a coordinate system relating to motion of the robot based on the position information. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram illustrating an example of the configuration of a robot control system according to an embodiment. [Figure 2] FIG. 1 is a schematic diagram illustrating an example of the configuration of a robot control system according to an embodiment. [Figure 3] FIG. 2 is a schematic diagram illustrating a configuration example of an end effector. [Figure 4] 1 is a flowchart illustrating an example of a procedure for a robot control method according to an embodiment. [Figure 5A] 10A and 10B are schematic diagrams showing an example of an operation in which the outer side of the first finger comes into contact with the object to be measured when the first finger and the second finger are spread apart. [Figure 5B] 10A and 10B are schematic diagrams showing an example of an operation in which the side surface of the first finger comes into contact with the object to be measured when the first finger and the second finger are in an open position. [Figure 5C] 10A and 10B are schematic diagrams showing an example of an operation in which the tip of the first finger comes into contact with the object to be measured when the first finger and the second finger are spread apart. [Figure 6A] 10A and 10B are schematic diagrams showing an example of an operation in which the outer side of the first finger comes into contact with the object to be measured when the first finger and the second finger are closed together. [Figure 6B] 10A and 10B are schematic diagrams showing an example of an operation in which the side of the first finger or the second finger comes into contact with the object to be measured when the first finger and the second finger are closed together. [Figure 6C] 10A and 10B are schematic diagrams showing an example of an operation in which the tip of the first finger or the second finger comes into contact with the object to be measured when the first finger and the second finger are closed together. [Figure 7A] 10A and 10B are schematic diagrams showing an example of an operation in which a first finger or a second finger moves in the X_RB axis direction along the top surface of the object to be measured while in contact with the top surface. [Figure 7B]10A and 10B are schematic diagrams showing an example of an operation in which a first finger or a second finger moves in the Y_RB axis direction along the top surface of the object to be measured while in contact with the top surface. [Figure 7C] 10A and 10B are schematic diagrams showing an example of an operation in which a first finger or a second finger simultaneously moves in the X_RB axis direction and the Y_RB axis direction along the top surface of the object to be measured while in contact with the top surface. [Figure 8] 10A and 10B are schematic diagrams showing an example of an operation in which a first finger and a second finger pinch a measurement object and rotate around the measurement object. [Figure 9A] 1 is a schematic diagram showing an example of the configuration of a measurement object having a mark; [Figure 9B] 1A and 1B are schematic diagrams showing an example of the configuration of a measurement object having a mark and a contact area. [Figure 9C] FIG. 10 is a schematic diagram showing an example of the configuration of a measurement object that does not have a mark. [Figure 9D] FIG. 2 is a schematic diagram showing an example of the configuration of a cylindrical measurement object. [Figure 10] FIG. 10 is a schematic diagram showing an example in which measurement targets are placed at diagonal positions on the top surface of a workbench. DETAILED DESCRIPTION OF THE INVENTION

[0009] It is necessary to perform robot calibration with higher accuracy than calibration by image recognition. According to a robot control device, a robot control system, and a robot control method according to an embodiment of the present disclosure, the accuracy of robot calibration can be improved.

[0010] (Overview of Robot Control System 1) As illustrated in FIGS. 1 and 2 , a robot control system 1 according to an embodiment includes a robot 40, a robot control device 10, and a spatial information acquisition unit 20. The robot 40 operates in a predetermined operating space. The spatial information acquisition unit 20 captures an image of the operating space in which the robot 40 operates and generates depth information of the operating space. As will be described later, the spatial information acquisition unit 20 calculates the distance to a measurement point located on the surface of an object 50 existing in the operating space. The distance from the spatial information acquisition unit 20 to the measurement point is also referred to as depth. The depth information is information relating to the depth measured for each measurement point. In other words, the depth information is information relating to the distance to a measurement point located on the surface of an object 50 existing in the operating space. The depth information may be expressed as a depth map that associates a direction as viewed from the spatial information acquisition unit 20 with a depth in that direction. The spatial information acquisition unit 20 generates the depth information of the operating space based on an (X_CA, Y_CA, Z_CA) coordinate system. The spatial information acquisition unit 20 may generate an image of the operating space. An image captured in the motion space is also called a motion space image.

[0011] As will be described later, the robot control device 10 may perform calibration based on an image of the operating space captured by the spatial information acquisition unit 20. The robot control device 10 can correct a coordinate system related to the robot's operation by detecting position information of the robot 40 relative to a measurement object 52 (see FIG. 3, etc.) placed in the operating space. The coordinate system related to the robot's operation may be, for example, a coordinate system of the operating space, a coordinate system of the robot 40, or a coordinate system of the spatial information acquisition unit 20. The robot control device 10 may detect position information of the robot 40 relative to the measurement object 52 by detecting various physical quantities such as load or pressure. For example, when the robot control device 10 detects load or pressure, the position of the robot 40 relative to the measurement object 52 is 0 (zero) or a certain distance. The robot control device 10 may also detect position information of the robot 40 relative to the measurement object 52 by measuring the distance between the robot 40 and the measurement object 52. The robot 40 has a sensor 48 capable of detecting position information relative to the measurement object 52.

[0012] The robot control device 10 operates the robot 40 based on the depth information generated by the spatial information acquisition unit 20. The robot control device 10 controls and operates the robot 40 based on the (X_RB, Y_RB, Z_RB) coordinate system. The robot 40 may be installed on, for example, a workbench 70. The robot control device 10 may cause the robot 40 to perform a task using an object 50 located within the operating space of the robot 40 as a work target. Alternatively, the robot control device 10 may recognize the object 50 located within the operating space of the robot 40 as an obstacle. The robot control device 10 may perform calibration before causing the robot 40 to perform a task. Before causing the robot 40 to perform a task, the robot control device 10 may correct the coordinates of the measurement target 52 or the coordinate system by bringing the robot 40 into contact with the measurement target 52 to detect a load acting on the robot 40.

[0013] The (X_RB, Y_RB, Z_RB) coordinate system is also referred to as the coordinate system of the robot 40. The (X_CA, Y_CA, Z_CA) coordinate system is also referred to as the coordinate system of the spatial information acquisition unit 20. The coordinate system of the robot 40 may be set to be the same as the coordinate system of the spatial information acquisition unit 20, or may be set to be a different coordinate system. When the coordinate system of the robot 40 is set to be a different coordinate system from the coordinate system of the spatial information acquisition unit 20, for example, the robot control device 10 converts the depth information generated in the coordinate system of the spatial information acquisition unit 20 into the coordinate system of the robot 40 and uses it. Note that the coordinate system of the spatial information acquisition unit 20 may be calibrated in advance with the coordinate system of the operating space. Specifically, the coordinate system of the spatial information acquisition unit 20 is expressed as a coordinate system (X_CA, Y_CA, Z_CA) calculated by the camera, for example, but the coordinate system (X_CA, Y_CA, Z_CA) calculated by the camera may be calibrated to match the coordinate system (X, Y, Z) indicating the operating space. Note that the coordinate system of the spatial information acquisition unit 20 does not need to be calibrated in advance.

[0014] The number of robots 40 and robot control devices 10 is not limited to one as illustrated, but may be two or more. The number of spatial information acquisition units 20 may be one for one operating space as illustrated, or may be two or more. Each component will be specifically described below.

[0015] <Robot control device 10> The robot control device 10 includes a control unit 11, a storage unit 12, and an interface 13. The interface 13 is also referred to as I / F13.

[0016] The control unit 11 may be configured to include at least one processor to realize various functions of the robot control device 10. The processor may execute programs that realize various functions of the robot control device 10. The processor may be realized as a single integrated circuit. An integrated circuit is also called an IC (Integrated Circuit). The processor may be realized as multiple integrated circuits and discrete circuits that are connected to each other in a communicative manner. The processor may be configured to include a CPU (Central Processing Unit). The processor may be configured to include a DSP (Digital Signal Processor) or a GPU (Graphics Processing Unit). The processor may be realized based on various other known technologies.

[0017] The storage unit 12 may be configured to include an electromagnetic storage medium such as a magnetic disk, or may be configured to include a memory such as a semiconductor memory or a magnetic memory. The storage unit 12 may be configured as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). The storage unit 12 stores various information and programs executed by the control unit 11. The storage unit 12 may function as a work memory for the control unit 11. The control unit 11 may be configured to include at least a part of the storage unit 12.

[0018] The I / F 13 acquires images of the operating space of the robot 40 captured by the spatial information acquisition unit 20 and outputs the images to the control unit 11. The I / F 13 acquires information about the robot 40 and outputs the images to the control unit 11. The information about the robot 40 includes information from sensors 48, which will be described later. The I / F 13 acquires information for controlling the robot 40 from the control unit 11 and outputs the information to the robot 40. The I / F 13 may be configured to include a communication device configured to be able to communicate with the spatial information acquisition unit 20 and the robot 40 via wired or wireless communication. The communication device may be configured to be able to communicate using communication methods based on various communication standards. The communication device may be configured using known communication technology. Detailed description of the hardware of the communication device will be omitted. The functions of the communication device may be realized by a single communication interface, or may be realized by separate communication interfaces for each connection destination. The control unit 11 may be configured to be able to communicate with the spatial information acquisition unit 20 and the robot 40. The control unit 11 may be configured to include a communication device.

[0019] <Robot 40> As illustrated in FIG. 1, the robot 40 includes a sensor 48. As illustrated in FIG. 2, the robot 40 also includes an arm 42, an end effector 44 attached to the arm 42, and a mark 46 attached to the end effector 44. The robot 40 further includes an interface 49, although this is not essential. The interface 49 is also referred to as an I / F 49. The mark 46 may be attached to the arm 42 instead of the end effector 44.

[0020] The arm 42 may be configured as, for example, a six-axis or seven-axis vertical articulated robot. The arm 42 may be configured as a three-axis or four-axis horizontal articulated robot or a SCARA robot. The arm 42 may be configured as a two-axis or three-axis Cartesian robot. The arm 42 may be configured as a parallel link robot or the like. The number of axes configuring the arm 42 is not limited to those exemplified.

[0021] The end effector 44 may include, for example, a gripping hand configured to grip a workpiece. The gripping hand may have multiple fingers. The gripping hand may have two or more fingers. The fingers of the gripping hand may have one or more joints. The end effector 44 may include a suction hand configured to pick up a workpiece by suction. The end effector 44 may include a scooping hand configured to scoop up a workpiece. The end effector 44 may include a tool such as a drill and be configured to perform various processes, such as drilling a hole in a workpiece. The end effector 44 is not limited to these examples and may be configured to perform various other operations. Sensor information may be controlled by the end effector 44 or the robot 40. For example, as shown in FIG. 1, the end effector 44 may be equipped with a control unit 440. Alternatively, the robot 40 may be equipped with a control unit 410. The control unit 440 of the end effector 44 or the control unit 410 of the robot 40 may take in the sensor information. The control unit 440 of the end effector 44 or the control unit 410 of the robot 40 may output status information (e.g., the amount of misalignment or the amount of contact) that can be inferred from the sensor information to the robot control device 10. The sensor information may be handled on the robot control device 10 side, or may be handled by the control unit 410 of the robot 40, or may be handled by the control unit 440 of the end effector 44.

[0022] As shown in FIG. 3, the end effector 44 of the robot 40 according to this embodiment is configured as a gripping hand and includes a first finger 441, a second finger 442, and a drive unit 443. In the end effector 44 configured as a gripping hand or a suction hand, the number of fingers or suction nozzles included in the end effector 44 is not limited to two, and may be one, three, or more. In FIG. 3, the object to be worked on by the end effector 44 is represented as a measurement object 52. The end effector 44 is configured to contact the measurement object 52 with a holding unit such as at least one finger or suction nozzle. The end effector 44 may also be configured to hold the measurement object 52 with three or more fingers or holding units such as suction nozzles.

[0023] The robot 40 can control the position of the end effector 44 by operating the arm 42. The end effector 44 may have an axis that serves as a reference for the direction in which it acts on a workpiece. If the end effector 44 has an axis, the robot 40 can control the direction of the axis of the end effector 44 by operating the arm 42. The robot 40 controls the start and end of the operation of the end effector 44 acting on the workpiece. The robot 40 can move or process the workpiece by controlling the operation of the end effector 44 while controlling the position of the end effector 44 or the direction of the axis of the end effector 44.

[0024] The sensor 48 detects the state of each component of the robot 40. The sensor 48 may detect the actual position or posture of each component of the robot 40, or information related to the speed or acceleration of each component of the robot 40. The sensor 48 may detect the load acting on each component of the robot 40. The sensor 48 may detect the current flowing through a motor that drives each component of the robot 40 or the torque of the motor. The sensor 48 can detect information obtained as a result of the actual movement of the robot 40. The sensor 48 may detect the distance between the robot 40 and another object. The robot control device 10 can grasp the result of the actual movement of the robot 40 by obtaining the detection result of the sensor 48. In other words, the robot control device 10 can obtain the state of the robot 40 based on the detection result of the sensor 48.

[0025] The sensor 48 includes a force sensor 444 that detects the direction or magnitude of a load acting on the end effector 44 when the end effector 44 comes into contact with a work object. The force sensor 444 is mounted on the end effector 44. The force sensor 444 may be configured to include, for example, a strain gauge, but is not limited to this.

[0026] The sensor 48 further includes, but is not limited to, a tactile sensor 445. The tactile sensor 445 is mounted on the first finger 441 or the second finger 442. The tactile sensor 445 may include, for example, a pressure sensor, but is not limited to this.

[0027] The sensor 48 may further include a distance sensor. The distance sensor may be provided, for example, at the tip of the end effector 44. Specifically, the distance sensor may be mounted on the first finger 441 or the second finger 442.

[0028] The robot control device 10 recognizes the position of the mark 46 or the position of the end effector 44 on which the mark 46 is placed based on an image of the mark 46 captured by the spatial information acquisition unit 20. The robot control device 10 also recognizes the state of the robot 40 based on an image of the mark 46 captured by the spatial information acquisition unit 20. The robot control device 10 can calibrate the robot 40 by comparing the state of the robot 40 acquired based on the detection results of the sensor 48 with the state of the robot 40 acquired based on the image of the mark 46.

[0029] <Spatial information acquisition unit 20> The spatial information acquisition unit 20 acquires spatial information related to the operation space of the robot 40. The spatial information acquisition unit 20 may capture an image of the operation space and acquire an image of the operation space as spatial information. As illustrated in FIG. 2, the spatial information acquisition unit 20 may capture an image of a work target, such as an object 50 or a measurement target 52, present in the operation space. The spatial information acquisition unit 20 may be configured as a camera. A 3D stereo camera captures an image of the object 50 present in the operation space, calculates the distance to a measurement point located on the surface of the object 50 present in the operation space as the depth, and generates depth information. The spatial information acquisition unit 20 may be configured as a 3D stereo camera. The spatial information acquisition unit 20 may be configured as a LiDAR (light detection and ranging). The LiDAR measures the distance to a measurement point located on the surface of the object 50 present in the operation space and generates depth information. In other words, the spatial information acquisition unit 20 may acquire depth information of the operation space as spatial information. The spatial information acquisition unit 20 is not limited to these, and may be configured as various devices. The spatial information acquisition unit 20 may acquire various types of spatial information, including but not limited to images or depth information of the operation space. The spatial information acquisition unit 20 may include an imaging element. The spatial information acquisition unit 20 may further include an optical system. The spatial information acquisition unit 20 may output an image of the operation space captured to the robot control device 10. The spatial information acquisition unit 20 may generate depth information in the operation space of the robot 40 and output it to the robot control device 10. The spatial information acquisition unit 20 may generate point cloud information in the operation space of the robot 40 and output it to the robot control device 10. That is, the spatial information may be output in the form of point cloud data. In other words, the point cloud information may have spatial information. The point cloud information is information about a set of measurement points located on the surface of an object 50 existing in the operation space, and includes coordinate information or color information of each measurement point. The point cloud information can also be said to be data representing the object 50 in the measurement space with a plurality of points. By having the spatial information in the form of point cloud data, the data density can be made smaller than that of spatial information based on the initial data acquired by the spatial information acquisition unit 20.

[0030] The spatial information acquisition unit 20 has a field of view (FOV). The FOV corresponds to the imaging range of the spatial information acquisition unit 20. The spatial information acquisition unit 20 can capture the range included in the FOV. The actual field of view size of the spatial information acquisition unit 20 is determined based on the FOV of the spatial information acquisition unit 20 and depth information. The robot control device 10 can acquire the position and orientation of the mark 46 of the robot 40 based on the actual field of view size of the spatial information acquisition unit 20 and spatial information including images captured by the spatial information acquisition unit 20 so as to capture the mark 46 of the robot 40. Specifically, the robot control device 10 can calculate the position and orientation of the mark 46 based on the spatial information including the images by analyzing the spatial information including the images capturing the mark 46 using a predetermined algorithm. The predetermined algorithm may include, for example, a mathematical formula or a table, or may include a program specifying a calculation process. The predetermined algorithm may include parameters for correcting calculation results based on spatial information including images.

[0031] (Operation example of the robot control device 10) The robot control device 10 operates the robot 40 to act on a work target such as an object 50 or a measurement target 52 present in the operating space, or to operate the robot 40 to avoid the object 50 or the measurement target 52. The robot control device 10 operates the robot 40 to act on a work target such as the object 50 or the measurement target 52, or to avoid the object 50 or the measurement target 52, based on a photographed image of the object 50 or the measurement target 52 taken by the spatial information acquisition unit 20.

[0032] <Calibration> The control unit 11 of the robot control device 10 acquires the state of the robot 40 based on the position and orientation of the mark 46 captured in the image captured by the spatial information acquisition unit 20, and can acquire the positional relationship between the robot 40 and the object 50 or the measurement target 52. Meanwhile, the control unit 11 acquires the state of the robot 40 based on the sensor 48 of the robot 40. The state based on the sensor 48 of the robot 40 represents the position and orientation of the robot 40 with higher accuracy than the state based on the image captured by the spatial information acquisition unit 20. Therefore, the control unit 11 can control the robot 40 with high accuracy in the operating space by matching the state of the robot 40 based on the image captured by the spatial information acquisition unit 20 with the state of the robot 40 based on the sensor 48 of the robot 40. The operation of matching the state of the robot 40 based on spatial information including the image captured by the spatial information acquisition unit 20 with the state of the robot 40 based on the sensor 48 of the robot 40 is also referred to as first calibration. The control unit 11 performs the first calibration at least once. Specifically, the control unit 11 performs the first calibration so as to match the depth information generated in the (X_CA, Y_CA, Z_CA) coordinate system by the spatial information acquisition unit 20 with the (X_RB, Y_RB, Z_RB) coordinate system of the robot 40. In other words, by performing the first calibration, the control unit 11 becomes able to convert the coordinate system of the spatial information acquisition unit 20 into the coordinate system of the robot 40. The control unit 11 may estimate the relative positional relationship between the coordinate system of the spatial information acquisition unit 20 and the coordinate system of the robot 40, and align the coordinate system of the spatial information acquisition unit 20 with the coordinate system of the robot 40 based on the estimated relative positional relationship.

[0033] The control unit 11 may perform the first calibration within a range that is at least a part of the FOV of the spatial information acquisition unit 20. In this embodiment, the control unit 11 performs the first calibration within a calibration range 60 shown in FIG. 2. The calibration range 60 is shown as an area surrounded by a two-dot chain line in FIG. 2. The calibration range 60 corresponds to the range within which the first calibration of the robot 40 is performed. The calibration range 60 may include the working area of ​​the robot 40. The calibration range 60 may be a range in which the working area of ​​the robot 40 and the FOV overlap.

[0034] The control unit 11 also sets a point for performing the first calibration by moving the mark 46 of the robot 40 within the calibration range 60. The point for performing the first calibration is also referred to as a calibration position. The control unit 11 moves the mark 46 of the robot 40 to the calibration position and causes the spatial information acquisition unit 20 to capture an image of the mark 46. The control unit 11 calculates the position and orientation of the mark 46 based on spatial information including an image, etc., capturing the mark 46. The control unit 11 corrects the position and orientation of the mark 46 based on the spatial information including an image, etc., so that the position and orientation of the mark 46 calculated based on the spatial information including an image, etc., coincide with the position and orientation of the mark 46 determined based on the detection result of the sensor 48 of the robot 40. The correction of the position and orientation of the mark 46 based on the spatial information including an image, etc. corresponds to the first calibration. The position and orientation of the mark 46 are also referred to as the mark position and orientation. The first calibration corresponds to the correction of the mark position and orientation. The calibration position corresponds to the position at which the mark position and orientation is corrected.

[0035] Specifically, the control unit 11 may perform the first calibration as described below. The control unit 11 generates control information for the robot 40 for moving the mark 46 of the robot 40 to a calibration position. The control unit 11 operates the robot 40 based on the control information to move the mark 46 of the robot 40 to the calibration position. The control unit 11 acquires spatial information including an image of the mark 46 from the spatial information acquisition unit 20. The control unit 11 calculates the position and orientation of the mark 46 based on the spatial information including the image. The position and orientation of the mark 46 calculated based on the spatial information including the image is also referred to as the mark position and orientation based on the spatial information including the image. The control unit 11 calculates the position and orientation of the mark 46 determined based on the detection result of the sensor 48 of the robot 40. The position and orientation of the mark 46 calculated based on the detection result of the sensor 48 is also referred to as the mark position and orientation based on the sensor 48. The control unit 11 compares the mark position and orientation based on the spatial information including the image with the mark position and orientation based on the sensor 48. The control unit 11 corrects the mark position and orientation based on the spatial information including images, etc. so that the mark position and orientation based on the spatial information including images, etc. coincides with the mark position and orientation based on the sensor 48. The control unit 11 may correct an algorithm that calculates the mark position and orientation based on the spatial information including images, etc. The control unit 11 may correct parameters included in the algorithm, or may correct a mathematical formula, table, or program. When multiple calibration positions are set, the control unit 11 moves the robot 40 to each calibration position, acquires spatial information including images, etc. of the mark 46 at each calibration position, and corrects the mark position and orientation based on the spatial information including images, etc.

[0036] <<Calibration items>> Before performing the first calibration, the control unit 11 sets a calibration range 60 in advance. Furthermore, the control unit 11 sets a calibration position included in the calibration range 60. The control unit 11 sets the calibration position within the calibration range 60.

[0037] The control unit 11 generates control information for the robot 40 to move the robot 40 to the calibration position. The control unit 11 generates, as calibration items, information that specifies the mark position and orientation when the robot 40 is moved to the calibration position and the recognition result of the mark 46 of the robot 40. The calibration items are, for example, coordinate information. Specifically, the calibration items are, for example, coordinate information that indicates the mark position and orientation based on the detection result of the sensor 48 of the robot 40 when the robot 40 is moved to the calibration position, or coordinate information that indicates the mark position and orientation based on the recognition result of the mark 46 recognized by the spatial information acquisition unit 20.

[0038] The control unit 11 may generate the calibration items as described below.

[0039] The control unit 11 acquires, for example, information about the field of view size or FOV of the spatial information acquisition unit 20 from the spatial information acquisition unit 20. The control unit 11 sets the calibration range 60 based on the field of view size or FOV of the spatial information acquisition unit 20 and the working area of ​​the robot 40. The control unit 11 may set the calibration range 60 based on the position of the object 50, the measurement target 52, etc. in the operating space of the robot 40. The control unit 11 may set the calibration range 60 based on depth information or point cloud information of the object 50, the measurement target 52, etc. detected by the spatial information acquisition unit 20. In FIG. 2 , the shape of the calibration range 60 is set to a quadrangular pyramid truncated shape. The shape of the calibration range 60 is not limited to this and may be set to various other shapes.

[0040] The control unit 11 matches the mark position and orientation based on the sensor 48 of the robot 40 with the mark position and orientation based on the image of the spatial information acquisition unit 20. Specifically, the control unit 11 moves the robot 40 to a first position. The control unit 11 generates control information for operating the robot 40 so that the mark 46 of the robot 40 is at a predetermined position and orientation, and controls the robot 40 based on the control information to move the robot 40 to the first position. The first position may be a predetermined position included in the FOV of the spatial information acquisition unit 20. The first position may be, for example, the center position of the FOV of the spatial information acquisition unit 20. The control unit 11 acquires an image of the mark 46 when the robot 40 moves to the first position, and calculates the position and orientation of the mark 46 as the mark position and orientation based on the image. The control unit 11 also calculates the mark position and orientation based on the sensor 48. Based on a comparison between the mark position and orientation based on the image and the mark position and orientation based on the sensor 48, the control unit 11 corrects the control information for the robot 40 so that the position of the robot 40 in the image becomes the first position based on the detection result of the sensor 48. The control unit 11 moves the robot 40 based on the corrected control information, thereby updating the state of the robot 40 so that the position of the robot 40 in the coordinate system of the robot 40 matches the position of the robot 40 in the coordinate system of the spatial information acquisition unit 20. In other words, it can be said that the control unit 11 updates the state of the robot 40 so that the position of the robot 40 becomes the first position in the image.

[0041] The control unit 11 may generate a position within the calibration range 60 that is a candidate for the calibration position, different from the first position. The position that is a candidate for the calibration position is also referred to as the second position. The second position is included in the calibration range 60. The control unit 11 estimates the state of the robot 40 when the robot 40 moves to the second position by simulating the operation of the robot 40. In other words, the control unit 11 calculates the state of the robot 40 when it is assumed that the robot 40 moves to the second position. As a result, the control unit 11 can determine whether the robot 40 can move to the second position.

[0042] The control unit 11 registers the second position as a calibration position if the state of the robot 40 when it is assumed to move to the second position is a state in which it is not in contact with the object 50, the measurement target 52, etc., is within the joint range of motion, and is not a singular point. When registering the second position as a calibration position, the control unit 11 generates, as multiple calibration items, information that identifies the mark position and orientation based on the detection result of the sensor 48 of the robot 40 when the robot 40 is moved to the second position and the tip position and orientation based on the recognition result of the mark 46 of the robot 40. If the control unit 11 does not register the second position as a calibration position, it may generate a new, different second position and determine whether the new second position can be registered as a calibration position. The control unit 11 may determine that the state of the robot 40 is not in a joint-restricted state if the numerical values ​​representing the angles of the joints of the robot 40 are within the range of motion. The control unit 11 may determine that the state of the robot 40 is in a joint-restricted state if the numerical values ​​representing the angles of the joints of the robot 40 are outside the range of motion.

[0043] A singularity corresponds to a posture at which the robot 40 becomes structurally uncontrollable. If a singularity is included in the trajectory for moving the robot 40, the robot 40 will move (runaway) at high speed near the singularity and stop at the singularity. There are three types of singularities of the robot 40: (1) to (3) below. (1) A point outside the working area when controlling the robot 40 to the vicinity of the outer limit of the working area. (The working area is an area corresponding to the operating space of the robot 40.) (2) Points when controlling the robot 40 directly above and below the robot base even within the working area. (3) A point where the angle of the joint immediately before the joint at the tip of the arm 42 of the robot 40 is zero or 180 degrees (wrist alignment singular point).

[0044] The control unit 11 may determine that the state of the robot 40 is in a singular point state when the numerical value representing the state of the robot 40 matches the numerical value representing the singular point state. The control unit 11 may determine that the state of the robot 40 is in a singular point state when the difference between the numerical value representing the state of the robot 40 and the numerical value representing the singular point state is less than a predetermined value. The numerical value representing the state of the robot 40 may include, for example, the angle of the joint of the arm 42 or the torque of the motor that drives the robot 40.

[0045] As described above, the control unit 11 sets the calibration range 60 and sets the calibration positions of the first and second positions within the calibration range 60. The control unit 11 can also generate a calibration item as information that specifies the mark position and orientation of the robot 40 when the robot 40 is moved to the calibration position.

[0046] <<Performing calibration>> The control unit 11 performs a first calibration so that the calibration item for the tip position and orientation related to the recognition result of the mark 46 matches the calibration item for the mark position and orientation related to the detection result of the sensor 48 of the robot 40. Specifically, the control unit 11 moves the robot 40 to the calibration position. The control unit 11 acquires the recognition result of the mark 46 of the robot 40 by using the spatial information acquisition unit 20 when the robot 40 moves to the calibration position. The control unit 11 calculates the relative positional relationship of the calibration item for the mark position and orientation acquired as the recognition result of the mark 46 with respect to the calibration item for the mark position and orientation based on the sensor 48 of the robot 40. The relative positional relationship corresponds to the coordinate difference and angle difference between the mark position and orientation specified by both calibration items and the recognition result of the mark 46. The control unit 11 corrects the coordinate system of the spatial information acquisition unit 20 to align it with the coordinate system of the robot 40 so that the coordinate error and angle error corresponding to the relative positional relationship with respect to both calibration items are zero or close to zero (i.e., so that the error is less than a predetermined value). By doing this, the control unit 11 can calculate the relative positional relationship by matching the recognition result of the mark 46 when the robot 40 moves to the calibration position with the mark position and posture identified by the sensor 48 of the robot 40.

[0047] The control unit 11 can set a calibration position by generating a calibration item. Conversely, the calibration position corresponds to a position to which the robot 40 is moved in order to generate the calibration item. The control unit 11 can perform calibration by moving the robot 40 to the calibration position by applying the calibration item to the control of the robot 40. Specifically, the control unit 11 performs a first calibration to correct the (X_CA, Y_CA, Z_CA) coordinate system of the spatial information acquisition unit 20 to match it with the (X_RB, Y_RB, Z_RB) coordinate system of the robot 40. The control unit 11 may specify the relationship between the coordinate system of the spatial information acquisition unit 20 and the coordinate system of the robot 40 by performing the first calibration.

[0048] To improve the accuracy of the first calibration, the control unit 11 may perform further calibration using a measurement object 52 such as a pin arranged in the operating space of the robot 40. The calibration performed using the measurement object 52 to improve the accuracy of the first calibration is also referred to as a second calibration.

[0049] Specifically, the control unit 11 recognizes the measurement object 52 based on the image of the spatial information acquisition unit 20 and acquires the position of the measurement object 52. The control unit 11 may recognize the measurement object 52 by image recognition of an image of the measurement object 52. The measurement object 52 may have a measurement object mark for recognizing the position of the measurement object 52 in the image. The mark attached to the measurement object 52 may be configured identically to the mark 46 mounted on the robot 40. For example, in the present disclosure, the object mark may be referred to as the mark 46. The control unit 11 may acquire the position of the measurement object 52 based on an image of the measurement object mark of the measurement object 52. In this embodiment, the control unit 11 presses the first finger 441 or the second finger 442 of the end effector 44 against the measurement object 52. The control unit 11 detects the load acting from the measurement object 52 on the first finger 441 or the second finger 442 using a force sensor 444 or a tactile sensor 445. The control unit 11 calculates the position of the measurement object 52 based on the detection result of the force sensor 444 or the tactile sensor 445. The control unit 11 may estimate whether or not a coordinate system that affects the operation of the robot 40 can be corrected based on the calculation result of the position of the measurement object 52, and may correct the coordinate system if correction is possible. Note that, as will be described later, the second calibration may be performed in a state where the robot 40 is in contact with the measurement object 52 by manual operation.

[0050] <<<Coordinate system correction>>> The number of measurement objects 52 placed in the motion space is not limited to one and may be two or more. That is, one or more measurement objects 52 may be placed in the motion space. The control unit 11 detects one or more measurement objects 52 captured in the motion space image and acquires the coordinates of each measurement object 52 in the coordinate system of the space information acquisition unit 20. The control unit 11 also acquires the coordinates of each measurement object 52 based on position information of the end effector 44 relative to each measurement object 52. Specifically, in this embodiment, based on the detection result of the load acting on the end effector 44 when the end effector 44 is brought into contact with the measurement object 52, the control unit 11 acquires the coordinates of each measurement object 52 in the coordinate system of the robot 40, assuming that the position of the end effector 44 relative to each measurement object 52 is 0 (zero) or a constant distance while the load is being detected. The control unit 11 may perform the second calibration by correcting the coordinate system of the spatial information acquisition unit 20 or the coordinate system of the robot 40 based on the coordinates of each measurement object 52 in the coordinate system of the spatial information acquisition unit 20 and the coordinates of each measurement object 52 in the coordinate system of the robot 40. Specifically, for example, the coordinate system may be corrected so that depth information generated in the (X_CA, Y_CA, Z_CA) coordinate system by the spatial information acquisition unit 20 coincides with the (X_RB, Y_RB, Z_RB) coordinate system of the robot 40. The control unit 11 may correct the coordinate system in, for example, a rotational direction or a translational direction. The control unit 11 may correct the coordinate system so as to enlarge or reduce, for example. The control unit 11 may correct distortion of the coordinate system, for example.

[0051] <<<Configuration example of the measurement object 52>>> The measurement object 52 may be configured so as to be less likely to deform when contacted by the end effector 44. For example, the rigidity of the measurement object 52 may be increased so that the measurement object 52 is not deformed by contact with the end effector 44. The rigidity of the measurement object 52 may be determined based on the magnitude and direction of the load acting on the end effector 44 and the measurement object 52 when the end effector 44 is brought into contact with the measurement object 52.

[0052] The measurement object 52 may be configured to have a portion that is smaller than the distance between the first finger 441 and the second finger 442 when the first finger 441 and the second finger 442 are in the widest open state. In this way, the control unit 11 can hold the measurement object 52 between the first finger 441 and the second finger 442. The measurement object 52 may be configured to have a size that allows it to carry a measurement object mark that appears in an image of the spatial information acquisition unit 20. The measurement object 52 may be configured to have a size that allows its position to be recognized in the image of the spatial information acquisition unit 20.

[0053] The measurement object 52 may be placed in a space where the robot 40 performs a task. The measurement object 52 is located, for example, within the calibration range 60. The measurement object 52 may be configured so that the position at which it is contacted by the end effector 44 can be easily moved. For example, when the measurement object 52 is placed on a work table 70 having an upper surface extending along the X_RB axis and the Y_RB axis, the measurement object 52 may be configured so that its height in the Z_RB axis direction is adjustable. The measurement object 52 may be configured as an assemblable block so that it can be stacked in the Z_RB axis direction.

[0054] <<Flowchart example>> The control unit 11 of the robot control device 10 may execute a robot control method including the steps of the flowchart illustrated in Fig. 4 to perform a second calibration to improve the accuracy of the first calibration. The robot control method may be realized as a robot control program executed by a processor constituting the control unit 11. The robot control program may be stored in a non-transitory computer-readable medium.

[0055] The control unit 11 recognizes the measurement object 52 based on the image of the spatial information acquisition unit 20 (step S1). The control unit 11 brings the end effector 44 of the robot 40 into contact with the measurement object 52 (step S2). The control unit 11 acquires the detection result of the force sensor 444 or the tactile sensor 445 when the end effector 44 comes into contact with the measurement object 52, and detects the load (step S3). As will be described later, the end effector 44 of the robot 40 may come into contact with the measurement object 52 by manual operation. Therefore, the control unit 11 may acquire not only the detection result of the force sensor 444 or the tactile sensor 445 when the end effector 44 comes into contact with the measurement object 52, but also the detection result of the state when the end effector 44 is in contact with the measurement object 52 by manual operation. The detection result of the force sensor 444 or the tactile sensor 445 when the end effector 44 comes into contact with the measurement object 52 and the detection result of the state when the end effector 44 is in contact with the measurement object 52 by manual operation are collectively referred to as contact information. The control unit 11 determines whether the end effector 44 has come into contact with all of the measurement objects 52 and detected a load (step S4). If the control unit 11 has not detected a load on all of the measurement objects 52 (step S4: NO), the control unit 11 returns to the procedure of step S1 and repeats the procedures from step S1 to S3. Note that in this embodiment, the end effector 44 is brought into contact with the measurement object 52; however, if position information of the end effector 44 relative to the measurement object 52 is known using, for example, a non-contact distance sensor, the end effector 44 does not need to come into contact with the measurement object 52.

[0056] If the control unit 11 detects loads on all of the measurement objects 52 (step S4: YES), the control unit 11 corrects the coordinate system of the spatial information acquisition unit 20 based on the load detection results (step S5). After executing the procedure of step S5, the control unit 11 ends the execution of the procedure of the flowchart in FIG.

[0057] The control unit 11 may correct the coordinates recognized as the position of the object to be measured 52 in the coordinate system of the spatial information acquisition unit 20 based on the load detected when the end effector 44 comes into contact with the object to be measured 52 in the procedure of step S3.

[0058] The control unit 11 may execute the procedure of the flowchart illustrated in FIG. 4 after calibration based on an image of the spatial information acquisition unit 20, or after calibration by another method.

[0059] <<Example of a Mode in Which the End Effector 44 is Contacted with the Measurement Object 52>> The control unit 11 brings the end effector 44 into contact with the measurement object 52 in various ways, and acquires the position of the measurement object 52 based on the detection result of the force sensor 444 or the tactile sensor 445. Examples of ways in which the end effector 44 is brought into contact with the measurement object 52 will be described below. In this case, the position of the measurement object 52 may be, for example, the center coordinates of the measurement object mark (mark 46) or the coordinates of the edge of the measurement object 52.

[0060] The control unit 11 may bring the end effector 44 into contact with the measurement object 52 so that the measurement object 52 is sandwiched between a first finger 441 and a second finger 442 of the end effector 44, as shown in FIG. 3 . In this case, the inside of the first finger 441 or the second finger 442 comes into contact with the measurement object 52. The control unit 11 may calculate the position of the measurement object 52 based on the result of detection by the force sensor 444 of the load acting from the measurement object 52 on the first finger 441 or the second finger 442 and the position of the first finger 441 or the second finger 442. The control unit 11 may calculate the position of the measurement object 52 based on the detection result by a tactile sensor 445 installed on the inside of the first finger 441 or the second finger 442.

[0061] 3, when the first finger 441 and the second finger 442 are aligned along the X_RB axis, the control unit 11 can calculate the position of the measurement object 52 along the X_RB axis by bringing the inside of the first finger 441 or the second finger 442 into contact with the measurement object 52. That is, the control unit 11 can calculate the X_RB axis component of the coordinates of the measurement object 52. The control unit 11 can calculate the Y_RB axis component of the coordinates of the measurement object 52 by controlling the first finger 441 and the second finger 442 to be aligned along the Y_RB axis and bringing the inside of the first finger 441 or the second finger 442 into contact with the measurement object 52. The control unit 11 can calculate the Z_RB axis component of the coordinates of the measurement object 52 by controlling the first finger 441 and the second finger 442 to be aligned along the Z_RB axis and bringing the inside of the first finger 441 or the second finger 442 into contact with the measurement object 52. The control unit 11 can calculate the three-dimensional coordinates of the measurement object 52 by controlling the first finger 441 and the second finger 442 to be aligned along each of the three axes and bringing the inside of the first finger 441 or the second finger 442 into contact with the measurement object 52. The control unit 11 can also calculate the three-dimensional coordinates of the measurement object 52 by controlling the first finger 441 and the second finger 442 to be aligned along three directions independent of each other and bringing the inside of the first finger 441 or the second finger 442 into contact with the measurement object 52. In this case, for example, each component of the coordinates (X_RB, Y_RB, Z_RB) of the measurement object 52 may be an intermediate value of each component of the coordinates (X_RB, Y_RB, Z_RB) of the first finger 441 or the second finger 442.

[0062] As shown in FIGS. 5A, 5B, and 5C, for example, the control unit 11 may contact the measurement object 52 with surfaces other than the inner surfaces of the first finger 441 and the second finger 442 of the end effector 44 while the first finger 441 and the second finger 442 are in an open state. The first finger 441 and the second finger 442 are aligned in the X_RB axis direction. The control unit 11 can calculate the coordinates of the measurement object 52 by moving the end effector 44 along each of the three axes and bringing it into contact with the measurement object 52. The control unit 11 may also move the end effector 44 along three mutually independent directions and bring it into contact with the measurement object 52. Note that in this case, for example, an end effector 44 having a single suction nozzle can also be used.

[0063] 5A , the control unit 11 moves the end effector 44 in the negative direction of the X_RB axis to bring the outer side of the finger 441 (the negative side of the X_RB axis) into contact with the measurement object 52. In this case, the control unit 11 can calculate the position of the measurement object 52 in the X_RB axis direction based on the result of detection of the load acting on the finger 441 by the force sensor 444 and the position of the finger 441. If a tactile sensor 445 is installed on the outer side of the finger 441, the control unit 11 can calculate the position of the measurement object 52 in the X_RB axis direction based on the detection result of the tactile sensor 445 and the position of the finger 441. Note that in this case, for example, the X_RB component of the coordinates of the measurement object 52 may be a value obtained by adding or subtracting half the width of the measurement object 52 to or from the X_RB component of the coordinates of the outer surface of the finger 441, taking into account the dimensions of the measurement object 52.

[0064] 5B , the control unit 11 moves the end effector 44 in the positive direction of the Y_RB axis, and brings the side surface of the first finger 441 (the negative side of the X_RB axis) into contact with the measurement object 52. In this case, the control unit 11 can calculate the position of the measurement object 52 in the Y_RB axis direction based on the result of detection of the load acting on the first finger 441 by the force sensor 444 and the position of the first finger 441. If a tactile sensor 445 is installed on the side surface of the first finger 441, the control unit 11 can calculate the position of the measurement object 52 in the Y_RB axis direction based on the detection result of the tactile sensor 445 and the position of the first finger 441. Note that in this case, for example, the Y_RB component of the coordinates of the measurement object 52 may be a value obtained by adding or subtracting half the width of the measurement object 52 to or from the Y_RB component of the coordinates of the side surface of the first finger 441, taking into account the dimensions of the measurement object 52.

[0065] 5C , the control unit 11 moves the end effector 44 in the negative direction of the Z_RB axis, and brings the tip of the first finger 441 (on the negative side of the Z_RB axis) into contact with the surface of the measurement object 52 on the positive side of the Z_RB axis. In this case, the control unit 11 can calculate the position of the measurement object 52 in the Z_RB axis direction based on the result of detection of the load acting on the first finger 441 by the force sensor 444 and the position of the first finger 441. If a tactile sensor 445 is installed at the tip of the first finger 441, the control unit 11 can calculate the position of the measurement object 52 in the Z_RB axis direction based on the detection result of the tactile sensor 445 and the position of the first finger 441. Note that in this case, for example, the Z_RB component of the coordinates of the measurement object 52 may be the same as the Z_RB component of the coordinates of the tip surface of the first finger 441.

[0066] For example, as shown in FIGS. 6A, 6B, and 6C, the control unit 11 may bring the first finger 441 or the second finger 442 of the end effector 44 into contact with the measurement object 52 while the first finger 441 and the second finger 442 are closed. The control unit 11 can calculate the coordinates of the measurement object 52 by moving the end effector 44 along each of three axes to bring it into contact with the measurement object 52. The control unit 11 may also move the end effector 44 along three mutually independent directions to bring it into contact with the measurement object 52. Note that, although the following example describes an example in which the end effector 44 is brought into contact with only one side of each of the three axes or directions, it may also be brought into contact with both sides of each of the three axes or directions.

[0067] 6A , the control unit 11 moves the end effector 44 in the negative direction of the X_RB axis to bring the outer side of the finger 441 (the negative side of the X_RB axis) into contact with the measurement object 52. In this case, the control unit 11 can calculate the position of the measurement object 52 in the X_RB axis direction based on the result of detection of the load acting on the finger 441 by the force sensor 444 and the position of the finger 441. If a tactile sensor 445 is installed on the outer side of the finger 441, the control unit 11 can calculate the position of the measurement object 52 in the X_RB axis direction based on the detection result of the tactile sensor 445 and the position of the finger 441. Note that in this case, for example, the X_RB component of the coordinates of the measurement object 52 may be a value obtained by adding or subtracting half the width of the measurement object 52 to or from the X_RB component of the coordinates of the outer surface of the finger 441, taking into account the dimensions of the measurement object 52.

[0068] 6B , the control unit 11 moves the end effector 44 in the positive direction of the Y_RB axis, and brings the side surface (negative side of the X_RB axis) of the first finger 441 or the second finger 442 into contact with the measurement object 52. In this case, the control unit 11 can calculate the position of the measurement object 52 in the Y_RB axis direction based on the result of detection of the load acting on the first finger 441 or the second finger 442 by the force sensor 444 and the position of the first finger 441 or the second finger 442. If a tactile sensor 445 is installed on the side surface of the first finger 441 or the second finger 442, the control unit 11 can calculate the position of the measurement object 52 in the Y_RB axis direction based on the detection result of the tactile sensor 445 and the position of the first finger 441 or the second finger 442. In this case, for example, the Y_RB component of the coordinates of the measurement object 52 may be set to a value obtained by adding or subtracting half the width of the measurement object 52 to or from the Y_RB component of the coordinates of the side surface of the first finger 441, taking into account the dimensions of the measurement object 52.

[0069] 6C , the control unit 11 moves the end effector 44 in the negative direction of the Z_RB axis to bring the tip (negative side of the Z_RB axis) of the first finger 441 or the second finger 442 into contact with the measurement object 52. In this case, the control unit 11 can calculate the position of the measurement object 52 in the Z_RB axis direction based on the result of detection of the load acting on the first finger 441 or the second finger 442 by the force sensor 444 and the position of the first finger 441 or the second finger 442. If a tactile sensor 445 is installed at the tip of the first finger 441 or the second finger 442, the control unit 11 can calculate the position of the measurement object 52 in the Z_RB axis direction based on the detection result of the tactile sensor 445 and the position of the first finger 441 or the second finger 442. In this case, for example, the Z_RB component of the coordinates of the measurement object 52 may be the same as the Z_RB component of the coordinates of the tip surface of the first finger 441.

[0070] 7A, 7B, and 7C, the control unit 11 may move the end effector 44 in the X_RB axis direction or the Y_RB axis direction while the first finger 441 or the second finger 442 is in contact with the surface of the measurement object 52 on the positive side of the Z_RB axis. In other words, the control unit 11 may move the end effector 44 in the surface direction of the upper surface while the end effector 44 is in contact with the upper surface of the measurement object 52. In this way, the control unit 11 can calculate the coordinate of the Z_RB axis of the measurement object 52 and the coordinate of the X_RB axis or the Y_RB axis all at once. In this case, the measurement object 52 may be placed in the motion space so that the surface (upper surface) on the positive side of the Z_RB axis of the measurement object 52 is reflected in the motion space image.

[0071] Specifically, in a first step, the control unit 11 brings the first finger 441 or the second finger 442 into contact with the surface of the measurement object 52 on the positive side of the Z_RB axis. The control unit 11 can calculate the position of the measurement object 52 in the Z_RB axis direction based on the detection result of the force sensor 444 or the tactile sensor 445 and the position of the first finger 441 or the second finger 442.

[0072] In the second stage, the control unit 11 moves the first finger 441 or the second finger 442 along the X_RB axis or the Y_RB axis. As a result of moving the first finger 441 or the second finger 442 along the X_RB axis or the Y_RB axis, the first finger 441 or the second finger 442 moves out of contact with the surface of the object 52 on the positive side of the Z_RB axis. In other words, the first finger 441 or the second finger 442 is no longer in contact with the surface of the object 52 on the positive side of the Z_RB axis. The control unit 11 can calculate the position of the object 52 in the X_RB axis direction or the Y_RB axis direction based on the detection result that the first finger 441 or the second finger 442 has lost contact with the surface of the object 52 on the positive side of the Z_RB axis.

[0073] The control unit 11 may detect a change in the force acting on the first finger 441 or the second finger 442 in the Z_RB axis direction based on the detection result of the force sensor 444 or the tactile sensor 445. The control unit 11 can calculate the position of the measurement object 52 in the X_RB axis direction or the Y_RB axis direction based on the change in the force in the Z_RB axis direction. For example, the control unit 11 moves the first finger 441 or the second finger 442 in the X_RB axis direction. The control unit 11 may determine that the first finger 441 or the second finger 442 has reached the end of the measurement object 52 in the X_RB axis direction when the load acting on the first finger 441 or the second finger 442 in the Z_RB axis direction decreases by a predetermined value or more. The control unit 11 may calculate the position at which it is determined that the first finger 441 or the second finger 442 has reached the end of the measurement object 52 in the X_RB axis direction as the position of the measurement object 52 in the X_RB axis direction.

[0074] 7A , the control unit 11 moves the end effector 44 in the positive direction of the X_RB axis while the first finger 441 or the second finger 442 is in contact with the surface of the measurement object 52 on the positive side of the Z_RB axis. In this case, the control unit 11 can calculate the position of the measurement object 52 in the Z_RB axis direction based on the result of detection by the force sensor 444 or the tactile sensor 445 of the load acting on the first finger 441 or the second finger 442 and the measurement object 52 and the position of the first finger 441 or the second finger 442. Furthermore, the control unit 11 can calculate the position of the measurement object 52 in the X_RB axis direction based on the detection result of the loss of contact between the first finger 441 or the second finger 442 and the measurement object 52 and the position of the first finger 441 or the second finger 442.

[0075] 7B , the control unit 11 moves the end effector 44 in the positive direction of the Y_RB axis while the first finger 441 or the second finger 442 is in contact with the surface of the measurement object 52 on the positive side of the Z_RB axis. In this case, the control unit 11 can calculate the position of the measurement object 52 in the Z_RB axis direction based on the result of detection of the load acting on the first finger 441 or the second finger 442 by the force sensor 444 or the tactile sensor 445 and the position of the first finger 441 or the second finger 442. Furthermore, the control unit 11 can calculate the position of the measurement object 52 in the Y_RB axis direction based on the detection result when the first finger 441 or the second finger 442 loses contact with the measurement object 52 and the position of the first finger 441 or the second finger 442.

[0076] 7C , the control unit 11 simultaneously moves the end effector 44 in the positive direction of the X_RB axis and the positive direction of the Y_RB axis with the first finger 441 or the second finger 442 in contact with the surface of the measurement object 52 on the positive side of the Z_RB axis. That is, the control unit 11 moves the first finger 441 or the second finger 442 in a direction oblique to each of the X_RB axis and the Y_RB axis. In this case, the control unit 11 can calculate the position of the measurement object 52 in the Z_RB axis direction based on the result of detection of the load acting on the first finger 441 or the second finger 442 by the force sensor 444 or the tactile sensor 445 and the position of the first finger 441 or the second finger 442. Furthermore, the control unit 11 can calculate the position in the X_RB axis direction and the position in the Y_RB axis direction of the measurement object 52 based on the detection result when the first finger 441 or the second finger 442 loses contact with the measurement object 52 and the position of the first finger 441 or the second finger 442.

[0077] In the method of moving the end effector 44 in different directions while keeping it in contact with the object to be measured 52, the first finger 441 and the second finger 442 may be in a closed or open state as shown in Figures 7A, 7B, and 7C.

[0078] For example, as shown in FIG. 8 , the control unit 11 may rotate the end effector 44 so that the first finger 441 and the second finger 442 pinch the measurement object 52 between them and move the first finger 441 and the second finger 442 along the outer periphery of the measurement object 52. In this case, the shape of the measurement object 52 may be a cylinder with a circular upper surface. The control unit 11 may control the robot 40 to move at least one of the first finger 441 and the second finger 442 so as to be close to the other finger, and to rotate around the measurement object 52 while the first finger 441 and the second finger 442 are in contact with the measurement object 52. The control unit 11 may rotate the end effector 44 based on the detection results of tactile sensors 445 installed inside the first finger 441 and the second finger 442. The control unit 11 can calculate the central coordinates of the measurement object 52 in the X_RB axis direction and the Y_RB axis direction based on the trajectory of the movement of the first finger 441 or the second finger 442. The shape of the measurement object 52 is not limited to a cylinder, but may be a prism shape having a polygonal upper surface. The shape of the measurement object 52 is not limited to these and may be various other shapes. The measurement object 52 may be placed in the operation space so that the upper surface of the measurement object 52 is reflected in the operation space image.

[0079] As described above, the control unit 11 may control the robot 40 so that the end effector 44 contacts at least one surface of the measurement object 52. By bringing the end effector 44 into contact with at least one surface of the measurement object 52, the control unit 11 can improve the accuracy of detecting the position of the measurement object 52. In the above example, the coordinates of the measurement object 52 are calculated based on the end effector 44 contacting the measurement object 52. However, if the positional relationship between the end effector 44 and the measurement object 52 is known, the end effector 44 does not need to contact the measurement object 52. In other words, if a distance sensor capable of non-contact measurement is mounted on the first finger 441 of the end effector 44, the coordinates of the measurement object 52 can be calculated as described above based on the coordinates of the first finger 441 and the distance from the first finger 441 to the measurement object 52. Furthermore, each component of the coordinates of the measurement object 52 may be calculated based on different sensors.

[0080] The control unit 11 may control the robot 40 so that the end effector 44 comes into contact with a surface of the measurement object 52 that is displayed in the motion space image. In this way, the control unit 11 can easily correct the position of the measurement object 52 based on the spatial information acquisition unit 20, using the position of the measurement object 52 based on the contact with the measurement object 52. As a result, the accuracy of the coordinate system of the spatial information acquisition unit 20 can be improved.

[0081] In the above-described embodiments of the robot control device 10 and the robot control method, the first calibration or the second calibration of the robot 40 configured to grasp the object 50 with the first finger 441 and the second finger 442 has been described. The second calibration aspect in the present embodiment is not limited to being applied to the robot 40 having two fingers, but may also be applied to the robot 40 having three or more fingers, or to the robot 40 having a holder including a suction nozzle, fingers, or the like. Furthermore, the second calibration aspect in the present embodiment may also be applied to the robot 40 having a jig such as a measuring rod in addition to the holder.

[0082] (Small summary) As described above, according to the robot control device 10 and robot control method of this embodiment, the coordinates of the measurement object 52 are detected by bringing the robot 40 into contact with the measurement object 52 arranged in the operating space and detecting the load. In this way, the coordinates of the measurement object 52 based on the image of the spatial information acquisition unit 20 can be corrected to match the coordinates of the measurement object 52 based on the contact of the robot 40. Furthermore, the coordinate system of the spatial information acquisition unit 20 can be corrected to match the coordinate system of the robot 40. As a result, the calibration accuracy of the robot 40 can be improved.

[0083] Furthermore, because the coordinates or coordinate system can be corrected by the contact of the robot 40 with the measurement target 52, the accuracy of calibration can be ensured even if the accuracy of calculation of depth information in the spatial information acquisition unit 20 is reduced. As a result, the cost of the spatial information acquisition unit 20 can be reduced. Furthermore, the effect on the accuracy of calibration due to changes in the configuration or arrangement of the spatial information acquisition unit 20 can be reduced.

[0084] Furthermore, since the coordinates or coordinate system can be corrected by the contact of the robot 40 with the measurement target 52, the accuracy of the calibration can be ensured without the need for visual confirmation by an operator. As a result, the workload and work costs can be reduced. Furthermore, the automation of the calibration can be promoted. Furthermore, the coordinates or coordinate system can be corrected even if the workspace of the robot 40 is not of a uniform height.

[0085] In the above example, the robot 40 is installed on the workbench 70, but the robot 40 may be installed on a support table other than the workbench 70. Even if the robot 40 is located on a support table other than the workbench 70, the work target and the like are installed on the workbench 70, and therefore the coordinate system of the operating space is also the coordinate system of the workbench 70. Therefore, by performing the calibration of the present disclosure, the robot 40 can perform accurate work even if it is not installed on the workbench 70.

[0086] (Other embodiments) Other embodiments are described below.

[0087] <When performing the second calibration first> In the above-described embodiment, the control unit 11 of the robot control device 10 first performs the first calibration between the coordinate system of the robot 40 and the coordinate system of the spatial information acquisition unit 20, and then performs the second calibration based on the measurement object 52. However, in another embodiment of the present disclosure, the control unit 11 of the robot control device 10 may perform the second calibration first.

[0088] Specifically, the control unit 11 of the robot control device 10 may acquire the coordinates of the measurement object 52 in the coordinate system of the robot 40 for the measurement object 52 installed at a specific position, as in the first embodiment. Then, the control unit 11 may perform calibration by correcting the robot coordinates using the acquired coordinates of the measurement object 52 in the coordinate system of the robot 40 as the origin of the coordinate system of the robot 40. Alternatively, the control unit 11 may acquire coordinates (X, Y, Z) of the motion space of the specific location, for example, through user input, and correct the coordinate system of the robot 40 with respect to the coordinate system of the motion space based on the input motion space coordinates, thereby performing calibration. Note that the robot 40 may be manually brought into contact with the measurement object 52, or may be manually moved to the vicinity of the measurement object 52 and then moved in a certain direction by the control unit 11 to bring the robot 40 into contact with the measurement object 52.

[0089] When the robot control device 10 receives input from a user, the robot control device 10 may acquire the input from the user via the interface 13. In this case, the robot control system 1 may further include a terminal device having a user interface connected to the interface 13 of the robot control device 10 by wire or wirelessly. The user interface inputs information from the user and outputs information to the user. The user interface may include, for example, a touch sensor. The touch sensor detects contact with a user's finger or stylus pen, and identifies the contact position. The touch sensor may be integrated with a display to form a touch panel display.

[0090] Furthermore, when the robot control system 1 does not include a camera as the spatial information acquisition unit 20, the control unit 11 may complete calibration and start the work of the robot 40. When the robot control system 1 includes a camera as the spatial information acquisition unit 20, the control unit 11 may perform calibration between the coordinate system of the spatial information acquisition unit 20 and the coordinate system of the operation space, or calibration between the coordinate system of the robot 40 and the coordinate system of the spatial information acquisition unit 20, and determine a transformation formula between the coordinate system of the robot 40 and the coordinate system of the spatial information acquisition unit 20. Note that when the positions of the robot 40 and the spatial information acquisition unit 20 are fixed from the beginning, the transformation between the coordinate system of the robot 40 and the coordinate system of the spatial information acquisition unit 20 is possible without performing calibration between the coordinate system of the robot 40 and the coordinate system of the spatial information acquisition unit 20, so calibration is not necessary.

[0091] <Other Configuration Examples of the Measurement Object 52> 9A, the measurement object 52 may be configured to have a mark 46. The measurement object 52 may be configured to have a mark 46 that functions as a measurement object mark. The measurement object 52 may be configured to have a measurement object mark and a mark 46 separately.

[0092] Furthermore, the measurement object 52 may have an isotropic planar shape when viewed from above, and the upper surface may be flat. The measurement object 52 may be, for example, a polygonal prism such as a cube, a rectangular parallelepiped, a quadrangular prism, or a triangular prism. When the measurement object 52 has a shape with an upper surface including corners or straight sides, such as a polygonal prism, the presence of the corners or sides of the measurement object 52 can improve the accuracy of identifying the coordinate position when the end effector 44 is moved along the upper surface of the measurement object 52. In this way, the space occupied by the jig for performing the first calibration and the second calibration in the operating space can be reduced.

[0093] 9B, the measurement object 52 may be configured to have the mark 46 and also have a portion that is contacted by the end effector 44 of the robot 40. In this way, the upper surface of the measurement object 52 can be used for the second calibration. As a result, the accuracy of the second calibration can be improved.

[0094] 9C, the measurement object 52 may be configured not to have the mark 46. When the robot control system 1 does not include a camera as the spatial information acquisition unit 20, the second calibration can be performed by manually bringing the robot 40 close to the measurement object 52 in the configuration illustrated in FIG.

[0095] 9D, the measurement object 52 may be cylindrical and may have the mark 46 on the upper surface of the cylindrical object 52. In this way, the accuracy of the second calibration may be improved.

[0096] <Example of arrangement of measurement object 52> 10, the measurement target 52 may be placed at a diagonal position on the top surface of the work table 70. In this way, even if the top surface of the work table 70 is tilted or distorted, this can be corrected in the coordinate system of the operating space of the robot 40.

[0097] Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art could make various modifications or alterations based on the present disclosure. Therefore, it should be noted that these modifications or alterations are included in the scope of the present disclosure. For example, the functions included in each component can be rearranged so as not to cause logical inconsistencies, and multiple components can be combined or divided into one.

[0098] All of the features described in this disclosure and / or all steps of all of the disclosed methods or processes may be combined in any combination except combinations in which these features are mutually exclusive. Furthermore, each feature described in this disclosure may be replaced by an alternative feature serving the same, equivalent, or similar purpose, unless expressly denied. Thus, unless expressly denied, each disclosed feature is only one example of a generic series of identical or equivalent features.

[0099] Furthermore, embodiments of the present disclosure are not limited to the specific configurations of any of the above-described embodiments, but rather extend to any novel feature or combination thereof described herein, or any novel method or process step or combination thereof described herein.

[0100] Furthermore, in this disclosure, descriptions such as "first" and "second" are identifiers for distinguishing the configurations. Configurations distinguished by descriptions such as "first" and "second" in this disclosure can have their numbers swapped. For example, the first calibration can have its identifiers "first" and "second" swapped with the second calibration. The identifier swapping is performed simultaneously. The configurations remain distinguished even after the identifier swapping. Identifiers may be deleted. A configuration from which an identifier has been deleted is distinguished by a symbol. The identifiers "first" and "second" in this disclosure should not be used solely to interpret the order of the configurations or to justify the existence of an identifier with a smaller number. [Explanation of symbols]

[0101] 1. Robot Control System 10 Robot control device (11: control unit, 12: memory unit, 13: interface) 20 Spatial information acquisition section 40 Robot (410: control unit, 42: arm, 44: end effector, 440: control unit, 441: first finger, 442: second finger, 443: drive unit, 444: force sensor, 445: tactile sensor, 46: mark, 48: sensor, 49: interface) 50 objects 52 Measurement object (54: Contactable range) 60 calibration ranges 70 Workbench

Claims

1. a control unit that controls a robot having a sensor and an end effector; The control unit acquiring position information of the end effector relative to a measurement object disposed in a motion space of the robot; correcting a coordinate system related to the robot's motion based on the position information; Robot control device.

2. The control unit acquiring coordinates of the measurement object based on spatial information relating to the motion space; controlling the robot to move the end effector to the object to be measured based on the coordinates of the object to be measured; correcting a coordinate system relating to the operation of the robot based on position information of the end effector relative to the measurement object; The robot control device according to claim 1 .

3. the measurement object has a measurement object mark; The robot control device according to claim 2 , wherein the control unit calculates coordinates of the measurement object based on spatial information relating to the measurement object mark.

4. The robot control device according to claim 1 , wherein the control unit corrects a coordinate system relating to the operation of the robot based on position information of the end effector relative to the plurality of measurement targets.

5. The robot control device according to claim 1 , wherein the control unit acquires, as the position information, contact information based on contact of the end effector with the measurement object.

6. 6. The robot control device according to claim 1, wherein the control unit corrects a coordinate system based on spatial information regarding the operating space of the robot based on information detected when the end effector is moved while in contact with the object to be measured, and the contact between the end effector and the object to be measured is lost.

7. The robot control device according to claim 6 , wherein the control unit moves the end effector in a planar direction of the upper surface while the end effector is in contact with the object to be measured from above.

8. the end effector includes a holder; The robot control device according to claim 1 , wherein the control unit controls the robot to move the holding unit, thereby bringing the end effector into contact with the measurement object.

9. 9. The robot control device according to claim 8, wherein, when the holding portion includes at least two fingers, the control unit controls the robot to move at least one of the fingers so as to be in close proximity to the other fingers and to rotate around the measurement object while in contact with the measurement object.

10. The robot control device according to claim 1 , wherein the control unit controls the robot so that the end effector comes into contact with at least one surface of the object to be measured.

11. The robot control device according to claim 10 , wherein the control unit controls the robot so that the end effector contacts a surface of the object to be measured that is shown in an operating space image included in spatial information regarding the operating space of the robot.

12. The shape of the measurement object is a cylindrical shape having a circular upper surface or a prismatic shape having a polygonal upper surface, The robot control device according to claim 1 , wherein the object to be measured is placed in the operating space so that an upper surface of the object to be measured appears in an operating space image included in spatial information regarding the operating space of the robot.

13. A robot control system comprising: the robot control device according to any one of claims 1 to 12; and the robot.

14. A robot control method for controlling a robot having a sensor and an end effector, comprising: acquiring position information of the end effector relative to a measurement object disposed within an operating space of the robot; correcting a coordinate system relating to the operation of the robot based on the position information; A robot control method comprising:

Images