ROBOT CONTROL DEVICE, ROBOT CONTROL SYSTEM, AND ROBOT CONTROL METHOD

The robot control device employs a two-tiered calibration method to reduce calibration time and enhance precision, addressing inefficiencies in conventional calibration processes by separating first and second calibrations based on density.

JP7583942B2Active Publication Date: 2024-11-14KYOCERA CORP
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
JP2023540414
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2021-08-04
Filing Date
2022-08-04
Publication Date
2024-11-14
Estimated Expiration
2042-08-04

AI Technical Summary

Technical Problem

Conventional robot calibration processes are time-consuming and inefficient, necessitating a need for improved methods to shorten calibration work.

Method used

A robot control device and method that employs a two-tiered calibration approach, including a first calibration at lower density positions and a second calibration at higher density positions, allowing for efficient and precise robot operation.

Benefits of technology

The two-tiered calibration method reduces the overall calibration workload while ensuring high precision and efficiency in robot operations, enabling the robot to perform tasks accurately and quickly.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007583942000001
    Figure 0007583942000001
  • Figure 0007583942000002
    Figure 0007583942000002
  • Figure 0007583942000003
    Figure 0007583942000003
Patent Text Reader

Abstract

This robot control device comprises a control unit. The control unit executes first calibration of a robot in one or more first calibration positions included in a first calibration range set in the operating space of the robot and executes second calibration of the robot in one or more second calibration positions that are included in a second calibration range constituting part of the first calibration range and in which the density is set higher than that of the one or more first calibration positions.
Need to check novelty before this filing date? Find Prior Art

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to Japanese Patent Application No. 2021-128491 (filed August 4, 2021), the entire disclosure of which is incorporated herein by reference. [Technical field]

[0002] The present disclosure relates to a robot control device, a robot control system, and a robot control method. [Background technology]

[0003] Conventionally, a robot vision system that enables calibration work to be performed automatically and without hassle is known (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2010-172986 A Summary of the Invention

[0005] A robot control device according to an embodiment of the present disclosure includes a control unit that controls a robot. The control unit executes a first calibration of the robot at at least one first calibration position included in a first calibration range set in an operating space of the robot. The control unit executes a second calibration of the robot at at least one second calibration position. The at least one second calibration position is included in a second calibration range that is a portion of the first calibration range, and is set at a higher density than the at least one first calibration position.

[0006] A robot control system according to an embodiment of the present disclosure includes the robot control device and the robot.

[0007] A robot control method according to an embodiment of the present disclosure includes a control unit that controls a robot performing a first calibration of the robot at at least one first calibration position included in a first calibration range set in a motion space of the robot, and a control unit that performs a second calibration of the robot at at least one second calibration position included in a second calibration range that is a part of the first calibration range and set at a higher density than the at least one first calibration position. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of the configuration of a robot control system according to an embodiment. [Diagram 2] FIG. 1 is a schematic diagram illustrating an example of the configuration of a robot control system according to an embodiment. [Diagram 3] FIG. 13 is a schematic diagram showing an example of a calibration range. [Figure 4] 5 is a flowchart showing an example of a procedure for executing a first calibration as a robot control method according to an embodiment. [Diagram 5] 10 is a flowchart illustrating an example of a procedure for performing a second calibration as a robot control method according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] In robot operations, there is a demand for shortening the calibration work. According to a robot control device, a robot control system, and a robot control method according to an embodiment of the present disclosure, the calibration work can be shortened.

[0010] (Overview of Robot Control System 1) As illustrated in FIG. 1 and FIG. 2, a robot control system 1 according to an embodiment includes a robot 40, a robot control device 10, and a spatial information acquisition unit 20. The robot 40 operates in a predetermined operating space. The spatial information acquisition unit 20 generates depth information of the operating space in which the robot 40 operates. As described later, the spatial information acquisition unit 20 calculates a distance to a measurement point located on the surface of an object 50 existing in the operating space. The distance from the spatial information acquisition unit 20 to the measurement point is also called a depth. The depth information is information about the depth measured for each measurement point. In other words, the depth information is information about the distance to a measurement point located on the surface of an object 50 existing in the operating space. The depth information may be expressed as a depth map that associates a direction as viewed from the spatial information acquisition unit 20 with a depth in that direction. The spatial information acquisition unit 20 generates depth information of the operating space based on an (X, Y, Z) coordinate system. The robot control device 10 operates the robot 40 based on the depth information generated by the spatial information acquisition unit 20. The robot control device 10 operates the robot 40 based on the (X_RB, Y_RB, Z_RB) coordinate system.

[0011] The (X_RB, Y_RB, Z_RB) coordinate system is also referred to as the coordinate system of the robot 40. The (X, Y, Z) coordinate system is also referred to as the coordinate system of the spatial information acquisition unit 20. The coordinate system of the robot 40 may be set to be the same as the coordinate system of the spatial information acquisition unit 20, or may be set to be a different coordinate system. When the coordinate system of the robot 40 is set to be a different coordinate system from the coordinate system of the spatial information acquisition unit 20, the robot control device 10 converts the depth information generated in the coordinate system of the spatial information acquisition unit 20 into the coordinate system of the robot 40 for use.

[0012] The number of robots 40 and robot control devices 10 is not limited to one as illustrated, but may be two or more. The number of spatial information acquisition units 20 may be one for one operating space as illustrated, or may be two or more. Each component will be specifically described below.

[0013] <Robot control device 10> The robot control device 10 includes a control unit 11 and a storage unit 12 .

[0014] The control unit 11 may include at least one processor to realize various functions of the robot control device 10. The processor may execute a program to realize various functions of the robot control device 10. The processor may be realized as a single integrated circuit. The integrated circuit is also called an IC (Integrated Circuit). The processor may be realized as a plurality of integrated circuits and discrete circuits connected in a communicable manner. The processor may include a CPU (Central Processing Unit). The processor may include a DSP (Digital Signal Processor) or a GPU (Graphics Processing Unit). The processor may be realized based on various other known technologies.

[0015] The storage unit 12 may be configured to include an electromagnetic storage medium such as a magnetic disk, or may be configured to include a memory such as a semiconductor memory or a magnetic memory. The storage unit 12 may be configured as a hard disk drive (HDD) or a solid state drive (SSD). The storage unit 12 stores various information and programs executed by the control unit 11. The storage unit 12 may function as a work memory for the control unit 11. The control unit 11 may be configured to include at least a part of the storage unit 12.

[0016] The robot control device 10 may further include a communication device configured to be able to communicate with the spatial information acquisition unit 20 and the robot 40 in a wired or wireless manner. The communication device may be configured to be able to communicate using a communication method based on various communication standards. The communication device may be configured using known communication technology. Detailed descriptions of the hardware, etc. of the communication device will be omitted. The functions of the communication device may be realized by one interface, or may be realized by separate interfaces for each connection destination. The control unit 11 may be configured to be able to communicate with the spatial information acquisition unit 20 and the robot 40. The control unit 11 may be configured to include a communication device.

[0017] <Robot 40> 2, the robot 40 includes an arm 42, an end effector 44 attached to the arm 42, and a mark 46 provided on the end effector 44. Note that the mark 46 may be provided on the arm 42 instead of the end effector 44.

[0018] The arm 42 may be configured, for example, as a 6-axis or 7-axis vertical articulated robot. The arm 42 may be configured as a 3-axis or 4-axis horizontal articulated robot or SCARA robot. The arm 42 may be configured as a 2-axis or 3-axis Cartesian robot. The arm 42 may be configured as a parallel link robot or the like. The number of axes constituting the arm 42 is not limited to those exemplified.

[0019] The end effector 44 may include, for example, a gripping hand configured to grip a work object. The gripping hand may have multiple fingers. The number of fingers of the gripping hand may be two or more. The fingers of the gripping hand may have one or more joints. The end effector 44 may include a suction hand configured to suck the work object. The end effector 44 may include a scooping hand configured to scoop the work object. The end effector 44 may include a tool such as a drill and be configured to perform various processing such as drilling a hole in the work object. The end effector 44 is not limited to these examples and may be configured to perform various other operations.

[0020] The robot 40 can control the position of the end effector 44 by operating the arm 42. The end effector 44 may have an axis that is a reference for the direction in which it acts on a work object. If the end effector 44 has an axis, the robot 40 can control the direction of the axis of the end effector 44 by operating the arm 42. The robot 40 controls the start and end of the operation of the end effector 44 acting on the work object. The robot 40 can move or process the work object by controlling the operation of the end effector 44 while controlling the position of the end effector 44 or the direction of the axis of the end effector 44.

[0021] The robot 40 may further include a sensor that detects the state of each component of the robot 40. The sensor may detect the actual position or posture of each component of the robot 40, or information related to the speed or acceleration of each component of the robot 40. The sensor may detect a force acting on each component of the robot 40. The sensor may detect a current flowing through a motor that drives each component of the robot 40, or a torque of the motor. The sensor can detect information obtained as a result of the actual operation of the robot 40. The robot control device 10 can grasp the result of the actual operation of the robot 40 by acquiring the detection result of the sensor. In other words, the robot control device 10 can acquire the state of the robot 40 based on the detection result of the sensor.

[0022] The robot control device 10 recognizes the position of the mark 46 or the position of the end effector 44 on which the mark 46 is placed based on an image of the mark 46 captured by the spatial information acquisition unit 20. The robot control device 10 also recognizes the state of the robot 40 based on an image of the mark 46 captured by the spatial information acquisition unit 20. The robot control device 10 can calibrate the robot 40 by comparing the state of the robot 40 acquired based on the detection results of the sensor with the state of the robot 40 acquired based on the image capturing the mark 46.

[0023] <Spatial information acquisition unit 20> The spatial information acquisition unit 20 acquires spatial information related to the motion space of the robot 40. The spatial information acquisition unit 20 may photograph the motion space and acquire an image of the motion space as the spatial information. As illustrated in FIG. 2, the spatial information acquisition unit 20 may photograph an object 50 present in the motion space. The spatial information acquisition unit 20 may be configured as a camera. The 3D stereo camera photographs the object 50 present in the motion space, calculates the distance to a measurement point located on the surface of the object 50 present in the motion space as the depth, and generates the depth information. The spatial information acquisition unit 20 may be configured as a 3D stereo camera. The spatial information acquisition unit 20 may be configured as a LiDAR (light detection and ranging). The LiDAR measures the distance to a measurement point located on the surface of the object 50 present in the motion space and generates the depth information. That is, the spatial information acquisition unit 20 may acquire the depth information of the motion space as the spatial information. The spatial information acquisition unit 20 may be configured as various devices without being limited to these. The spatial information acquisition unit 20 may acquire various kinds of information other than the image or depth information of the motion space as the spatial information. The spatial information acquisition unit 20 may include an imaging element. The spatial information acquisition unit 20 may further include an optical system. The spatial information acquisition unit 20 may output an image of the motion space captured to the robot control device 10. The spatial information acquisition unit 20 may generate depth information in the motion space of the robot 40 and output it to the robot control device 10. The spatial information acquisition unit 20 may generate point cloud information in the motion space of the robot 40 and output it to the robot control device 10. That is, the spatial information acquisition unit 20 may output the spatial information in the form of point cloud data. In other words, the point cloud information may have spatial information. The point cloud information is information of a set of measurement points located on the surface of the object 50 existing in the motion space, and is information including coordinate information or color information of each measurement point. It can also be said that the point cloud information is data representing the object 50 in the measurement space by a plurality of points. Since the spatial information is in the form of point cloud data, the data density can be made smaller than that of spatial information based on the initial data acquired by the spatial information acquisition unit 20.

[0024] The spatial information acquisition unit 20 has a FOV (Field Of View). The FOV corresponds to the shooting range of the spatial information acquisition unit 20. The spatial information acquisition unit 20 can shoot the range included in the FOV. The actual field of view size of the spatial information acquisition unit 20 is determined based on the FOV of the spatial information acquisition unit 20 and the depth information. The robot control device 10 can acquire the position and posture of the mark 46 of the robot 40 based on the actual field of view size of the spatial information acquisition unit 20 and the image including the mark 46 of the robot 40 captured by the spatial information acquisition unit 20. Specifically, the robot control device 10 can calculate the position and posture of the mark 46 based on the image by analyzing the image of the mark 46 with a predetermined algorithm. The predetermined algorithm may include, for example, a formula or a table, or may include a program that specifies a calculation process. The predetermined algorithm may include a parameter for correcting a calculation result based on an image.

[0025] (Operation example of the robot control device 10) The robot control device 10 operates the robot 40 to act on a work target such as an object 50 present in the operating space, or to avoid the object 50. The robot control device 10 operates the robot 40 to act on or avoid the object 50, based on an image of the object 50 captured by the spatial information acquisition unit 20.

[0026] <Calibration> The control unit 11 of the robot control device 10 can acquire the state of the robot 40 based on the position and posture of the mark 46 captured in the image captured by the spatial information acquisition unit 20, and can acquire the positional relationship between the robot 40 and the object 50. On the other hand, the control unit 11 acquires the state of the robot 40 based on a sensor of the robot 40, such as an encoder installed on the arm 42, for example. The state based on the sensor of the robot 40 represents the position and posture of the robot 40 with higher accuracy than the state based on the image captured by the spatial information acquisition unit 20. Therefore, the control unit 11 can control the robot 40 with high accuracy in the operating space by matching the state of the robot 40 based on the image captured by the spatial information acquisition unit 20 with the state of the robot 40 based on the sensor of the robot 40. The task of matching the state of the robot 40 based on the image captured by the spatial information acquisition unit 20 with the state of the robot 40 based on the sensor of the robot 40 is also called calibration. Specifically, the control unit 11 performs calibration so as to match the (X, Y, Z) coordinate system of the spatial information acquisition unit 20 with the (X_RB, Y_RB, Z_RB) coordinate system of the robot 40. The control unit 11 may estimate a relative positional relationship between the coordinate system of the spatial information acquisition unit 20 and the coordinate system of the robot 40, and align the coordinate system of the spatial information acquisition unit 20 with the coordinate system of the robot 40 based on the estimated relative positional relationship.

[0027] The control unit 11 may perform calibration with at least a part of the FOV of the spatial information acquisition unit 20 as a calibration range. In this embodiment, the control unit 11 performs calibration in a first calibration range 60 and a second calibration range 62 shown in FIG. 2 and FIG. 3. The first calibration range 60 includes the second calibration range 62. In other words, the second calibration range 62 corresponds to a part of the range of the first calibration range 60. The first calibration range 60 is shown as an area surrounded by a two-dot chain line in FIG. 2 and FIG. 3. The second calibration range 62 is shown as an area surrounded by a dashed line in FIG. 2 and FIG. 3. The first calibration range 60 and the second calibration range 62 are simply referred to as calibration ranges when they are not distinguished from each other. The calibration range corresponds to a range in which the calibration of the robot 40 is performed. The calibration range may include the working area of ​​the robot 40. The calibration range may be a range in which the working area of ​​the robot 40 and the FOV overlap.

[0028] Furthermore, the control unit 11 sets a point for performing calibration within the calibration range. The point for performing calibration is also referred to as a calibration position. The calibration position set within the first calibration range 60 is also referred to as a first calibration position. The calibration position set within the second calibration range 62 is also referred to as a second calibration position. Note that the second calibration position is set at a position different from the first calibration position.

[0029] The control unit 11 moves the mark 46 of the robot 40 to a calibration position and causes the spatial information acquisition unit 20 to photograph the mark 46. The control unit 11 calculates the position and orientation of the mark 46 based on the image of the mark 46. The control unit 11 corrects the position and orientation of the mark 46 based on the image so that the position and orientation of the mark 46 calculated based on the image coincide with the position and orientation of the mark 46 determined based on the detection result of the sensor of the robot 40. The correction of the position and orientation of the mark 46 based on the image corresponds to calibration. The position and orientation of the mark 46 are also referred to as the tip position and orientation. Calibration corresponds to the correction of the tip position and orientation. The calibration position corresponds to the position at which the tip position and orientation is corrected.

[0030] Specifically, the control unit 11 may perform the calibration as described below. The control unit 11 generates control information for the robot 40 for moving the mark 46 of the robot 40 to the calibration position. The control unit 11 operates the robot 40 based on the control information, and moves the mark 46 of the robot 40 to the calibration position. The control unit 11 acquires an image of the mark 46 from the spatial information acquisition unit 20. The control unit 11 calculates the position and orientation of the mark 46 based on the image. The position and orientation of the mark 46 calculated based on the image are also referred to as the tip position and orientation based on the image. The control unit 11 calculates the position and orientation of the mark 46 determined based on the detection result of the sensor of the robot 40. The position and orientation of the mark 46 calculated based on the detection result of the sensor are also referred to as the tip position and orientation based on the sensor. The control unit 11 compares the tip position and orientation based on the image with the tip position and orientation based on the sensor. The control unit 11 corrects the tip position and orientation based on the image so that the tip position and orientation based on the image coincides with the tip position and orientation based on the sensor. The control unit 11 may correct an algorithm for calculating the tip position and orientation based on an image. The control unit 11 may correct parameters included in the algorithm, or may correct a formula, a table, or a program. When multiple calibration positions are set, the control unit 11 moves the robot 40 to each calibration position, obtains an image of the mark 46 at each calibration position, and corrects the tip position and orientation based on the image.

[0031] In the above example, calibration is performed for the position and orientation of the mark 46, but the location where calibration is performed is not limited to the position and orientation of the mark 46. That is, the control unit 11 may store in advance the positional relationship between the position of the mark 46 and a calibration target part, which is a part of the robot 40 to be calibrated, and calculate the position and orientation of the calibration target part from the position and orientation of the mark 46 based on the image. Then, calibration can be performed by comparing the position and orientation of the calibration target part based on the detection result of the sensor of the robot 40 with the position and orientation of the mark 46. Therefore, calibration can be performed at a position and orientation other than the mark 46. In the above example, the calibration target part is the tip position and orientation of the robot 40, but the calibration target part is not limited to the tip position and orientation of the robot 40 as long as the position and orientation can be calculated.

[0032] <<Calibration item>> The control unit 11 sets a calibration range in advance before performing calibration. The control unit 11 also sets calibration positions included in the calibration range. The control unit 11 sets at least one first calibration position in the first calibration range 60. The control unit 11 sets at least one second calibration position in the second calibration range 62. The control unit 11 sets the calibration positions such that the density of the second calibration positions is higher than the density of the first calibration positions. In other words, the control unit 11 sets the second calibration positions at a higher density than the first calibration positions. Conversely, the control unit 11 sets the first calibration positions at a lower density than the second calibration positions. The number of the second calibration positions may be equal to or greater than the number of the first calibration positions.

[0033] The control unit 11 generates control information for the robot 40 so as to move the robot 40 to the calibration position. The control unit 11 generates information for specifying the tip position and orientation when the robot 40 is moved and the recognition result of the mark 46 of the robot 40 as a calibration item. A calibration item for specifying the tip position and orientation when the robot 40 is moved to at least one first calibration position included in a first calibration range 60 set in the operating space of the robot 40 and the recognition result of the mark 46 of the robot 40 is also referred to as a first calibration item. A calibration item for specifying the tip position and orientation when the robot 40 is moved to at least one second calibration position included in a second calibration range 62 set in the operating space of the robot 40 and the recognition result of the mark 46 of the robot 40 is also referred to as a second calibration item. Note that the calibration item is, for example, information on coordinates. Specifically, the calibration item is, for example, coordinate information indicating the tip position and orientation based on the detection result of the sensor of the robot 40 when the robot 40 is moved to the calibration position, or coordinate information indicating the tip position and orientation based on the recognition result of the mark 46 recognized by the space information acquisition unit 20.

[0034] The control unit 11 may generate the calibration items as described below.

[0035] The control unit 11 acquires, for example, information on the real field of view size of the spatial information acquisition unit 20 or information on the FOV from the spatial information acquisition unit 20. The control unit 11 sets a calibration range based on the real field of view size or FOV of the spatial information acquisition unit 20 and the working area of ​​the robot 40. The control unit 11 may set the calibration range based on the position of the object 50 in the operating space of the robot 40. The control unit 11 may set the calibration range based on the depth information or point cloud information of the object 50 detected by the spatial information acquisition unit 20. In addition, the control unit 11 may set the first calibration range 60 based on the real field of view size and FOV, for example. In addition, the control unit 11 may set the second calibration range 62 based on the working area, for example. Note that the calibration range may have a plurality of second calibration ranges 62. For example, when the control unit 11 causes the robot 40 to perform a pick-and-place operation, the control unit 11 may set the second calibration range 62 as the working area in each of a place where the object is grasped and a place where the object is placed. 2 and 3, the shape of the first calibration range 60 is set to a quadrangular pyramid shape. The shape of the second calibration range 62 is set to a rectangular parallelepiped shape. The shapes of the calibration ranges are not limited to these and may be set to various other shapes.

[0036] The control unit 11 matches the tip position and orientation of the robot 40 based on the sensor with the tip position and orientation based on the image of the spatial information acquisition unit 20. Specifically, the control unit 11 moves the robot 40 to a first position. The control unit 11 generates control information for operating the robot 40 so that the mark 46 of the robot 40 is at a predetermined position and orientation, and controls the robot 40 based on the control information to move the robot 40 to the first position. The first position may be a predetermined position included in the FOV of the spatial information acquisition unit 20. The first position may be, for example, the center position of the FOV of the spatial information acquisition unit 20. The control unit 11 acquires an image of the mark 46 when the robot 40 moves to the first position, and calculates the position and orientation of the mark 46 as the tip position and orientation based on the image. In addition, the control unit 11 calculates the tip position and orientation based on the sensor. Based on a comparison between the tip position and orientation based on the image and the tip position and orientation based on the sensor, the control unit 11 corrects the control information of the robot 40 so that the position of the robot 40 becomes the first position based on the detection result of the sensor in the image. The control unit 11 moves the robot 40 based on the corrected control information, thereby updating the state of the robot 40 so that the position of the robot 40 in the coordinate system of the robot 40 coincides with the position of the robot 40 in the coordinate system of the spatial information acquisition unit 20. In other words, it can be said that the control unit 11 updates the state of the robot 40 so that the position of the robot 40 becomes the first position in the image.

[0037] The control unit 11 may generate at least one position that is a candidate for the calibration position within the calibration range. The position that is a candidate for the at least one calibration position is also referred to as at least one second position. The control unit 11 may set the first position as one of the at least one second position. The control unit 11 may also set the at least one second position to a position different from the first position. The control unit 11 may select the first calibration position and the second calibration position from the first position or at least one second position set in each of the first calibration range 60 and the second calibration range 62. The at least one second position may include a point arranged at a corner or a side of the calibration range. Alternatively, the at least one second position may include a point arranged inside the calibration range. The control unit 11 estimates the state of the robot 40 when the robot 40 moves to the second position by simulating the operation of the robot 40. That is, the control unit 11 calculates the state of the robot 40 when it is assumed that the robot 40 moves to the second position. In other words, the state of the robot 40 when it is assumed that the robot 40 moves to the first calibration position or the second calibration position is estimated. As a result, the control unit 11 can determine whether the robot 40 can move to a second position such as the first calibration position or the second calibration position.

[0038] As described below, the first calibration is a calibration performed at the first calibration position, and is a calibration performed simply at positions set at a low density (positions set at wide intervals). Therefore, by performing the first calibration, the control unit 11 can correct the robot 40 with fewer operations. The second calibration is a calibration performed at the second calibration position, and is a calibration performed in more detail at positions set at a high density (positions set at narrow intervals). Therefore, by performing the second calibration, the control unit 11 can correct the operation of the robot 40 with high accuracy.

[0039] When the state of the robot 40 when it is assumed to move to the second position is a state where the robot 40 is not in contact with the object 50, is within the range of motion of the joint, and is not a singular point, the control unit 11 registers the second position as a calibration position. When registering the second position as a calibration position, the control unit 11 generates, as a plurality of calibration items, information that specifies each of the tip position and orientation based on the detection result of the sensor of the robot 40 when the robot 40 is moved to the second position and the tip position and orientation based on the recognition result of the mark 46 of the robot 40. When not registering the second position as a calibration position, the control unit 11 may generate a new second position that is a different position and determine whether the new second position can be registered as a calibration position. When the numerical value representing the angle of the joint of the robot 40 is within the range of motion, the control unit 11 may determine that the state of the robot 40 is not in the joint restricted state. When the numerical value representing the angle of the joint of the robot 40 is outside the range of motion, the control unit 11 may determine that the state of the robot 40 is in the joint restricted state.

[0040] A singularity corresponds to a posture where the robot 40 becomes uncontrollable structurally. If a singularity is included in the trajectory for moving the robot 40, the robot 40 moves (runs out of control) at high speed near the singularity and stops at the singularity. There are three types of singularities of the robot 40: (1) to (3) below. (1) A point outside the working area when controlling the robot 40 to be close to the outer limit of the working area. (The working area is an area corresponding to the operating space of the robot 40.) (2) Points when controlling the robot 40 directly above and directly below the robot base, even within the working area. (3) A point where the angle of the joint immediately before the tip joint of the arm 42 of the robot 40 becomes zero or 180 degrees (wrist alignment singular point).

[0041] The control unit 11 may determine that the state of the robot 40 is a singular point state when the numerical value representing the state of the robot 40 matches a numerical value representing a singular point state. The control unit 11 may determine that the state of the robot 40 is a singular point state when the difference between the numerical value representing the state of the robot 40 and the numerical value representing a singular point state is less than a predetermined value. The numerical value representing the state of the robot 40 may include, for example, the angle of the joint of the arm 42, or the torque of the motor that drives the robot 40.

[0042] As described above, the control unit 11 sets a calibration range and a calibration position within the calibration range. In addition, the control unit 11 can generate a calibration item as information specifying the tip position and orientation of the robot 40 when the robot 40 is moved to the calibration position.

[0043] <<Perform calibration>> The control unit 11 performs calibration so that the calibration item of the tip position and orientation related to the recognition result of the mark 46 matches the calibration item of the tip position and orientation related to the detection result of the sensor of the robot 40. Specifically, the control unit 11 moves the robot 40 to the calibration position. The control unit 11 acquires the recognition result of the mark 46 of the robot 40 when the robot 40 moves to the calibration position by the spatial information acquisition unit 20. The control unit 11 calculates the relative positional relationship of the calibration item of the tip position and orientation acquired as the recognition result of the mark 46 with respect to the calibration item of the tip position and orientation based on the sensor of the robot 40. The relative positional relationship corresponds to the difference in coordinates and the difference in angle between the two calibration items. The control unit 11 corrects the coordinate system of the spatial information acquisition unit 20 to match the coordinate system of the robot 40 so that the coordinate error and the angle error corresponding to the relative positional relationship with respect to the two calibration items are zero or close to zero (i.e., the error is less than a predetermined value). In this way, the control unit 11 can calculate the relative positional relationship by matching the recognition result of the mark 46 when the robot 40 moves to the calibration position with the tip position and orientation specified by the sensor of the robot 40. Note that the control unit 11 may match the tip position and orientation specified by the sensor of the robot 40 with the tip position and orientation recognized by the recognition result of the mark 46.

[0044] The control unit 11 can set a calibration position by generating a calibration item. Conversely, the calibration position corresponds to a position to which the robot 40 is moved in order to generate the calibration item. The control unit 11 can perform calibration by moving the robot 40 to the calibration position by applying the calibration position to the control of the robot 40. Calibration by applying a first calibration item is also referred to as a first calibration. Calibration by applying a second calibration item is also referred to as a second calibration. The control unit 11 can perform the first calibration by moving the robot 40 to the first calibration position by applying the first calibration position to the control of the robot 40. The control unit 11 can perform the second calibration by moving the robot 40 to the second calibration position by applying the second calibration position to the control of the robot 40.

[0045] The control unit 11 executes a first calibration and a second calibration. By executing the first calibration, the control unit 11 can correct the robot 40 with fewer operations. On the other hand, by executing the second calibration, the control unit 11 can correct the operation of the robot 40 with high accuracy. The first calibration is a calibration executed at the first calibration position, and is a calibration executed simply at positions set with low density (positions set at wide intervals). Therefore, the control unit 11 can correct the robot 40 with fewer operations by executing the first calibration. The second calibration is a calibration executed at the second calibration position, and is a calibration executed in more detail at positions set with high density (positions set at narrow intervals). Therefore, the control unit 11 can correct the operation of the robot 40 with high accuracy by executing the second calibration.

[0046] The first calibration may be a calibration that is required regardless of the work to be performed by the robot 40. The second calibration may be a calibration that is required depending on the work to be performed by the robot 40. In other words, the first calibration range 60 may be a constant range regardless of the work to be performed by the robot 40, and the second calibration range 62 may be a range that is set to a different size depending on the work to be performed by the robot 40.

[0047] The control unit 11 performs calibration before starting work by the robot 40. The control unit 11 may perform only the first calibration so that the robot 40 can be corrected with an accuracy that allows the robot 40 to perform work.

[0048] The control unit 11 may execute the second calibration after executing the first calibration. The control unit 11 may start the work by the robot 40 after executing the first calibration, and execute the second calibration while the robot 40 is working. In this case, the control unit 11 generates the first calibration item in advance before the robot 40 starts working. The control unit 11 may also generate the second calibration item in advance before the robot 40 starts working, or while the robot 40 is working.

[0049] Specifically, the control unit 11 starts the work by the robot 40 in the coordinate system corrected by executing the first calibration. The control unit 11 controls the position and posture of the robot 40 to the position and posture determined for the work. When the control unit 11 controls the robot 40 to the position and posture determined for the work, the control unit 11 acquires the tip position and posture based on the image by recognizing the mark 46 with the spatial information acquisition unit 20. In addition, the control unit 11 acquires the tip position and posture based on the sensor based on the detection result of the sensor of the robot 40. The control unit 11 can execute the calibration based on the tip position and posture based on the image and the tip position and posture based on the sensor.

[0050] The control unit 11 may register, as a calibration position, the tip position and posture of the robot 40 when the robot 40 is controlled to a position and posture determined in a task. The control unit 11 generates, as a second calibration item, a calibration item that specifies the calibration position and the recognition result of the mark 46 at the calibration position.

[0051] The control unit 11 may generate the second calibration items so that the density of the second calibration positions is higher than the density of the first calibration positions. Conversely, the control unit 11 may generate the first calibration items based on the density of the second calibration positions determined based on the work content of the robot 40. Specifically, the control unit 11 may set the first calibration positions so that the density of the first calibration positions applied to the first calibration performed before the robot 40 starts working is lower than the density of the second calibration positions. The calibration at the first calibration positions is a calibration that is performed simply. The calibration at the second calibration positions is a calibration that is performed in more detail. By setting the first calibration positions so that the density of the first calibration positions is lower than the density of the second calibration positions, the load of the first calibration, which is a calibration that is performed simply, can be reduced.

[0052] Furthermore, the control unit 11 may set the second calibration positions in a space in the operating space of the robot 40 where the robot 40 performs the work, at a density determined based on the accuracy of the work of the robot 40. In this way, the second calibration positions can be set at an appropriate density.

[0053] (Example of robot control procedure) The control unit 11 of the robot control device 10 may execute a robot control method including the procedures of the flowcharts exemplified in Figures 4 and 5. The robot control method may be realized as a robot control program executed by a processor constituting the control unit 11. The robot control program may be stored in a non-transitory computer-readable medium.

[0054] First, the control unit 11 executes the procedure of the flowchart illustrated in Fig. 4 as the first calibration. The control unit 11 moves the robot 40 to the first calibration position (step S1). The control unit 11 acquires the state of the robot 40 at the first calibration position to which the robot 40 has moved based on the detection result of the sensor (step S2). The control unit 11 acquires the recognition result of the mark 46 at the first calibration position to which the robot 40 has moved (step S3).

[0055] The control unit 11 calculates the relative positional relationship and error when the robot 40 is moved to the first calibration position (step S4). Specifically, the control unit 11 acquires the tip position and orientation based on the image based on the recognition result of the mark 46, and acquires the tip position and orientation based on the sensor based on the detection result of the sensor. The control unit 11 calculates the relative positional relationship and error between the tip position and orientation based on the image and the tip position and orientation based on the sensor. The control unit 11 may calculate an overall calibration error as the error.

[0056] The control unit 11 judges whether the calculation of the relative positional relationship and the error is completed for all the calibration items included in the first calibration item (step S5). If the calculation of the relative positional relationship and the error is not completed for all the calibration items (step S5: NO), the control unit 11 returns to the procedure of step S1 and repeats the operation for the calibration items that are not completed. If the calculation of the relative positional relationship and the error is completed for all the calibration items (step S5: YES), the control unit 11 corrects the coordinate system based on the relative positional relationship and the error calculated for each calibration item (step S6). Specifically, the control unit 11 corrects the coordinate system of the spatial information acquisition unit 20 to match it with the coordinate system of the robot 40. After executing the procedure of step S6, the control unit 11 ends the execution of the procedure of the flowchart in FIG. 4.

[0057] Next, the control unit 11 executes, as the second calibration, the procedure of the flowchart exemplified in Fig. 5. In the example of the procedure in Fig. 5, it is assumed that the second calibration item has not been set in advance.

[0058] The control unit 11 starts the work of the robot 40 (step S11). The control unit 11 acquires the state of the robot 40 at a predetermined position to which the robot 40 has moved for work (step S12). The control unit 11 acquires the recognition result of the mark 46 at the predetermined position to which the robot 40 has moved for work (step S13). In steps S12 and S13, the predetermined position corresponds to the tip position and orientation of the robot 40. The control unit 11 may set the predetermined position to a part of the position to which the robot 40 moves for work. The control unit 11 may set the predetermined position to a position to which the robot 40 temporarily stops for work.

[0059] The control unit 11 calculates the relative positional relationship and error of the tip position and orientation of the robot 40 at a predetermined position (step S14). The control unit 11 determines whether the error is greater than a threshold value (step S15). Specifically, the control unit 11 may calculate one evaluation value that combines the errors of the position and orientation, and compare the evaluation value with the threshold value. In this case, the evaluation value is calculated to be greater as the error is greater. The control unit 11 may also compare the magnitude of the position error and the magnitude of the orientation error with the threshold value. The threshold value compared with the magnitude of the position error is also referred to as a first threshold value. The threshold value compared with the magnitude of the orientation error is also referred to as a second threshold value. The control unit 11 may determine that the error is greater than the threshold value when at least one of the following is true: the magnitude of the position error is greater than the first threshold value or the magnitude of the orientation error is greater than the second threshold value. The control unit 11 may determine that the error is greater than the threshold value when the magnitude of the position error is greater than the first threshold value and the magnitude of the orientation error is greater than the second threshold value.

[0060] If the error is not greater than the threshold (step S15: NO), that is, if the error is equal to or less than the threshold, the control unit 11 registers the predetermined position as the second calibration position (step S16). Specifically, the control unit 11 may generate a second calibration item with the predetermined position as the second calibration position, and store it in the storage unit 12. On the other hand, if the error is greater than the threshold (step S15: YES), the control unit 11 returns to the procedure of step S12 without registering the predetermined position as the second calibration position, and repeats the operations from step S12 to S16 at the next predetermined position.

[0061] The control unit 11 determines whether the work of the robot 40 is complete (step S17). If the work of the robot 40 is not complete (step S17: NO), the control unit 11 returns to the procedure of step S12 and repeats the operations of steps S12 to S16 at the next predetermined position. If the work of the robot 40 is complete (step S17: YES), the control unit 11 ends the execution of the procedure of the flowchart in FIG. 5.

[0062] (Small summary) As described above, according to the robot control device 10 and the robot control method of this embodiment, the first calibration and the second calibration are executed separately. The density of the first calibration positions set in the first calibration is set lower than the density of the second calibration positions set in the second calibration, so that the load of the first calibration can be reduced. As a result, the first calibration can be shortened. Furthermore, after the first calibration is executed, the robot 40 can start working. As a result, the calibration work required to make the robot 40 perform the work can be shortened.

[0063] Moreover, according to the robot control device 10 and the robot control method according to the present embodiment, the second calibration can be performed while the robot 40 is performing a task. Moreover, the second calibration can be performed within the range in which the robot 40 performs a task. In this way, the calibration can be performed efficiently. Moreover, the calibration can be performed with high accuracy.

[0064] As described above, the relationship between the coordinate system of the spatial information acquisition unit 20 and the coordinate system of the robot 40 is specified by calibration. In this embodiment, the coordinate system of the spatial information acquisition unit 20 and the coordinate system of the robot 40 coincide with each other. Here, the relationship between the coordinate systems may change due to various causes. The relationship between the coordinate systems may change when an abnormality occurs in the robot 40 or the robot control system 1. The relationship between the coordinate systems may change when the robot 40 is stopped or when the robot 40 is started up.

[0065] In this case, the control unit 11 may determine whether the relationship of the coordinate systems identified by the calibration has changed when the robot 40 that has been calibrated at least once is started normally or when the robot 40 is started to recover from the occurrence of an abnormality. If the relationship of the coordinate systems has not changed, the control unit 11 does not need to correct the relationship of the coordinate systems or perform calibration. On the other hand, if the relationship of the coordinate systems has changed when the robot 40 is started, the control unit 11 determines whether the relationship of the coordinate systems can be corrected. If the relationship of the coordinate systems can be corrected, the control unit 11 corrects the relationship of the coordinate systems and does not perform calibration. If the relationship of the coordinate systems cannot be corrected, the control unit 11 may again specify the relationship of the coordinate systems by performing the first calibration and the second calibration.

[0066] In other words, the necessity of the first calibration and the second calibration may be determined when the robot 40 stops or when the robot 40 is started. In addition, when the robot 40 stops, it is not limited to when it stops abnormally, but may be when it completes a designated task. In addition, when the robot 40 is started, it is not limited to when it starts after it stops abnormally, but may be when it starts a designated task.

[0067] Furthermore, the second calibration may be performed each time the work content is changed. Alternatively, the second calibration may be performed each time the work content is changed to determine whether the relationship of the coordinate systems specified by the calibration has changed. If the relationship of the coordinate systems has not changed, the control unit 11 does not need to perform correction or calibration of the relationship of the coordinate systems. On the other hand, if the relationship of the coordinate systems has changed when the robot 40 is started, the control unit 11 determines whether the relationship of the coordinate systems can be corrected. If the relationship of the coordinate systems can be corrected, the control unit 11 corrects the relationship of the coordinate systems and does not perform calibration. If the relationship of the coordinate systems cannot be corrected, the control unit 11 may perform the second calibration again to specify the relationship of the coordinate systems anew. Note that only the second calibration may be performed as the calibration when the work content is changed.

[0068] When determining whether to re-execute calibration, the control unit 11 determines whether the relationship between the coordinate system of the robot 40 and the coordinate system of the spatial information acquisition unit 20 has changed, as described below, and determines whether re-calibration is necessary.

[0069] The control unit 11 moves the robot 40 to a measurement position. The control unit 11 sets the measurement position to a point included in the calibration range. The measurement position may include, for example, a part of the calibration position. The measurement position may include, for example, a corner point of the calibration range. The measurement position may include, for example, the first position or the second position described above, or may include a position different from the first position and the second position. The measurement position may include a calibration position used in the previous calibration, or may include a position different from the previous calibration position. The control unit 11 may set, for example, a point inside the calibration range as the measurement position. The control unit 11 may set various points included in the calibration range as the measurement position, without being limited to these points.

[0070] The control unit 11 acquires the recognition result of the mark 46 when the robot 40 is moved to the measurement position. The control unit 11 calculates the position of the robot 40 as a measurement result based on the recognition result of the mark 46. The control unit 11 calculates the difference between the initial value of the measurement position and the measurement result. Since the control unit 11 moves the robot 40 to the measurement position based on the detection result of the sensor, it may calculate the difference between the set measurement position itself and the measurement result without calculating the position of the robot 40 based on the sensor. The control unit 11 may acquire the detection result of the sensor when the robot 40 is moved to the measurement position, calculate the position of the robot 40 based on the sensor as the measurement position from this detection result, and calculate the difference from the measurement result using the calculated measurement position as the initial value of the measurement position.

[0071] The control unit 11 determines whether to correct the relationship of the coordinate system based on the difference between the initial value of the measurement position and the measurement result. For example, the control unit 11 determines to correct the relationship of the coordinate system when the difference between the initial value of the measurement position and the measurement result is greater than a predetermined threshold. When the measurement results are acquired at a plurality of measurement positions, the control unit 11 determines to correct the relationship of the coordinate system when the difference between the initial value of at least one measurement position and its measurement result is greater than a predetermined threshold. The control unit 11 determines that correction of the relationship of the coordinate system and recalibration are not necessary when the difference between the initial value of the measurement position and the measurement result is equal to or less than a predetermined threshold. The control unit 11 may set the predetermined threshold as appropriate. The control unit 11 may set the predetermined threshold based on, for example, the specifications of the position accuracy during the operation of the robot 40.

[0072] When correcting the relationship between the coordinate systems, the control unit 11 may correct the coordinate system of the spatial information acquisition unit 20 so as to align the coordinate system of the robot 40 with the coordinate system of the spatial information acquisition unit 20. The control unit 11 may correct the coordinate system of the robot 40 so as to align the coordinate system of the robot 40 with the coordinate system of the spatial information acquisition unit 20.

[0073] Specifically, the control unit 11 may correct the coordinate system of the spatial information acquisition unit 20 or the coordinate system of the robot 40 by rotating or translating it. The control unit 11 may correct the coordinate system of the spatial information acquisition unit 20 or the coordinate system of the robot 40 by enlarging or shrinking it. The control unit 11 may correct distortion of the coordinate system of the spatial information acquisition unit 20 or the coordinate system of the robot 40. The control unit 11 may calculate a correction value for the measurement position based on the correction of the coordinate system.

[0074] The control unit 11 may acquire the tip position and orientation of the robot 40 at one measurement position, and based on information such as the rotation angle indicating the orientation of the mark 46, correct the coordinate system not only in the translation direction but also in the rotation direction.

[0075] Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can make various modifications or alterations based on the present disclosure. Therefore, it should be noted that these modifications or alterations are included in the scope of the present disclosure. For example, the functions, etc. included in each component, etc. can be rearranged so as not to cause logical inconsistencies, and multiple components, etc. can be combined into one or divided.

[0076] All of the features described in this disclosure and / or all steps of any method or process disclosed herein may be combined in any combination except those combinations in which the features are mutually exclusive. Furthermore, each feature described in this disclosure may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly denied. Thus, unless expressly denied, each disclosed feature is merely one example of a generic series of the same or equivalent features.

[0077] Furthermore, the embodiments of the present disclosure are not limited to any of the specific configurations of the embodiments described above, but rather extend to any novel feature or combination of features described herein, or any novel method or process step or combination of features described herein. [Explanation of symbols]

[0078] 1. Robot Control System 10 Robot control device (11: control unit, 12: memory unit) 20 Spatial information acquisition section 40 Robot (42: Arm, 44: End effector, 46: Mark) 50 objects 60 First Calibration Range 62 Second Calibration Range

Claims

1. A control unit for controlling the robot is provided, The control unit is performing a first calibration of the robot at at least one first calibration position included in a first calibration range set in a motion space of the robot; performing a second calibration of the robot at at least one second calibration position that is included in a second calibration range that is a part of the first calibration range and that is set at a higher density than the at least one first calibration position; Robot control device.

2. The robot control device according to claim 1 , wherein the second calibration positions are set in a space in which the robot performs a task at a density determined based on the accuracy of the task performed by the robot.

3. The robot control device according to claim 1 , wherein the control unit executes the first calibration and the second calibration based on a captured image of a motion space of the robot.

4. A robot control system comprising: the robot control device according to any one of claims 1 to 3; and the robot.

5. a control unit that controls a robot performs a first calibration of the robot at at least one first calibration position included in a first calibration range set in a motion space of the robot; the control unit executes a second calibration of the robot at at least one second calibration position that is included in a second calibration range that is a part of the first calibration range and that is set at a higher density than the at least one first calibration position; A robot control method comprising:

Citation Information

Patent Citations

  • High-precision hand-eye calibration method and system for SCARA manipulator Eye-in-Hand

    CN110148187A

  • Robot positioning error correction method

    JP1989002104A

  • Calibration method for manipulator

    JP1993277973A

  • Robot control device

    JP2009142903A

  • Robot vision system and automatic calibration method

    JP2010172986A