Robot control device

Abstract

A robot control device for controlling a robot includes: a storage unit that stores a first imaging position for stereoscopically measuring an object provided in a work space by a vision sensor mounted on the robot, and a second imaging position for stereoscopically measuring the object provided in the work space by a vision sensor mounted on the robot; the second shooting position is used for carrying out stereoscopic measurement on the object through the visual sensor, and the parallax of the second shooting position is larger than that of stereoscopic measurement at the first shooting position; a first measurement unit that performs three-dimensional measurement of the object at a first imaging position and acquires a first amount of shift of the robot with respect to the work space; a correction unit that corrects the position of the robot at the second imaging position during stereoscopic measurement on the basis of the first shift amount; a second measurement unit that performs three-dimensional measurement of the object at a second imaging position and acquires a second amount of shift of the robot with respect to the work space; and an operation control unit that corrects the operation of the robot on the basis of the second offset amount.

Description

Technical Field

[0001] The present disclosure relates to a robot control device. Background Art

[0002] In recent years, for example, an automation system has been proposed in which a robot is placed on a trolley or AGV (Automated Guided Vehicle) and moved, and various operations are automated by the robot being arranged on the working space side of industrial machinery such as machine tools. In such an automation system, when the robot performs various operations such as loading / unloading the processing object on the industrial machinery, the stop position of the trolley or AGV on which the robot is mounted changes. Therefore, the robot needs to measure the deviation of the stop position of the trolley or AGV relative to the industrial machinery and correct the movement of the robot so that it can correctly operate in the working space.

[0003] Patent document 1 records the following: "A robot system comprising: a robot; a robot conveying device for carrying the robot and moving the robot to a prescribed working space; at least two target marks provided in the working space; a target mark position acquisition unit for obtaining a three-dimensional position of the at least two target marks by stereoscopically measuring the at least two target marks using a visual sensor provided on the robot; an offset acquisition unit for obtaining an offset between the robot and the working space relative to a desired relative position based on the obtained three-dimensional position; and a robot control unit for using the obtained offset to cause the robot to move at a value corrected according to a prescribed movement amount." (Claim 1).

[0004] Patent document 2 relates to a robot system equipped with a two-dimensional camera at the front end of the robot's arm, and records the following content: "First, the camera photographs a first mark. Then, after the robot moves to an extent where the first mark does not exceed the camera's field of view, the camera photographs the first mark again. The camera photographs the same mark from two different viewpoints. Through this stereo measurement, the coordinate value (x, y, z) of the three-dimensional position of the first mark can be obtained. Then, the same measurement is performed on the second mark and the third mark. The three-dimensional positions of the first mark, the second mark, and the third mark can be used to calculate a three-dimensional plane passing through three points. As a result, the three-dimensional positional relationship between the robot and the machine tool can be obtained." (Paragraph 0084).

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: International Publication No. WO2022 / 075303

[0008] Patent Document 2: Japanese Patent Application Publication No. 2018-058142 Summary of the invention

[0009] Problems to be solved by the invention

[0010] In measuring the position of a robot relative to a workspace, it is widely practiced to install a camera on the fingertips of the robot and perform stereo measurement of a target mark set in the workspace, thereby measuring the positional relationship between the robot and the workspace. Regarding stereo measurement, it is known that if the parallax in the captured image is not enlarged, the accuracy will decrease. If the target mark is to be captured at the end of the image in order to obtain a longer parallax, the target mark will deviate from the field of view due to a slight positional deviation of the robot. A robot control device that can reliably perform high-precision stereo measurement is desired.

[0011] Means for solving problems

[0012] One method of the present disclosure is a robot control device for controlling a robot, comprising: a storage unit, which stores a first shooting position and a second shooting position, the first shooting position being used to perform stereo measurement of an object set in a working space by a visual sensor mounted on the robot, and the second shooting position being used to perform stereo measurement of the object by the visual sensor and having a parallax greater than that of the stereo measurement at the first shooting position; a first measuring unit, which performs stereo measurement of the object at the first shooting position to obtain a first offset of the robot relative to the working space; a correction unit, which corrects the position of the robot during stereo measurement at the second shooting position according to the first offset; a second measuring unit, which performs stereo measurement of the object at the second shooting position to obtain a second offset of the robot relative to the working space; and a motion control unit, which corrects the motion of the robot according to the second offset.

[0013] Another embodiment of the present disclosure is a robot control device, comprising: a storage unit, which stores a first shooting position and a second shooting position, the first shooting position being used to perform stereo measurement of an object set in a working space by a visual sensor mounted on the robot, and the second shooting position being used to perform stereo measurement of the object by the visual sensor and having a parallax greater than that of the stereo measurement at the first shooting position; a measuring unit, which sequentially performs stereo measurement of the object at the first shooting position and stereo measurement of the object at the second shooting position; a coordinate system data output unit, which outputs coordinate system data representing a specific coordinate system based on a first offset between the robot and the working space obtained by the stereo measurement at the first shooting position; and a motion control unit, which corrects the motion of the robot based on a second offset relative to the working space obtained by the stereo measurement at the second shooting position, the second shooting position being represented by the specific coordinate system. Another other embodiment of the present disclosure is a robot control device, comprising: a storage unit, which stores a first shooting position and a second shooting position, the first shooting position being used to perform stereo measurement of an object set in a working space by a visual sensor mounted on the robot, and the second shooting position being used to perform stereo measurement of the object by the visual sensor and having a parallax greater than that of the stereo measurement at the first shooting position; a measuring unit, which sequentially performs stereo measurement of the object at the first shooting position and stereo measurement of the object at the second shooting position; a coordinate system data output unit, which outputs coordinate system data representing a specific coordinate system based on the detection position of the object obtained by the stereo measurement at the first shooting position; and a motion control unit, which corrects the motion of the robot based on the offset of the robot relative to the working space obtained by the stereo measurement at the second shooting position, the second shooting position being represented by the specific coordinate system.

[0014] These and other objects, features and advantages of the present invention will become more apparent from the detailed description of typical embodiments of the present invention as shown in the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a diagram showing the equipment configuration of a robot system including a robot control device according to an embodiment.

[0016] Figure 2 This is a diagram for explaining an example of detecting a target mark using a stereo method.

[0017] Figure 3 This is a functional block diagram of a robot control device.

[0018] Figure 4This is a diagram for explaining an example of calculating the offset between the robot and the work space.

[0019] Figure 5 is a flow chart of the stereo measurement process.

[0020] Figure 6 FIG. 1 is a diagram showing an example of a user interface screen for designating a coordinate system of an object to be shifted according to a measured offset amount.

[0021] Figure 7 This is a diagram showing an example of a user interface screen related to user coordinate system setting. DETAILED DESCRIPTION

[0022] Next, the embodiments of the present disclosure are described with reference to the accompanying drawings. In the referenced drawings, the same structural parts or functional parts are marked with the same reference symbols. For easy understanding, the scales of these drawings are appropriately changed. In addition, the method shown in the drawings is an example for implementing the present invention, and the present invention is not limited to the method shown in the drawings.

[0023] Figure 1 1 is a diagram showing the equipment structure of a robot system 100 including a robot control device 30 according to an embodiment. The robot system 100 includes: a machine tool 10, an industrial robot (hereinafter referred to as a robot) 20, a robot control device 30 for controlling the robot 20, a conveying device 81 for conveying the robot 20 and the robot control device 30, and a teaching device 50. A visual sensor 71 is mounted on the front end portion (wrist) 21 of the robot 20. The robot 20 is arranged at a predetermined position in front of the machine tool 10 in a state of being mounted on the conveying device 81, and performs predetermined operations such as loading / unloading of a work object (hereinafter referred to as a workpiece) into the machine tool 10. That is, the robot system 100 is configured as an automation system that automates the loading / unloading of the workpiece into the machine tool 10 by the robot 20.

[0024] exist Figure 1 In the figure, as an example, the robot 20 is a vertical multi-joint robot. As the robot 20, various types of robots such as a parallel linkage robot and a dual-arm robot can also be used according to the work object. The robot 20 has a base 20a and an arm 20b installed on a conveying device 81. The robot 20 can perform the desired work by an end effector (tool) (not shown) installed on the front end 21 of the arm. The end effector is an external device that can be replaced according to the purpose, such as a manipulator, a welding gun, a tool, etc. The conveying device 81 is, for example, an automatic conveying device such as a trolley or AGV (Automated Guided Vehicle; unmanned conveying vehicle), a conveying rail system, etc.

[0025] The robot control device 30 controls the operation of the robot 20 according to the operation program or the instruction from the teaching device 50. In addition, the robot control device 30 may have a configuration as a general computer having a CPU, ROM, RAM, a storage device, an operation unit, a display unit, an input / output interface, a network interface, etc. In the case where the conveying device 81 is an automatic conveying device, the robot control device 30 may also be configured to control the operation of the conveying device 81.

[0026] The teaching device 50 is connected to the robot control device 30 by wire or wirelessly, and is used to teach (program) the robot 20 and input various settings. The teaching device 50 may also be composed of an information processing device such as a teaching operation panel or a tablet terminal. The teaching device 50 may also have a structure as a general computer having a CPU, ROM, RAM, a storage device, an operation unit, a display unit, an input / output interface, a network interface, etc.

[0027] exist Figure 1 In such a robot system 100, when the conveyor 81 moves to the position of the robot system 100, the conveyor 81 may not always stop at the same position, so the position deviation of the robot 20 relative to the machine tool 10 may occur. Therefore, the robot 20 (robot control device 30) needs to be configured to be able to measure the position deviation of the robot 20 relative to the machine tool 10 and correctly perform work on the machine tool 10. Therefore, the visual sensor 71 is mounted on the arm front end 21 of the robot 20, and the robot 20 (robot control device 30) is configured to detect the position deviation of the robot 20 relative to the work space S (machine tool 10) using the visual sensor 71, and correct the position deviation to perform work.

[0028] In the present embodiment, the robot 20 (robot control device 30) is configured to measure the position and posture of the robot 20 relative to the work space S by measuring the three-dimensional position of one or more target marks provided in the work space S. The robot 20 (robot control device 30) uses a stereo method to measure the three-dimensional position of the target mark. Figure 1 An example is shown in which three target marks Ta, Tb, and Tc are provided in the work space S (machine tool 10 ).

[0029] In the present embodiment, the visual sensor 71 is a two-dimensional camera. As the visual sensor 71, other types of cameras capable of performing stereoscopic measurement may also be used. The visual sensor 71 is connected to the robot control device 30. In the present embodiment, the robot control device 30 has functions for controlling the visual sensor 71, performing various image processing functions on the images captured by the visual sensor 71, etc. The visual sensor 71 is calibrated, and calibration data including data indicating the position of the visual sensor 71 with respect to the robot 20 is pre-stored in the storage unit 136 of the robot control device 30 (see Figure 3 ).

[0030] Here, refer to Figure 2 An example of detecting a target marker using the stereo method is described below. Figure 2 In the embodiment, the two shooting positions and postures of the visual sensor 71 when the target mark Ta is stereoscopically measured are set as shooting positions and postures C1 and C2. For ease of explanation, in the shooting positions and postures C1 and C2, the visual sensor 71 is configured in parallel. Figure 2 This indicates a state where the camera coordinate system C is set for the position of the photographing position posture C1. Figure 2 For convenience, the imaging planes M1 and M2 are arranged in front of the visual sensor 71. In the imaging position postures C1 and C2, the visual sensor 71 is separated by a baseline length B, the optical axis O of the visual sensor 71 is arranged in parallel, the imaging plane (image sensor) of the visual sensor 71 is arranged in a plane orthogonal to the optical axis O, and the x-direction and y-direction of the imaging plane are oriented in the same direction.

[0031] If the shooting position posture C1 of the shooting position postures C1 and C2 of the two parts is set as the origin of the camera coordinate system C, the focal length of the visual sensor 71 is set to f, the parallax of the target mark Ta appearing in the two captured images is set to D, and the pixel spacing of the visual sensor 71 is set to 1 mm, then the distance z (the distance in the depth direction) from the origin of the camera coordinate system C to the target mark Ta is calculated by the following formula using the stereo method.

[0032] z = B•f / D … (1)

[0033] The baseline length B and the focal length f are determined by the shooting position postures C1, C2 and the specifications of the visual sensor 71. Therefore, the robot control device 30 can detect the distance z of the target mark Ta by simply calculating the parallax D of the target mark appearing in the two captured images based on the detection processing of the target mark Ta (including matching processing).

[0034] It is known that in stereo measurement, the target mark is photographed from two positions, but at this time, if the parallax D between the two shots is not obtained for a long time, the measurement accuracy of the line of sight direction will decrease. However, if the target mark is photographed at the end of the image in order to obtain the parallax D for a long time, the target mark may deviate from the field of view (photographed image) due to a slight positional deviation of the conveying device 81 (that is, a slight positional deviation of the robot 20 relative to the work space S). In view of this, the robot control device 30 performs steering measurement in the following process.

[0035] (Process 1) A first stereo measurement is performed at a first imaging position where a target mark is captured near the center of the image, and the position of the robot 20 relative to the work space S (machine tool 10 ) is roughly corrected based on the measurement result.

[0036] (Process 2) Next, a second stereo measurement is performed at a second imaging position having a wider parallax than the first imaging position (a imaging position where the target mark is captured at a position close to the end of the image), and the position of the robot 20 relative to the work space S is accurately corrected based on the measurement result.

[0037] The operator pre-taught the first and second shooting positions using the robot 20 disposed at the reference position. In addition, the first and second shooting positions for stereo measurement actually include shooting position postures of two parts (in Figure 2 In the example, the shooting position postures are C1 and C2), but for the sake of convenience of explanation, they are recorded as the first shooting position, the second shooting position, etc.

[0038] The second shooting position is a shooting position that has been taught in a manner of capturing a target mark at the end of an image. However, since the position of the robot 20 is corrected with a certain accuracy through the first stereo measurement, the shooting at the second shooting position can avoid the situation in which the target mark deviates from the field of view due to position offset of the robot 20, etc., thereby enabling reliable and accurate stereo measurement.

[0039] Figure 3 is a functional block diagram of the robot control device 30. Figure 3 As shown, the robot control device 30 includes: an action control unit 131, a measuring unit 132, a correction unit 133, a coordinate system setting unit 134, a visual sensor control unit 135, and a storage unit 136. The action control unit 131, the measuring unit 132, the correction unit 133, the coordinate system setting unit 134, and the visual sensor control unit 135 among these functional blocks may also be functions realized by the CPU (processor) of the robot control device 30 executing software.

[0040] The storage unit 136 stores, for example, various programs such as operation programs and measurement programs, first imaging positions, second imaging positions, coordinate system information, calibration data, information related to target marks (shape data, reference three-dimensional position information, etc.), and other various information.

[0041] The coordinate system setting unit 134 provides a function for setting an input coordinate system. The coordinate system information set by the coordinate system setting unit 134 is stored in the storage unit 136, for example. The coordinate systems that can be set in the robot control device 30 (coordinate system setting unit 134) include a world coordinate system, a robot coordinate system, a flange coordinate system, a tool coordinate system, a camera coordinate system, and a user coordinate system. For example, the base 20a (refer to Figure 1 ) sets the robot coordinate system. For example, as described above, the camera coordinate system C is set for the reference position of the visual sensor 71. The user coordinate system is a coordinate system that can be set by the user, and can include a workpiece coordinate system set for a workpiece.

[0042] The motion control unit 131 controls the motion of the robot 20 according to the motion program stored in the storage unit 136 , for example. The robot control device 30 includes a servo control unit (not shown) that servo-controls the motors of the axes of the robot 20 according to the command generated by the motion control unit 131 .

[0043] The measuring unit 132 measures the three-dimensional position of the object by stereo measurement. The measuring unit 132 performs stereo measurement of the object using, for example, an image processing function (matching processing, etc.) of the visual sensor control unit 135 (image processing unit 135a). The measuring unit 132 includes a first measuring unit 132a and a second measuring unit 132b.

[0044] The first measuring unit 132 a obtains the three-dimensional position of the target mark based on the first imaging position stored in the storage unit 136 , and acquires the offset amount of the positional relationship between the workspace S and the robot 20 based on the three-dimensional position of the target mark.

[0045] The correction unit 133 corrects the position of the robot 20 based on the offset between the robot 20 and the workspace S obtained by the first measuring unit 132a. The correction unit 133 has a coordinate system data output unit 133a, which provides a function of correcting the position of the robot 20 by shifting a specific coordinate system based on the offset obtained by the first measuring unit 132a.

[0046] The second measuring unit 132b obtains the three-dimensional position of the target mark based on the second imaging position stored in the storage unit 136, and obtains the offset of the positional relationship between the workspace S and the robot 20 based on the three-dimensional position of the target mark. The second measuring unit 132b provides the calculated offset to the motion control unit 131. The motion control unit 131 corrects the motion of the robot based on the offset.

[0047] The visual sensor control unit 135 has a function of controlling the visual sensor 71 and a function of performing image processing on the image captured by the visual sensor 71. The visual sensor control unit 135 has an image processing unit 135a that performs the image processing function. The visual sensor control unit 135 is capable of performing a detection process for detecting an object by matching the image of the object in the captured image with a model pattern. Data such as the model pattern of the object can also be stored in the storage unit 136. The visual sensor 71 has been calibrated, and the calibration data (external parameters and internal parameters) are stored in the storage unit 136, for example. Therefore, in the robot control device 30 (visual sensor control unit 135), the relative positional relationship between the robot 20 and the visual sensor 71 is known.

[0048] Furthermore, in the present embodiment, the robot control device 30 includes the vision sensor control unit 135 , but the function of the vision sensor control unit 135 may be provided as a device separate from the robot control device 30 .

[0049] The following describes an example of calculation of the offset amount between the robot 20 and the work space S (machine tool 10) by the measuring unit 132 (the first measuring unit 132a and the second measuring unit 132b). Figure 4 As shown, in this example, three target marks Ta, Tb, and Tc provided on the surface 10a of the machine tool are respectively measured by stereo measurement to measure the three-dimensional position of each target mark Ta, Tb, and Tc. Figure 4 , the visual sensor 71 takes images of the target marks Ta, Tb, and Tc from two locations.

[0050] Next, the measuring unit 132 synthesizes the three-dimensional positions of the three target marks Ta, Tb, and Tc obtained to obtain the three-dimensional position and posture of the work space S (machine tool 10) relative to the robot 20. That is, three-dimensional measurements are performed on three parts of an object, and the position and posture of the entire object are obtained by synthesizing these measurement results. Here, three parts on the surface 10a of the machine tool 10 are measured, and the position and posture of the entire machine tool are calculated.

[0051] The measuring unit 132 calculates the three-dimensional position (X, Y, Z, W, P, R) of the entire tool 10 based on the three-dimensional positions (X, Y, Z) of the three target marks Ta, Tb, and Tc. At this time, the measuring unit 132 calculates a coordinate system in which the position of the first target mark is determined as the origin, the position of the second target mark is determined as a point in the X-axis direction, and the position of the third target mark is determined as a point in the XY plane, thereby calculating the three-dimensional position (X, Y, Z, W, P, R) of the entire tool.

[0052] The measuring unit 132 obtains the offset between the three-dimensional position (X, Y, Z, W, P, R) of the machine tool 10 obtained by measurement and the reference three-dimensional position (or the past three-dimensional position) of the machine tool 10. That is, the measuring unit 132 performs translation and rotation operations so that the coordinate system of the machine tool 10 measured this time overlaps the coordinate system of the reference position, and uses the movement amount of the coordinate system thus obtained as the offset (correction amount).

[0053] In addition, here, an example of obtaining the positional relationship between the robot 20 and the workspace S (machine tool 10) by using three target marks is described, but in the case of two points, the rotation amount about the line segment connecting the two points cannot be determined. However, if the rotation amount is difficult to change in the system structure, a sufficiently practical measurement can be performed.

[0054] When the positional relationship between the robot 20 and the work space S (machine tool 10 ) is determined using one target marker, this can be achieved by using a specific target marker (a marker whose position, shape, size, etc. are known).

[0055] Figure 5 1 is a flowchart showing a specific operation example of the stereo measurement process performed by the measuring unit 132. The stereo measurement process is executed under the control of the CPU of the robot control device 30. The measurement program of the stereo measurement process can also be stored in the storage unit 136. In this operation example, the calculation of the offset between the robot 20 and the work space S is performed by synthesizing the measurement positions of the three target marks (Ta, Tb, Tc) mentioned above. For each of the three target marks (Ta, Tb, Tc), a shooting position with a relatively narrow parallax (first shooting position) and a shooting position with a relatively wide parallax (second shooting position) are taught in advance and stored in the storage unit 136.

[0056] As an example, the first imaging position is taught according to the robot coordinate system (expressed by the robot coordinate system). In teaching the first imaging position, it is not necessary to expand the parallax, so the operator does not need to teach the position and posture of the visual sensor 71 so that the target mark is located at the end of the field of view.

[0057] The second shooting position is a shooting position where the parallax is wider than that of the first shooting position. For the second shooting position, the operator teaches in such a way that the target mark is located close to the end of the field of view. As an example, the second shooting position can also be taught according to the user coordinate system set by the operator (it can also be expressed by the user coordinate system). In this case, the operator can make the robot 20 move according to the user coordinate system and teach the second shooting position, so it is easy to teach the second shooting position. The user coordinate system in this case can also be, for example, a coordinate system (1) set on the target mark in the working space S, or (2) set at a position relatively close to the target mark.

[0058] For convenience of the following description, the coordinate system used for teaching the second imaging position is referred to as user coordinate system UF1.

[0059] like Figure 5 As shown, first, the conveying device 81 is moved to a predetermined position for performing work by user operation or automatically (step S101). The processing of steps S102 to S106 is handled by the first measuring unit 132a and the correction unit 133. The first measuring unit 132a cooperates with the motion control unit 131 to move the robot 20 (arm 20b) to the first imaging position (step S102).

[0060] Next, the first measuring unit 132a performs imaging of the target mark at the first imaging position through the visual sensor 71 (step S103). Specifically, the first measuring unit 132a performs imaging of the target mark at the first imaging position through the visual sensor 71 via the visual sensor control unit 135, and causes the visual sensor control unit 135 (image processing unit 135a) to perform detection processing for detecting the target mark in the photographic image. The detection processing of the visual sensor control unit 135 (image processing unit 135a) includes extracting the features of the target mark from the photographic image and detecting the position of the target mark through matching processing.

[0061] The first measuring unit 132a then checks whether there is a target mark in the captured image (step S104). The first capturing position is set at a position where the target can be easily captured within the field of view of the visual sensor 71, so it is usually determined that there is a target mark (S104: Yes). If it is determined that there is no target mark in the captured image (S104: No), this process ends.

[0062] The processes of steps S102 to S104 are executed for each of the three target tags (Ta, Tb, Tc).

[0063] Next, in step S105, the first measuring unit 132a obtains the three-dimensional positions of the target marks (Ta, Tb, Tc) for each of the three target marks (Ta, Tb, Tc) by stereo measurement based on the captured image captured at the first capturing position. The first measuring unit 132a obtains the offset of the robot 20 and the workspace S relative to the desired relative position relationship based on the measured three-dimensional position. The positions of the target marks (Ta, Tb, Tc) detected on the captured image can be obtained as position information on the robot coordinate system (or world coordinate system) by using calibration data. In detail, the calibration data includes information indicating the position (camera coordinate system) of the camera (visual sensor 71) based on the coordinate system (mechanical coordinate system) set at the fingertips of the robot 20. On the other hand, the robot control device 30 can grasp the position of the fingertips of the robot in the robot coordinate system (mechanical coordinate system). Therefore, by using the calibration data to correspond the two-dimensional points of the sensor coordinate system to the three-dimensional points of the machine coordinate system, and transforming the machine coordinate system into the robot coordinate system, it is possible to associate the two-dimensional points on the sensor coordinate system when photographed by the visual sensor with the three-dimensional points on the robot coordinate system. In this way, the three-dimensional position of the target mark can be obtained. In this way, the offset of the robot relative to the work space S can be obtained. In addition, the known relationship between the robot coordinate system and the world coordinate system (the coordinate system set in the work space S) can also be used to calculate the position of the target mark as the position on the world coordinate system.

[0064] Next, the coordinate system data output unit 133a shifts the coordinate system (user coordinate system UF1) used in teaching the second imaging position according to the offset (step S106). As a result, the user coordinate system U1 is updated to a state in which the offset is corrected, so that the robot can be correctly moved to the teaching point previously taught in the user coordinate system UF1. That is, the position and posture of the fingertips of the robot 20 (the position and posture of the visual sensor 71) are corrected by the stereo measurement performed by the first measuring unit 132a.

[0065] In addition, the correction of the position of the robot 20 here is based on the stereo measurement of the captured image obtained by capturing the target mark at a position relatively close to the center of the captured image. Therefore, it is not a high-precision measurement, but in the shooting at the next second shooting position, it is a sufficient measurement to prevent the target mark from deviating from the captured image.

[0066] The offset amount acquired by the first measuring unit 132a is referred to as O1. The offset amount O1 is calculated as, for example, an offset amount in the robot coordinate system. The coordinate system data output unit 133a calculates a coordinate system UF1′ obtained by shifting the user coordinate system UF1 using the following calculation formula (2).

[0067] UF1'=UF1•O1 …(2)

[0068] Here, UF1 represents the origin position of the user coordinate system UF1 viewed from the robot coordinate system, and is represented by a simultaneous transformation matrix.

[0069] The coordinate system data output unit 133a outputs the value of the calculation result UF1' as the coordinate system data of the user coordinate system UF1. The coordinate system data output unit 133a can operate in a manner of updating the user coordinate system UF1 by sending the calculation result to the coordinate system setting unit 134. As a result, the user coordinate system UF1 becomes a coordinate system in which the original user coordinate system UF1 is shifted according to the offset.

[0070] The coordinate system data output unit 133 a may be configured to present a UI (user interface) for designating an extraction source of a detection result (offset amount) for shifting the coordinate system and designating a coordinate system to be shifted. Figure 6 4 is a diagram showing an example of a setting screen 460 for performing such setting. The setting screen 460 may be presented on the display screen of the display unit 51 of the teaching device 50 , and may be configured so that setting input can be performed via the operation unit 52 .

[0071] like Figure 6 As shown in FIG. 4 , the setting screen 460 includes a selection field 461 for selecting the source of the result set in the user coordinate system (i.e., the offset calculated by the first measuring unit 132a). The offset, which is the measurement result of the first measuring unit 132a, is output to a specific output destination (a specific register, etc.). In this case, in the selection field 461, for example, the register number from which the offset calculated by the first measuring unit 132a is to be extracted is specified.

[0072] The setting screen 460 also includes a designation column 462 for designating the user coordinate system number to be shifted. In the designation column 462, the user coordinate system to which the coordinate system is to be shifted is designated using the detection result selected in the selection column 461. In the case of the above-mentioned mathematical formula (2), the user coordinate system UF1 is designated in the designation column 462. In addition, the selection in the selection column 461 and the designation in the designation column 462 can also be performed via a pop-up menu. The setting screen 460 can also be activated as a setting screen for performing detailed settings of a command text (or a command in the form of an icon) indicating a function of shifting a coordinate system.

[0073] The processing of the next steps S107 to S110 is handled by the second measuring unit 132b. The second measuring unit 132b moves the robot 20 (arm 20b) to the second shooting position (step S107). As described above, the second shooting position is taught by the user coordinate system U1, and here, the robot 20 moves according to the user coordinate system U1 updated (corrected) as described above. Next, the second measuring unit 132b performs shooting at the second shooting position through the visual sensor 71 (step S108). In addition, for the sake of convenience, it is recorded that the shooting is performed by moving to the second shooting position, but in fact, the second shooting position includes shooting positions of two parts, and the visual sensor 71 shoots the target mark at the positions of the two parts. The processing of steps S107 and S108 is performed on the three target marks (Ta, Tb, Tc), respectively.

[0074] The second measuring unit 132b performs detection processing of the target mark based on the captured image through the visual sensor control unit 135. The second measuring unit 132b obtains the three-dimensional position of the target mark by calculating the stereo measurement method based on the position of the target mark obtained by the detection processing. The stereo measurement here is based on the measurement of the image obtained by capturing the target mark at a position close to the end of the image (the measurement is performed by photographing in a manner that the parallax becomes wider), so an accurate measurement result can be obtained. Therefore, the second measuring unit 132b can obtain an accurate offset related to the relative positional relationship between the robot 20 and the workspace S (step S109).

[0075] The second measuring unit 132b provides the obtained accurate offset to, for example, the motion control unit 131, and the motion control unit 131 corrects the motion of the robot 20 based on the obtained offset (step S110). Thus, the motion of the robot is corrected based on the offset of the positional relationship between the robot 20 and the work space S obtained by the second measuring unit 132b. The offset measured by the second measuring unit 132b is accurate, so the motion of the robot 20 is accurately corrected.

[0076] about Figure 5 In the above-mentioned embodiment, the coordinate system output unit 133a is described as operating in the case where the position of the robot 20 is corrected by shifting the preset coordinate system (user coordinate system). On the other hand, the coordinate system data output unit 133a can also operate in a manner of setting the user coordinate system. In this case, the operation is performed according to the following process.

[0077] (Process A1) Position the conveyor device 81 at the reference position and perform “user coordinate system setting” in advance. Here, “user coordinate system setting” means setting the conveyor device 81 at the reference position. Figure 7The UI screen 450 as illustrated performs (b1) selection of a storage destination of the detection result set in the user coordinate system, and (b2) designation of a user coordinate system that reflects the selected detection result. In this operation example, the coordinate system data output unit 133a provides such a setting screen 450. The setting screen 450 can also be displayed on the display screen of the display unit 51 of the teaching device 50, and is configured to enable setting input via the operation unit 52. The setting screen 450 includes (b1) a selection bar 451 for selecting a storage destination of the detection result set in the user coordinate system, and (b2) a designation bar 452 for designating a user coordinate system that reflects the selected detection result. In the selection of the storage destination of the detection result (b1), the storage destination of the measurement result at the first imaging position is selected. In the designation of the user coordinate system (b2), the user can designate the number of any user coordinate system. Then, the coordinate system data output unit 133 a performs measurement at the first imaging position, generates coordinate system data based on the detection result, and sets the coordinate system data as the coordinate system data of the number designated by the user in (b2).

[0078] Specifically, the coordinate system data output unit 133a operates as follows.

[0079] (Process A1-1) The detection position P1 is acquired from the storage destination specified in the above (b1). The detection position P1 is the position of the target mark on the coordinate system UFk used in teaching the first imaging position.

[0080] (Process A1-2) The coordinate system data output unit 133 a transforms the detected position P1 into a position P1 ′ in the robot coordinate system (or the world coordinate system) using the following mathematical formula (3).

[0081] P1'=UFk•P1 …(3)

[0082] Here, UFk represents the origin position of UFk observed from the robot coordinate system (or the world coordinate system), and is represented by a simultaneous transformation matrix.

[0083] (Process A1-3) The coordinate system data output unit 133a inputs the content of P1' as the coordinate system data of the user coordinate system specified in (b2). Thus, the user coordinate system specified in (b2) becomes a coordinate system with P1' as the origin when viewed from the robot coordinate system (or world coordinate system).

[0084] (Process A2) Next, the positional relationship between the robot 20 and the work space S is maintained at the state in process A1, and the second imaging position is taught in the user coordinate system set in process A1-3.

[0085] (Process A3) The positional relationship between the robot 20 and the workspace S remains unchanged, and the robot 20 is taught a correction action based on the offset obtained in process A2. In this case, the action of the robot 20 can also be corrected based on the accurate offset obtained by measuring the target mark at the second imaging position.

[0086] According to the stereo measurement process of the present embodiment described above, stereo measurement can be reliably performed at a shooting position where parallax becomes large (shooting in which a target is captured at an end of an image), thereby reliably performing high-precision stereo measurement.

[0087] Furthermore, according to the present embodiment, it is possible to increase the tolerance for positional deviation of a robot-mounted cart or AGV, and to improve the flexibility as an actual production system.

[0088] As described above, according to the present embodiment, high-precision stereo measurement can be reliably performed.

[0089] The present disclosure has been described in detail, but the present disclosure is not limited to the above-mentioned embodiments. These embodiments can be variously added, replaced, changed, partially deleted, etc. without departing from the scope of the main purpose of the present disclosure, or without departing from the scope of the main purpose of the present disclosure derived from the contents recorded in the claims and their equivalents. In addition, these embodiments can also be implemented in combination. For example, in the above-mentioned embodiments, the order of each action and the order of each processing are shown as an example and are not limited to this. In addition, the same applies to the case where numerical values ​​or mathematical formulas are used in the description of the above-mentioned embodiments.

[0090] Figure 3 The configuration of the functional blocks in the functional block diagram shown in FIG. 1 is an example, and various modifications can be made. For example, Figure 3 In the configuration example, at least a part of the functional blocks arranged in the robot control device 30 is arranged in the teaching device 50. In this case, the functions of the robot control device 30 and the teaching device 50 can be defined as a robot control device as a whole.

[0091] Figure 3 The functional blocks of the robot control device shown may be implemented by the CPU of the robot control device executing various software stored in a storage device, or may be implemented by a configuration mainly based on hardware such as an ASIC (Application Specific Integrated Circuit).

[0092] Execute the stereo measurement process in the above-mentioned embodiment ( Figure 5) and other processing programs can be recorded in various computer-readable recording media (for example, semiconductor memories such as ROM, EEPROM, and flash memory, magnetic recording media, and optical disks such as CD-ROM and DVD-ROM).

[0093] Description of Reference Numerals

[0094] 10 Machine Tools

[0095] 20 Robot

[0096] 20a Base

[0097] 20b Arm

[0098] 21 Front end of arm

[0099] 30 Robot control device

[0100] 71 Vision Sensor

[0101] 50 Teaching device

[0102] 51 Display

[0103] 52 Operation Department

[0104] 81 Conveying device

[0105] 100 Robotic Systems

[0106] 131 Motion Control Unit

[0107] 132 Measurement Department

[0108] 132a First measurement unit

[0109] 132b Second measurement unit

[0110] 133 Correction Department

[0111] 133a Coordinate system data output unit

[0112] 134 Coordinate system setting unit

[0113] 135 Vision sensor control unit

[0114] 135a Image processing unit

[0115] 136 Storage Department.

Claims

1. A robot control device for controlling a robot, characterized in that: The robot control device has: a storage unit storing a first shooting position for performing stereo measurement of an object set in a work space by a visual sensor mounted on the robot and a second shooting position for performing stereo measurement of the object by the visual sensor and having a larger parallax than the stereo measurement at the first shooting position; a first measuring unit configured to perform stereoscopic measurement of the object at the first photographing position to obtain a first offset of the robot relative to the working space; a correction unit configured to correct the position of the robot during stereo measurement at the second shooting position according to the first offset; a second measuring unit that performs stereo measurement of the object at the second imaging position to obtain a second offset of the robot relative to the work space; and A motion control unit is configured to correct the motion of the robot according to the second offset.

2. The robot control device according to claim 1, characterized in that: The robot control device further includes a correction unit that corrects the position of the robot by shifting a specific coordinate system expressing the second imaging position according to the first offset amount.

3. The robot control device according to claim 1, characterized in that: The first shooting position is a teaching position represented by a first coordinate system, and in the stereo measurement based on the first measuring unit, the robot moves according to the first coordinate system. The second shooting position is a teaching position expressed by a second coordinate system different from the first coordinate system, and in the stereo measurement based on the second measuring unit, the robot moves according to the second coordinate system. The robot control device includes a correction unit that corrects the position of the robot by shifting the second coordinate system according to the first offset amount.

4. The robot control device according to claim 3, characterized in that: The second coordinate system is a user coordinate system set according to user settings.

5. The robot control device according to claim 3 or 4, characterized in that: The correction unit provides a user interface for specifying the second coordinate system, which is an object to be shifted according to the first offset.

6. The robot control device according to claim 1, characterized in that: The calibration unit includes a coordinate system data output unit that generates and sets coordinate system data representing a specific coordinate system based on the detected position of the object by the first measuring unit. The second imaging position is represented by the specific coordinate system.

7. A robot control device, characterized in that: have: a storage unit storing a first shooting position for performing stereo measurement of an object set in a work space by a visual sensor mounted on a robot and a second shooting position for performing stereo measurement of the object by the visual sensor and having a larger parallax than the stereo measurement at the first shooting position; a measuring unit that sequentially performs stereo measurement of the object at the first photographing position and stereo measurement of the object at the second photographing position; a coordinate system data output unit configured to output coordinate system data representing a specific coordinate system based on a first offset between the robot and the work space obtained by the stereo measurement at the first imaging position; as well as a motion control unit that corrects the motion of the robot according to a second offset of the robot relative to the work space obtained by stereo measurement at the second shooting position, The second imaging position is represented by the specific coordinate system.

8. The robot control device according to claim 7, characterized in that: The coordinate system data output unit provides a user interface for accepting designation of a coordinate system of an output destination to output the coordinate system data.

9. The robot control device according to claim 7 or 8, characterized in that: The specific coordinate system is a user coordinate system set according to user settings.

10. A robot control device, characterized in that: have: a storage unit storing a first shooting position for performing stereo measurement of an object set in a work space by a visual sensor mounted on a robot and a second shooting position for performing stereo measurement of the object by the visual sensor and having a larger parallax than the stereo measurement at the first shooting position; a measuring unit that sequentially performs stereo measurement of the object at the first photographing position and stereo measurement of the object at the second photographing position; a coordinate system data output unit configured to output coordinate system data representing a specific coordinate system based on a detected position of the object obtained by the stereo measurement at the first imaging position; as well as a motion control unit that corrects the motion of the robot according to an offset of the robot relative to the work space obtained by stereo measurement at the second shooting position, The second imaging position is represented by the specific coordinate system.

11. The robot control device according to claim 10, characterized in that: The coordinate system data output unit provides a user interface for accepting designation of a coordinate system of an output destination to output the coordinate system data.

Images