Robot system, method for controlling robot system, method for manufacturing article using robot system, robot, method for controlling robot, information processing device, information processing method, program and recording medium

Abstract

To improve the operational accuracy of a robot.SOLUTION: A robot system includes a robot, an imaging device, and a control device. The control device acquires information about a position of a part that functions as a marker acquired by the robot and information about a position of the part that functions as the marker acquired by the imaging device.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a robot. [Background technology]

[0002] Conventionally, there are systems that use a robot and a camera together and correct the robot's movement using a robot vision such as a camera, as described in the following Patent Documents 1 and 2. In such systems, it is possible to make the robot perform a desired movement by correcting the robot vision. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2018 / 092236 [Patent Document 2] Japanese Patent Application Laid-Open No. 2016-78195 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the robot systems described in Patent Document 1 and Patent Document 2, the robot's position cannot reproduce the command value and deviates due to mechanical characteristics such as backlash in the robot's mechanical parts, or calculation errors during calibration of the relative posture between the imaging device and the robot. When a robot is tasked with performing a task requiring high positional accuracy, this can lead to the robot's failure. Patent Document 1 describes a method for correcting factors inherent in the robot's mechanism. However, Patent Document 1 does not take into account the camera of the robot vision system, and is unable to correct for factors inherent in the calibration between the camera and the robot. Patent Document 2 describes a method for improving the calibration accuracy between the camera and the robot. However, Patent Document 2 only considers the calibration of the relative posture between the camera and the robot, and is unable to correct for factors inherent in the robot's mechanism. [Means for solving the problem]

[0005] The present invention employs a robot system comprising a robot, an imaging device, and a control device, wherein the control device acquires information regarding the position of a part that functions as a marker acquired by the robot, and information regarding the position of the part that functions as a marker acquired by the imaging device. [Effects of the Invention]

[0006] According to the present invention, it is possible to improve the operation accuracy of a robot. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram of a robot system 1000 according to an embodiment. [Figure 2] 1 is a schematic diagram of a marker 1 according to an embodiment. [Figure 3] 1 is a block diagram of a control device 100 according to an embodiment. [Figure 4] 3 is a control flowchart according to the embodiment. [Figure 5] 1 is a diagram for explaining placement of a marker 1 in an embodiment. [Figure 6] 1 is a diagram for explaining placement of a marker 1 in an embodiment. [Figure 7] 1 is a diagram for explaining placement of a marker 1 in an embodiment. [Figure 8] 3 is a control flowchart according to the embodiment. [Figure 9] FIG. 7 is a diagram showing a correction value table 700 in the embodiment. [Figure 10] FIG. 4 is a diagram showing a correction map in the embodiment. [Figure 11] 10A and 10B are diagrams for explaining a method of interpolating correction values ​​in a correction map in the embodiment. [Figure 12]4A and 4B are diagrams showing a state in which an assembly workpiece 401 is being fitted into a hole in an assembly receiving workpiece 402 in the embodiment. [Figure 13] 10A and 10B are diagrams for explaining a case where a command value of the robot 5 is corrected in the embodiment. [Figure 14] 3 is a control flowchart according to the embodiment. [Figure 15] 10A and 10B are diagrams showing an outline of workpiece assembly when an on-hand imaging device 3 is used in the embodiment. [Figure 16] FIG. 6 is a diagram illustrating an example of a GUI for setting a placement position 603 in the embodiment. [Figure 17] FIG. 1 is a schematic diagram of a robot system 1000 according to an embodiment. [Figure 18] FIG. 1 is a schematic diagram of a robot system 1000 according to an embodiment. [Figure 19] FIG. 1 is a schematic diagram of a robot system 1000 according to an embodiment. [Figure 20] FIG. 1 is a schematic diagram of a robot system 1000 according to an embodiment. [Figure 21] FIG. 1 is a schematic diagram of a robot system 1000 according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, a description will be given of an embodiment of the present invention with reference to the accompanying drawings. The embodiment shown below is merely an example, and those skilled in the art can appropriately modify the detailed configuration, for example, without departing from the spirit of the present invention. Furthermore, the numerical values ​​used in the present embodiment are for reference only and do not limit the present invention. In the following drawings, the arrows x, y, and z in the figures indicate the coordinate system of the entire system. Generally, the xyz three-dimensional coordinate system indicates the world coordinate system of the entire installation environment. In addition, a local coordinate system may be used as appropriate for control purposes, etc.

[0009] (First embodiment) Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is an explanatory diagram showing a schematic configuration of a robot system 1000 according to this embodiment. The robot system 1000 includes a robot 5 equipped with an end effector 2, an imaging device 3, a robot control device 100, a PLC (Programmable Logic Controller) 200, a base 300, and a mounting base 400. The robot system 1000 also includes a work area 4 where the robot 5 actually performs work. As will be described later, to improve the accuracy of the robot 5's movement in the work area 4, the movement of the robot 5 is corrected while a marker 1 is placed on the mounting base 400 or the mounting base 300. The marker 1 has a portion that functions as a marker. The robot 5, the mounting base 400, the robot control device 100, and the PLC 200 are mounted on the mounting base 300, but this is not limiting. For example, the robot 5, the mounting base 400, the robot control device 100, and the PLC 200 may be mounted in a location separate from the mounting base 300. The robot control device 100 may be referred to as an information processing device.

[0010] While FIG. 1 shows the work area 4 as a rectangular parallelepiped, this is not a limitation and the work area 4 may be set to a cylindrical, flat, triangular, or other shape depending on the task to be performed by the robot. However, in the work area 4, the surface on which the marker 1 is placed must be a two-dimensional plane. When performing three-dimensional tasks within the work area 4, a stand 400 for placing the marker 1 may be placed, and the surface of the stand 400 may be made variable in the Z-axis direction. It is desirable that the vertical position of the stand 400 be known, but this is not necessarily the case if the depth accuracy of the posture measurement by the image capture device 3 is sufficiently high. The image capture device 3 is installed and fixed to a mechanism such as a ceiling or tabletop so that it can overlook the work area 4.

[0011] The robot 5 is a manipulator and is equipped with an end effector 2 as a holding unit. The robot 5 is a vertically articulated robot arm. In this embodiment, the robot 5 is illustrated as a six-axis robot, but this is not limited to this. The end effector 2 is a robot hand and is attached to a predetermined location, for example, the tip of the robot arm. In this embodiment, a robot hand is used as an example, but this is not limiting, and various tools such as a screwdriver, cutting tool, polishing tool, suction hand, etc. may be used. In this embodiment, the end effector 2 is configured to be able to grasp the marker 1 with its fingers.

[0012] With the above configuration, the robot 5 can move the end effector 2 to any position and perform a desired task. For example, by using a workpiece and another workpiece as materials and performing a process of assembling the workpiece and the other workpiece, an assembled workpiece can be manufactured as a finished product. As described above, an article can be manufactured by the robot system 1000. Note that in this embodiment, the manufacturing of an article by assembling workpieces using the robot system 1000 has been described as an example, but the present invention is not limited to this. For example, the end effector 2 may be provided with a tool such as a cutting tool or a polishing tool, and the article may be manufactured by processing the workpiece.

[0013] 2 shows an example of a marker 1. The marker 1 has a gripping portion 10 that can be gripped by the end effector 2. However, the gripping portion 10 does not necessarily have to be specially provided, and depending on the shape of the marker 1, the marker 1 itself, such as its outer shape or its components, may have the function of the gripping portion 10. However, when the gripping portion 10 is attached to the marker 1, the relative relationship between the tip orientation of the end effector 2 when the end effector 2 grips the gripping portion 10 and the orientation obtained when the marker 1 is measured by the imaging device 3 must be known.

[0014] The marker 1 has a mechanism for uniquely measuring the orientation of the marker 1 when photographed by the imaging device 3, and a figure consisting of one triangle and one circle is printed on it. However, this is not a limitation as long as the orientation of the marker 1 can be uniquely measured, and another symbol may also be used. Furthermore, the workpiece to be used in the task to be actually performed by the robot 5 may also be used as the marker 1. However, the marker 1 must be able to assume a stable orientation when grasped by the end effector 2 and placed in a specific position.

[0015] Next, the robot control device 100 will be described with reference to the drawings. FIG. 3 shows a block diagram of the robot control device 100. The robot control device 100 includes a CPU (Central Processing Unit) 101, which is an example of a processor. The CPU 101 is an example of a processing unit. The robot control device 100 also includes a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, and an HDD (Hard Disk Drive) 104 as storage units. The robot control device 100 is connected to the robot 5, the end effector 2, and the imaging device 3 so as to be able to communicate with each other, and each connected device can be controlled from the robot control device 100. The CPU 101, ROM 102, RAM 103, HDD 104, robot 5, end effector 2, and imaging device 3 are connected via a bus so as to be able to communicate with each other. In addition to these devices, devices such as a mouse 105, a keyboard 106, and a display 107 are connected as interfaces of the robot control device 100.

[0016] The ROM 102 stores a basic program related to the operation of the computer. The RAM 103 is a storage device that temporarily stores various data such as the results of calculations performed by the CPU 101. The HDD 104 stores the results of calculations performed by the CPU 101, various data acquired from the outside, such as data from sensors possessed by the robot 5 and images from the imaging device 3, as well as programs for causing the CPU 101 to execute various processes. The HDD 104 program is application software that can be executed by the CPU 101. By executing the programs stored in the HDD 104, the CPU 101 can execute control processes for the robot 5 and the imaging device 3, which will be described later.

[0017] In this embodiment, the non-transitory computer-readable recording medium is the HDD 104, and the program is recorded on the HDD 104, but this is not limiting. The program may be recorded on any non-transitory computer-readable recording medium. Examples of recording media that can be used to supply the program to a computer include flexible disks, hard disks, optical disks, magneto-optical disks, magnetic tapes, and non-volatile memories. Solid State Drives (SSDs) may also be used. In this embodiment, the robot 5 and the imaging device 3 are executed by one control device, but they may also be executed by multiple computers.

[0018] The CPU 101 executes the program stored in the HDD 104 to cause the imaging device 3 to capture an image of a predetermined area. Then, from the image obtained by capturing the image, the CPU 101 corrects control errors of the robot 5 due to mechanical factors of the robot 5 and factors resulting from the relative positional relationship between the robot 5 and the imaging device 3. The CPU 101 also executes the program stored in the HDD 104 to cause the imaging device 3 to capture an image of an area in real space and measure the position and orientation of the marker 1 by pattern matching processing, which is an example of image processing. The CPU 101 also executes the program stored in the HDD 104 to control the robot 5 and move the end effector 2 to a predetermined position in real space. The CPU 101 also executes the program stored in the HDD 104 to hold the marker 1 with the end effector 2 and move the marker 1 to a predetermined position.

[0019] Next, a method for acquiring a correction value for a command value of the robot 5 for correcting the operation of the robot 5 will be described with reference to the drawings. Fig. 4 shows a flowchart for acquiring data for acquiring the correction value. The flowchart shown in Fig. 4 is assumed to be executed by the robot control device 100.

[0020] 4, first, in step S1, the robot 5 and end effector 2 are controlled to have the end effector 2 grasp the marker 1. The grasping method is arbitrary, and the robot 5 may be taught a posture that allows the end effector 2 to grasp the marker 1, and then the robot 5 may grasp the marker 1, or the marker 1 may be manually placed on the end effector 2 of a stopped robot 5 and grasped.

[0021] Next, in step S2, the grasped marker 1 is placed at any position within the working area 4. The method of placement will be described later.

[0022] Next, in step S3, position information (placement position) of the end effector 2 when placing the marker 1 is acquired from the encoders built into each joint of the robot 5 and stored in a storage medium in the robot control device 100. The placement position in step S3 refers to a state in which the end effector 2 is statically fixed at the position where the marker 1 is placed, in other words, the position where the end effector 2 releases its grip on the marker 1. In this case, the end effector 2 is assumed to be position information of the end effector 2 when releasing the marker 1, but position information of the end effector 2 before release may also be used. In addition, the position information in this case refers to the three-dimensional coordinates (x, y, z) of the tip of the end effector 2. However, the acquired position information may be the center of the mounting flange of the end effector 2 as long as it is the part to be controlled, or may be in another format such as a homogeneous transformation matrix. Furthermore, it may also be position information of the fingers gripping the marker 1 based on the positional relationship of the mechanism in the end effector 2.

[0023] After acquiring the position information of the end effector 2 in step S3, the robot 5 is retracted in step S4 so that the placed marker 1 can be measured without any blind spots from the imaging device 3. However, if the marker 1 can be measured without retracting, step S4 may be skipped.

[0024] Thereafter, in step S5, the position of the marker 1 is acquired (measured) using the imaging device 3. It is desirable that the measured position information be in the same format as that saved in step S3, but a different format may be used as long as it is mutually convertible. The position information may be measured by a pattern matching method in which the pattern of the marker 1 is registered in advance, or by using external shape information. Other means capable of measuring position information based on a two-dimensional image may also be used.

[0025] In step S6, the measured position of the marker 1 is stored in the storage medium of the control device 100 in the same manner as in step S3.

[0026] In step S7, a correction value for the command value used when controlling the robot 5 is obtained (calculated) from the position information of the end effector 2 obtained in step S3 and the position information of the marker 1 measured by the imaging device 3 in step S6. The calculation method will be described later.

[0027] The correction value calculated in step S8 is stored in the storage medium of the robot control device 100. The above processing is performed at least two positions within the work area 4, and when it is confirmed in step S9 that the processing has been completed at the final position, data acquisition for calculating the correction value is completed. If the processing at the final position has not been completed in step S9, the process returns to step S1 again, where the marker 1 is grasped and placed at another position, and the acquisition of each position information and the acquisition of the correction value are performed. The number of positions at which the marker is placed can be set by the user according to the size of the work area to be performed and the precision of the work to be performed.

[0028] Here, when the robot 5 is moved to an arbitrary position within the working area 4, the mechanism for inputting the position coordinates of the destination of the robot 5, i.e., the command value, and outputting a corresponding correction value for the command value from the correction values ​​acquired in the flow shown in Fig. 4, is referred to as a correction map. However, in this case, the command value indicates three-dimensional coordinates (x, y, z) expressed as continuous values ​​within the working area 4, and is not limited to the placement position of the marker 1, which is a discrete value. In this case, the mechanism may be a function or may be in the form of a table, as long as it can output the relationship between the command value and the correction value.

[0029] Next, the method of placing the markers 1 in step S2 and the calculation of the correction values ​​in step S7 will be described with reference to the drawings. FIG. 5 is a diagram showing one embodiment of the method of placing the markers 1 in step S2, illustrating how the markers 1 are placed in a grid pattern on a table 400 placed in the work area 4. The positions at which the markers 1 are placed in step S2 must be set in advance. Furthermore, the table 400 need not be used, and the markers 1 may be placed directly on the platform 300 or the ground in the work area 4. Furthermore, the placement method does not need to be a grid pattern; the markers 1 may be placed in at least two positions. However, by placing a large number of markers evenly within the work area 4, it is possible to obtain local trends in the correction values ​​within the work area 4. Therefore, it is preferable to place the markers 1 in a large number of positions, as this will improve the correction effect of the ultimately obtained correction map.

[0030] Furthermore, the marker 1 to be placed may be the same one that has been re-grasped and then placed again, or if multiple markers 1 of the same shape are available, a new marker 1 may be placed without rearranging the previously placed marker 1. Furthermore, due to the influence of robot hysteresis, the actual destination posture of the robot 5 may change depending on the path that the robot 5 takes to move to the posture when the marker 1 is placed. To reduce this influence, the path along which the robot 5 will move when the marker 1 is placed may be generated so as to follow the path along which the robot 5 will move in the task that the actual robot 5 is to perform. This reduces the influence of hysteresis on the acquired correction value.

[0031] Furthermore, when the robot 5 moves along a complex path in the work that it actually performs, multiple path patterns for placing the marker 1 are prepared. Then, multiple correction maps may be generated using correction values ​​acquired for each path, and different correction maps may be used depending on the path that the robot 5 performs. Furthermore, when multiple TCPs (Tool Center Points) are used in the work that the robot actually performs, a correction map may be acquired for each TCP by placing the marker 1 in a state that is adapted to each TCP, and different correction maps may be used depending on the TCP that is used.

[0032] When placing the marker 1, it is necessary to keep the position of the marker 1 as stable as possible before and after the grip is released. Figure 6 shows details of placing the marker 1 on the stage 400 with the end effector 2. As shown in Figure 6(a), it is desirable to statically settle the end effector 2 with the marker 1 in as close contact as possible with the stage 400, and then release the grip. To achieve this, when setting the placement position of the marker 1, it is advisable to register the position, posture, and grip posture of the robot 5 and end effector 2 that will bring the marker 1 into contact with the stage 400. Furthermore, as shown in Figure 6(b), the stage 400 may be made of a resilient material 410, such as a rubber sheet, that has a low coefficient of friction and is resilient, to make it easier to press the marker 1 against it.

[0033] Alternatively, the marker 1 may be placed using a force sensor 210 that acquires (measures) the load on the end effector 2 in the placement direction of the marker 1. FIG. 7 is a diagram showing details of placing the marker 1 using an end effector 22 equipped with a force sensor 210. As shown in FIG. 7(a), the force sensor 210 measures the load (information about force) applied in the placement direction of the end effector 2, and the end effector 2 is moved in a direction approaching the table 400 by ΔP until the force sensor 210 indicates a predetermined value. Then, as shown in FIG. 7(b), the marker 1 comes into contact with the table 400, and the robot 5 is allowed to settle at a position where the force sensor 210 indicates a predetermined value, and the encoder value of the robot 5 at that time may be used as the placement position. ΔP represents a vector indicating three-dimensional coordinates (x, y, z).

[0034] Here, the control flow (steps S2 and S3) for placing the marker 1 when the force sensor 210 is used is shown in FIG. 8. The flowchart shown in FIG. 8 is executed by the robot control device 100. As shown in FIG. 8, in step S20, the command value P pThen, in step S21, the weight variable W is initialized to 0. Then, in step S22, the command value P p Next, in step S24, the robot 5 is displaced by ΔP in the pressing direction. p In step S24, the robot 5 transmits the command value P p Next, in step S25, it is determined (confirmed) that the robot 5 has settled. In determining whether the robot 5 has settled, if the change in the encoder value of the robot 5 is equal to or less than a predetermined amount within a predetermined time, it is determined that the robot 5 has settled. Alternatively, the robot 5 may be kept waiting for a predetermined time or longer in step S25 and then considered to have settled.

[0035] After confirming that the robot 5 is settled (step S25: Yes), in step S26, the force sensor 210 is used to measure the load W applied to the end effector 2. m is measured and substituted into the weight variable W. Next, in step S27, when a predetermined state is reached, W is measured and substituted into the weight variable W. M If it is equal to or greater than this, step S27: Yes is reached, and placement is completed at that position, the position of the end effector 2 at that time is recorded as the placement position, and the flow ends. If it is equal to or less than the threshold, step S27: No is reached, and the process repeats from step S22 again. As a result, it is possible to use the force sensor 210 to acquire the position and orientation of the end effector 2 when the marker 1 is placed.

[0036] 9 is a diagram showing a correction value table 700 in which correction values ​​are recorded. When a total of n markers 1 are placed, the position of the end effector 2 acquired by the encoder of the robot 5 when the 0≦k≦n−1th marker 1 is placed is defined as P RB k The recorded position P RB k The measurement position P of the marker 1 acquired by the image capturing device 3 in step S6 is recorded in the form of a correction value table 700 as shown in FIG. V k When recording, position P RB k and record it in the corresponding line. Vk and position P RB k The correction value table 700 may be saved in any format other than the correction value table 700 as long as it is possible to store the correspondence relationship.

[0037] As shown in the following formula, the recorded position P RB k and position P V k The difference between the x, y, and z components of the above is the correction value ΔP at the position k where the marker 1 is placed. k At this time, the correction value ΔP k As shown in Fig. 7, by marking the vector of θ as a vector with the magnitude of each component, starting from the position where the marker 1 is placed, it is possible to visualize the error tendency of the posture of the robot 5 within the surface where the marker 1 is placed. In the example of Fig. 10, correction values ​​are visualized only on the xy plane, but correction values ​​on the xz plane and yz plane may also be visualized.

[0038]

number

[0039] Next, a method for interpolating correction values ​​in the correction map will be explained with reference to Fig. 11. The position where the marker 1 is placed in step S2 is not necessarily the same as the position where the work is actually performed by the robot 5. Therefore, in order to associate the position information of the robot 5, which is a continuous value, with the correction value, it is necessary to continuously interpolate correction values ​​at positions other than the positions where the correction value was acquired. In this case, continuous interpolation between the acquired correction values ​​can be performed by calculating a weighted average of correction values ​​in the vicinity of the work position.

[0040] Figure 11 shows an overview of how the correction value ΔP at work position P is calculated from neighboring correction values. There are correction values ​​ΔP0, ΔP1, ΔP2, and ΔP3 calculated at the placement positions (k=0, 1, 2, 3) of marker 1 around work position P. In this case, if the Euclidean distances on the placement plane of marker 1 between the placement positions where each correction value was obtained and work position P are l0, l1, l2, and l3, the correction value ΔP at work position P can be calculated as the weighted average of the distances expressed in the following formula.

[0041]

number

[0042] Furthermore, in the above formula, the markers 1 are placed in a grid pattern in step S2. However, if the markers 1 are not placed in a grid pattern, ΔP may be calculated as a weighted average of correction values ​​within a certain Euclidean distance from the work position P on the plane on which the markers 1 are placed. Furthermore, if correction values ​​are obtained on multiple surfaces of different heights within the work area 4 using the placement table 400, the above weighted average may be expanded to three dimensions for calculation. Other interpolation methods, such as an unweighted average, may also be used. Furthermore, depending on the method for correcting the command value using a correction map, which will be described later, the above interpolation calculation may be omitted.

[0043] Next, a method for correcting the work position P of the robot 5 using the correction map shown in Fig. 10 will be described. Fig. 12 shows the state in which the workpiece 401 to be assembled is being fitted into the hole of the workpiece 402 without any correction using the correction map. In this case, the command value to the robot 5 obtained from the measurement results of the imaging device 3 without any correction is set to P w When the interpolation calculation of the correction map is performed, the command value P w The x and y position components of the correction value recorded at the x and y positions of the correction map corresponding to the x and y position components of k ) x , (ΔP k ) y Then, add it to the command value to obtain the corrected command value P' wFind it. Also, the command value P w A correction value ΔP that is closest to it may be used. In Equation 1, if l3 < l1 < l0 < l2, then ΔP becomes ΔP3. In this case, the correction effect may decrease, but the above interpolation calculation can be omitted.

[0044] Next, the control flow of the robot 5 using the correction map in this embodiment will be described. FIG. 14 is a control flow chart of the robot 5 using the correction map in this embodiment. The flowchart shown in FIG. 14 is assumed to be executed by the robot control device 100. From FIG. 14, first, in step S11, the command value P w before correction is obtained from a work program for causing the robot 5 to execute or a user instruction. The command value P w may be set in advance by the user through teaching, or may be dynamically obtained using image processing or the like. Next, in step S11, the correction value ΔP w corresponding to the command value P k is obtained from the correction map. Next, in step S12, the command value P′ w corrected from the command value P k and the correction value ΔP w is obtained by the method shown in FIG. 13. Next, the command value P′ w obtained in step S13 is transmitted as a command value to each drive device of the robot 5. Then, in step S14, the robot 5 moves based on the command value P′ w .

[0045] According to the present embodiment, a correction map for command values ​​used when controlling the robot 5 is obtained from the position information of the end effector 2 and the position information of the marker 1 measured by the imaging device 3. That is, the position information of the marker 1, a common position measurement target, is measured by the robot 5 and the imaging device 3, which have different measurement methods and measurement mechanisms. As a result, when obtaining the correction map, factors derived from the robot's mechanism can be taken into account using the position information of the marker 1 obtained from the position information of the end effector 2. Furthermore, factors derived from the calibration between the imaging device and the robot can be taken into account using the position information of the marker 1 measured by the imaging device 3. As a result, by controlling the robot 5 using the obtained correction map, command values ​​can be obtained that can correct for factors derived from the robot's mechanism and factors derived from the calibration between the imaging device and the robot, thereby improving the operation accuracy of the robot.

[0046] (Second embodiment) Next, a second embodiment will be described. In the following, the same reference numerals will be used for the same or equivalent components as those in the first embodiment, and their description will be omitted or simplified. The following description will focus on the differences from the first embodiment. In this embodiment, an on-hand camera format will be described in which the imaging device 3 is fixed to the end effector 2 and follows the movement of the robot 5.

[0047] FIG. 15 shows an overview of workpiece assembly using an on-hand imaging device 3. There are cases where the same workpiece needs to be repeatedly performed within the work area 4. In such cases, the robot origin P is first located at a representative point. origin The imaging position P is the reference position. v0 and the work position P0, and the imaging position P v0 and the relative relationship P r Next, the work position P0 and the imaging position P are calculated in the work plane using the design dimension Δy. v0 The robot origin P is moved horizontally by Δy. origin Working position P based on k and imaging position P vkThe control device 100 acquires (generates).

[0048] At this time, the difference between the design dimensions and the actual dimensions determines the work position P k and the imaging position P vk may deviate by δy from the target position on the workpiece 502. v0 The measurement position P of the workpiece 501 is V 0 and the imaging position P vk The measurement position P of the workpiece 501 is V k The difference between these two is equivalent to δy. k By adding δy to the k It is possible to obtain (generate) δy. Generally, δy can be obtained by image processing. v0 The workpiece 501 and the imaging position P vk The amount of displacement of the workpiece 502 captured in the displaced image 504 corresponds to δy.

[0049] However, even in this case, the actual working position P of the robot 5 varies depending on factors derived from the robot mechanism and factors derived from the calibration between the imaging device and the robot. k and the imaging position P vk Therefore, the work position P k needs to be corrected.

[0050] In this embodiment, too, the correction map can be obtained by executing the same flow as in Fig. 4. However, when calculating the correction value in step S7, v0 The measurement position P of the workpiece measured at V 0 as the reference, the measurement position P of the workpiece measured by the imaging device 3 V k Based on this, the correction value ΔP is calculated using the following formula: kThen, using the acquired correction value, the command value is corrected in the same way as when the overhead camera is used as in the first embodiment, and the operation of the robot 5 can be corrected by executing the control flowchart shown in Fig. 14.

[0051]

number

[0052] According to the present embodiment, by acquiring a correction map, it is possible to execute a correction that takes into account factors attributable to the robot's mechanism and factors attributable to the calibration between the imaging device and the robot. By controlling the robot 5 using the acquired correction map, it is possible to acquire command values ​​that can correct for factors attributable to the robot's mechanism and factors attributable to the calibration between the imaging device and the robot, thereby improving the robot's operational accuracy. Furthermore, the various embodiments and modified examples described above may be combined and implemented.

[0053] (Third embodiment) Next, a third embodiment will be described. In the following, the same reference numerals will be used for the same or corresponding components as those in the first and second embodiments, and their description will be omitted or simplified. The following description will focus on the differences from the first and second embodiments. In this embodiment, a stereo imaging device will be used as the imaging device 3. In this case, the imaging device 3 may be a twin-lens stereo camera, or a sensor with a three-dimensional measurement function such as LiDAR (Light Detection and Ranging). The stereo camera may be fixed in a position where it can be viewed from above over the work area 4, or may be fixed to the robot 5 or the end effector 2 and used on-hand.

[0054] Even when the imaging device 3 is used as a stereo camera as in this embodiment, a correction map can be acquired by executing the flow of Fig. 4. However, the method of measuring the position of the marker 1 in step S5 of Fig. 4 may use a known technique for alignment, such as an ICP (Iterative Closest Point) algorithm using a three-dimensional point cloud. Alternatively, matching may be performed by combining point cloud information and edge information in the image. Using the acquired position information of the marker 1, a correction map can be acquired by a method similar to that of the first or second embodiment.

[0055] According to the present embodiment, by acquiring a correction map, it is possible to execute a correction that takes into account factors attributable to the robot's mechanism and factors attributable to the calibration between the imaging device and the robot. By controlling the robot 5 using the acquired correction map, it is possible to acquire command values ​​that can correct for factors attributable to the robot's mechanism and factors attributable to the calibration between the imaging device and the robot, thereby improving the robot's operational accuracy. Furthermore, the various embodiments and modified examples described above may be combined and implemented.

[0056] (Fourth embodiment) Next, a fourth embodiment will be described. In the following, the same reference numerals will be used for the same or corresponding configurations as those in the first to third embodiments, and their description will be omitted or simplified. The following description will focus on the differences from the first to third embodiments. In this embodiment, the position where the marker 1 is placed is set by the user using a GUI (Graphical User Interface).

[0057] 16 shows how a workpiece placement position is determined using a GUI 600. The GUI 600 is displayed on a display 107 connected to the control device 100 so as to be able to communicate with the control device 100. The GUI 600 has a placement point specification window 601, a work area coordinate origin input window 604, a work area range specification window 605, a placement position indicator 606, and an execute button 608.

[0058] The user first inputs the origin orientation of the workpiece placement surface in the work area 4 into the work area coordinate origin input window 604. xyz is position information in three-dimensional space, and abc is orientation information expressed as angle information around each axis. At this time, the angle information is input using Euler angles. Next, the minimum and maximum values ​​of the x and y coordinates at which the marker 1 can be placed are input into the work area range specification window 605. The left side of the figure is the minimum value and the right side of the figure is the maximum value, but it is also possible to use the right side of the figure as the minimum value and the left side of the figure as the maximum value.

[0059] Next, the user can add a placement position 603 by clicking any location on the placement point specification window 601 with the mouse cursor 602. At this time, the coordinate value of the mouse cursor 602 in the xy plane in the work area 4 is always displayed on the placement position indicator 606. The display of the placement position indicator 606 is updated according to the position of the mouse cursor 602 in the work area 4 of the placement point specification window 601.

[0060] Furthermore, the placement point designation window 601 virtually displays the range entered in the work area range designation window 605. As a result, no matter which point on the placement point designation window 601 is designated, a location outside the range of the work area 4 will not be designated, making it possible to reduce the mistake of setting the placement position of the marker 1 outside the range of the work area 4. Of course, the placement point designation window 601 may highlight the work area 4 with a dotted or solid line, for example, so that the entire work area 4 is displayed in the placement point designation window 601. Also, it may be possible to enlarge or reduce the work area 4. Furthermore, if the work area 4 is on the placement table 400, the placement point designation window 601 may virtually display the placement table 400, and the work area 4 may be displayed on the virtually displayed placement table 400. Various information is notified to the user by such displays.

[0061] The placement order of the placement positions 603 is the order in which the placement positions 603 are set, and the execution order is indicated by arrows 607 on the GUI 600. Numbers may also be displayed near the placement positions 603. The execution order of the placement positions 603 can be changed by specifying the arrows 607 with the mouse cursor 602 to change the connection relationship, or by specifying and changing the numbers with the mouse cursor 602. The set placement positions 603 can also be moved on the placement point specification window 601 by dragging and dropping them with the mouse cursor 602.

[0062] Also, a set placement position 603 can be deleted by right-clicking the mouse 105 to display a pull-down menu and selecting Delete. When a placement position 603 is deleted, the placement points before and after the deleted placement position 603 are connected by an arrow 607, or at least one of the numbers before and after the deleted placement position 603 is changed, and the placement order is also changed to the order excluding the deleted placement position 603. For example, when the placement position 603, which is second in the execution order and indicated by "2," is deleted, the placement position 603 indicated by "1" and the placement position 603 indicated by "3" are connected by an arrow 607. Or the number of the placement position 603 indicated by "3" is changed to "2."

[0063] After setting any placement position 603, the user can transfer the coordinate information of all placement positions 603 to the control device 100 by pressing the execute button 608. At this time, the coordinate information transferred is six-axis coordinate information of each placement point based on the coordinate origin. In this embodiment, the placement position 603 is a coordinate value on the xy plane, and since the placement posture does not change depending on the placement position, the values ​​of z, a, b, and c are coordinate subtraction information, and the x and y information differ depending on the placement position 603. If the work area 4 is a three-dimensional space, the z information also differs depending on the placement position 603. After acquiring the placement position information, the control device 100 places the marker 1 based on the placement position set by the user in the control flowchart shown in FIG. 4, and becomes able to acquire a correction map.

[0064] According to the present embodiment, a correction map for command values ​​used to control the robot 5 is obtained from the position information of the end effector 2 and the position information of the marker 1 measured by the imaging device 3. That is, the position information of the common position measurement target, the marker 1, is measured by the robot 5 and the imaging device 3, which have different measurement methods and mechanisms. In this way, when obtaining the correction map, factors derived from the robot's mechanism can be taken into account using the position information of the end effector 2, and factors derived from the calibration between the imaging device and the robot can be taken into account using the position information of the marker 1 measured by the imaging device 3. By controlling the robot 5 using the obtained correction map, command values ​​can be obtained that can correct for factors derived from the robot's mechanism and factors derived from the calibration between the imaging device and the robot, thereby improving the robot's operating accuracy. Furthermore, the placement position of the marker 1 can be easily set using the GUI, making it easy to obtain the correction map.

[0065] (Other embodiments) Next, another embodiment of the present invention will be described in detail. FIG. 17 is a diagram showing an embodiment to which the present invention can be applied. In the embodiment shown in FIG. 17, at least two markers 1 are placed on a stand 400 in advance. Then, the fingers of an end effector 2 are sequentially brought into contact with or grasp the markers 1 placed on the stand 400, and the position of a predetermined part of the end effector 2 at the time of contact or grasp is acquired. The position of the predetermined part may be the position of the tip of the finger of the end effector 2, or may be a flange, etc. Then, the end effector 2 is retracted by the robot 5, and the position of the marker 1 that has been contacted or grasped is acquired by the imaging device 3. Note that the positions of each marker 1 placed on the stand 400 may be acquired by the imaging device 3 before the robot 5 is brought into contact with or grasps the marker 1.

[0066] As described above, according to the embodiment shown in FIG. 17 , a correction map for command values ​​used when controlling the robot 5 is obtained from the position information of the end effector 2 and the position information of the marker 1 measured by the imaging device 3. That is, the position information of the marker 1, a common position measurement target, is measured by the robot 5 and the imaging device 3, which have different measurement methods and measurement mechanisms. As a result, when obtaining the correction map, factors derived from the robot's mechanism can be taken into account using the position information of the marker 1 obtained from the position information of the end effector 2. Furthermore, factors derived from the calibration between the imaging device and the robot can be taken into account using the position information of the marker 1 measured by the imaging device 3. As a result, by controlling the robot 5 using the obtained correction map, command values ​​that can correct for factors derived from the robot's mechanism and factors derived from the calibration between the imaging device and the robot can be obtained, thereby improving the robot's operation accuracy.

[0067] Next, another embodiment of the present invention will be described in detail. FIG. 18 is a diagram showing an embodiment to which the present invention can be applied. In the embodiment shown in FIG. 18, a grid-shaped groove is provided on the mounting base 400. Intersecting portions of the grid function as markers. At least two intersecting portions are required. Then, the fingers of the end effector 2 are sequentially brought into contact with the intersecting portions of the grooves on the mounting base 400, and the position of a predetermined portion of the end effector 2 at the time of contact is acquired. The position of the predetermined portion may be the position of the tip of the finger of the end effector 2, or may be a flange, etc. Then, the end effector 2 is retracted by the robot 5, and the position of the intersecting portions that were contacted is acquired by the imaging device 3. Note that the positions of each intersecting portion provided on the mounting base 400 may be acquired by the imaging device 3 before the robot 5 is brought into contact with the intersecting portions. Note that, although the embodiment shown in FIG. 18 uses grid-shaped grooves, this is not limiting; any shape of groove that functions as a marker may be used. Also, as shown in FIG. 18, it is not necessary to use a groove; it is sufficient if there is a part that functions as a marker, such as a scratch on the stand 400.

[0068] As described above, according to the embodiment shown in FIG. 18 , a correction map of command values ​​used when controlling the robot 5 is obtained from the position information of the end effector 2 and the position information of the intersecting portion measured by the imaging device 3. That is, the position information of the intersecting portion, which is a common position measurement target, is measured by the robot 5 and the imaging device 3, which have different measurement methods and measurement mechanisms. As a result, when obtaining the correction map, factors derived from the robot's mechanism can be taken into account using the position information of the intersecting portion obtained from the position information of the end effector 2. Furthermore, factors derived from the calibration between the imaging device and the robot can be taken into account using the position information of the intersecting portion measured by the imaging device 3. As a result, by controlling the robot 5 using the obtained correction map, command values ​​that can correct for factors derived from the robot's mechanism and factors derived from the calibration between the imaging device and the robot can be obtained, thereby improving the robot's operation accuracy.

[0069] Next, another embodiment of the present invention will be described in detail. FIG. 19 is a diagram showing an embodiment to which the present invention can be applied. In the embodiment shown in FIG. 19, at least two reference portions that function as markers are provided on the base 400. The fingers of the end effector 2 are sequentially brought into contact with or grasped by the reference portions provided on the base 400, and the position of a predetermined portion of the end effector 2 at the time of contact or grasp is acquired. The position of the predetermined portion may be the position of the tip of the finger of the end effector 2, or may be a flange, etc. The end effector 2 is then retracted by the robot 5, and the position of the reference portion that has been contacted or grasped is acquired by the imaging device 3. Note that the positions of each reference portion arranged on the base 400 may be acquired by the imaging device 3 before the robot 5 is brought into contact with or grasps the reference portion.

[0070] As described above, according to the embodiment shown in FIG. 19 , a correction map of command values ​​used when controlling the robot 5 is obtained from the position information of the end effector 2 and the position information of the reference part measured by the imaging device 3. That is, the position information of the reference part, a common position measurement target, is measured by the robot 5 and the imaging device 3, which have different measurement methods and measurement mechanisms. As a result, when obtaining the correction map, factors derived from the robot's mechanism can be taken into account using the position information of the reference part obtained from the position information of the end effector 2. Furthermore, factors derived from the calibration between the imaging device and the robot can be taken into account using the position information of the marker 1 measured by the imaging device 3. As a result, by controlling the robot 5 using the obtained correction map, command values ​​that can correct for factors derived from the robot's mechanism and factors derived from the calibration between the imaging device and the robot can be obtained, thereby improving the robot's operation accuracy.

[0071] Next, another embodiment of the present invention will be described in detail. FIG. 20 is a diagram illustrating an embodiment to which the present invention can be applied. In the embodiment shown in FIG. 20, a stamp capable of writing an ink impression is held by the end effector 2, and at least two impressions functioning as markers are imprinted on the base 400 using the stamp. Then, the stamp held by the end effector 2 is used to imprint at least two locations on the base 400, creating at least two impressions. The position of a predetermined portion of the end effector 2 while imprinting the stamp on the base 400 is acquired. The position of the predetermined portion may be the position of the tip of the finger of the end effector 2, or a flange, etc. Alternatively, the tip of the stamp may be used if the stamp can be gripped with high precision. After imprinting with the stamp, the end effector 2 is retracted by the robot 5, and the position of the impression is acquired by the imaging device 3. After creating multiple impressions on the base 400, the order of the impressions is stored. Then, the image capturing device 3 may acquire the position of each seal imprint placed on the stand 400, and the acquired position may be associated with the order of the stored seal imprints.

[0072] In the embodiment shown in FIG. 20, the imprint is made by a stamp, but this may also be achieved by a dispenser capable of discharging a predetermined amount of ink. Also, in consideration of subsequent work using the stand 400, it is desirable that the ink be water-soluble. This allows the user or the robot 5 to easily clean the imprint stamped for acquiring position information. Of course, depending on the subsequent work, the ink may be oil-based. Also, depending on the subsequent work, an inscription may be engraved on the stand 400 by stamping it with a stamp.

[0073] As described above, according to the embodiment shown in FIG. 20 , a correction map of command values ​​used when controlling the robot 5 is obtained from the position information of the end effector 2 and the position information of the seal imprint measured by the imaging device 3. That is, the position information of the seal imprint, a common position measurement target, is measured by the robot 5 and the imaging device 3, which have different measurement methods and measurement mechanisms. As a result, when obtaining the correction map, factors derived from the robot's mechanism can be taken into account using the position information of the seal imprint obtained from the position information of the end effector 2. Furthermore, factors derived from the calibration between the imaging device and the robot can be taken into account using the position information of the seal imprint measured by the imaging device 3. As a result, by controlling the robot 5 using the obtained correction map, command values ​​can be obtained that can correct for factors derived from the robot's mechanism and factors derived from the calibration between the imaging device and the robot, thereby improving the robot's operating accuracy.

[0074] Next, another embodiment of the present invention will be described in detail. FIG. 21 is a diagram illustrating an embodiment to which the present invention can be applied. In the embodiment shown in FIG. 21, a sheet of paper on which at least two portions functioning as markers are printed is placed on the table 400. The sheet is placed on the table 400 so that it will not shift position on the table 400 even when the end effector 2 comes into contact with it. The fingers of the end effector 2 are then sequentially brought into contact with the marker portions printed on the sheet of paper on the table 400, and the position of a predetermined portion of the end effector 2 at the time of contact is acquired. The position of the predetermined portion may be the position of the tip of the finger of the end effector 2, or may be a flange, etc. The end effector 2 is then retracted by the robot 5, and the position of the contacted marker portion is acquired by the imaging device 3. Note that the position of each marker portion of the sheet of paper on the table 400 may be acquired by the imaging device 3 before the robot 5 comes into contact with the marker portions.

[0075] As described above, according to the embodiment shown in FIG. 21 , a correction map for command values ​​used when controlling the robot 5 is obtained from the position information of the end effector 2 and the position information of the marker portion measured by the imaging device 3. That is, the position information of the marker portion, a common position measurement target, is measured by the robot 5 and the imaging device 3, which have different measurement methods and measurement mechanisms. As a result, when obtaining the correction map, factors derived from the robot's mechanism can be taken into account using the position information of the marker portion obtained from the position information of the end effector 2. Furthermore, factors derived from the calibration between the imaging device and the robot can be taken into account using the position information of the intersecting portion measured by the imaging device 3. As a result, by controlling the robot 5 using the obtained correction map, command values ​​that can correct for factors derived from the robot's mechanism and factors derived from the calibration between the imaging device and the robot can be obtained, thereby improving the robot's operation accuracy.

[0076] The processing procedures of the above-described embodiments are specifically executed by at least one CPU and / or user input. Therefore, a recording medium on which a software program capable of executing the above-described functions is recorded can also be read and executed. In this case, the program read from the recording medium itself realizes the functions of each of the above-described embodiments, and the program itself and the recording medium on which the program is recorded constitute the present invention.

[0077] In addition, in each embodiment, the computer-readable recording medium is a ROM, a RAM, or a flash ROM, and the program is stored in the ROM, RAM, or flash ROM. However, the present invention is not limited to this embodiment. The program for implementing the present invention may be recorded on any computer-readable recording medium. An SSD (Solid State Drive) may also be used as the storage unit.

[0078] In the above-described embodiment, a correction map is acquired in the xy plane, but this is not limiting. For example, when acquiring a correction map in three-dimensional space, the mounting surface of the marker 1 on the mounting table 400 or the gantry 300 can be made variable in the z-axis direction. Then, the mounting surface is positioned at a predetermined position in the z-axis direction, and the marker 1 is placed at at least two positions, and information about each position is acquired. Thereafter, the mounting surface of the marker 1 is moved in the z-axis direction by a predetermined increment, and the marker 1 is again placed at at least two positions, and information about each position is acquired. By repeating this process within the range of the z-axis direction in the working area 4, a correction map in three-dimensional space can be acquired.

[0079] In the various embodiments described above, the robot may be a vertical multi-axis type, a horizontal multi-joint type, a parallel link type, an orthogonal robot, etc. The various embodiments described above are also applicable to machines that can automatically perform movements such as extension and contraction, bending and stretching, vertical movement, horizontal movement, or rotation, or a combination of these movements, based on information stored in a storage device provided in a control device.

[0080] The present invention is not limited to the above-described embodiments, and many modifications are possible within the technical concept of the present invention. Furthermore, the effects described in the embodiments of the present invention are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments of the present invention. Furthermore, the above-described various embodiments and modifications may be combined and implemented.

[0081] The disclosure of this embodiment also includes the following configurations and methods.

[0082] (Item 1) Robots and An imaging device; A robot system including a control device, The control device acquiring information regarding the position of the portion functioning as a marker acquired by the robot and information regarding the position of the portion functioning as a marker acquired by the imaging device; A robot system characterized by:

[0083] (Item 2) In the robot system according to item 1, acquiring information about a position of a predetermined part of the robot when the part functioning as the marker is placed by the robot as information about the position of the part functioning as the marker; A robot system characterized by:

[0084] (Item 3) In the robot system according to item 2, The control device acquiring information regarding the position of the predetermined part of the robot when the robot releases the part that functions as the marker; A robot system characterized by:

[0085] (Item 4) In the robot system according to item 2 or 3, The control device When the robot places the portion that functions as the marker, information regarding the position of the predetermined portion of the robot is acquired when it is determined that the robot has settled before releasing the portion that functions as the marker. A robot system characterized by:

[0086] (Item 5) In the robot system according to item 2 or 3, The control device acquiring information about the position of the predetermined part of the robot in a state where the part functioning as a marker is brought into contact with a place where the part functioning as a marker is to be placed; A robot system characterized by:

[0087] (Item 6) In the robot system according to item 5, The control device acquiring information about the position of the predetermined part of the robot in a state where the part functioning as the marker is in contact with an elastic material; A robot system characterized by:

[0088] (Item 7) In the robot system according to item 5, The control device and acquiring information about the position of the predetermined part of the robot when information about a force acquired when the part functioning as a marker is brought into contact with a place where the part functioning as a marker is to be placed is in a predetermined state. A robot system characterized by:

[0089] (Item 8) In the robot system according to item 2, The control device placing the part functioning as a marker at at least two positions, and acquiring information on the position of the predetermined part of the robot when the part functioning as a marker is placed by the robot, and information on the position of the placed part functioning as a marker acquired by the imaging device. A robot system characterized by:

[0090] (Item 9) In the robot system according to any one of items 1 to 8, The position where the portion functioning as the marker is placed can be set by the user. A robot system characterized by:

[0091] (Item 10) Item 9. The robot system according to item 9, The area in which the robot works can be set by the user. A robot system characterized by:

[0092] (Item 11) In the robot system according to item 9 or 10, notifying a user of the execution order at the position where the portion functioning as a marker is placed; A robot system characterized by:

[0093] (Item 12) In the robot system according to item 11, When a user deletes a position where the portion functioning as a marker is to be placed, the execution order is updated. A robot system characterized by:

[0094] (Item 13) In the robot system according to item 11 or 12, notifying the user of the execution order by arrows or numbers; A robot system characterized by:

[0095] (Item 14) In the robot system according to any one of items 9 to 13, When the user places a cursor in an area where the user is prompted to input a position where the portion functioning as a marker is to be placed, the user is notified of information regarding the position of the cursor in the area. A robot system characterized by:

[0096] (Item 15) In the robot system according to any one of items 1 to 14, The control device Placing the marker two-dimensionally or three-dimensionally; A robot system characterized by:

[0097] (Item 16) In the robot system according to item 2, The control device acquiring information indicating a correction value for correcting a command value for controlling the robot based on information regarding the position of the predetermined portion of the robot and information regarding the position of the placed portion that functions as the marker, the information being acquired by the imaging device; A robot system characterized by:

[0098] (Item 17) Item 17. The robot system according to Item 16, The control device acquiring information indicating the correction value for correcting the command value based on information regarding the position of the predetermined portion of the robot, information regarding the position of the placed portion functioning as the marker acquired by the imaging device, and a reference position when controlling the robot; A robot system characterized by:

[0099] (Item 18) Item 18. The robot system according to Item 16 or 17, The information indicating the correction value is in the form of at least one of a map, a function, and a table. A robot system characterized by:

[0100] (Item 19) 19. The robot system according to any one of items 16 to 18, The control device acquiring the correction value corresponding to the command value based on information indicating the command value and the correction value, and controlling the robot; A robot system characterized by:

[0101] (Item 20) 19. The robot system according to claim 16, The control device obtaining a weighted average of the differences obtained at surrounding positions where the portions functioning as markers are placed, for positions other than the position where the portion functioning as the marker is placed, in the information indicating the correction value; A robot system characterized by:

[0102] (Item 21) In the robot system according to any one of items 16 to 20, The control device The information indicating the correction value is visualized using an arrow that expresses the amount and direction by a size and a direction, respectively, starting from the position where the portion functioning as the marker is placed. A robot system characterized by:

[0103] (Item 22) 22. The robot system according to any one of items 1 to 21, The imaging device is provided at a position overlooking an area where the robot works, or is provided on the robot, or is a stereo camera. A robot system characterized by:

[0104] (Item 23) In the robot system according to item 2, The control device acquiring information about the position of the predetermined part of the robot using an encoder mounted on the robot; A robot system characterized by:

[0105] (Item 24) In the robot system according to item 2, The predetermined portion is an end effector or a flange to which the end effector is attached. A robot system characterized by:

[0106] (Item 25) 25. The robot system according to any one of items 1 to 24, The control device placing the portion that functions as the marker by the robot; The imaging device captures an image of the portion that functions as the marker. A robot system characterized by:

[0107] (Item 26) 26. The robot system according to any one of items 1 to 25, The control device The portion functioning as the marker is placed in a grid pattern. A robot system characterized by:

[0108] (Item 27) In the robot system according to item 1, The control device acquiring, by the robot, information regarding the position of the portion functioning as a marker in a pre-placed state, and information regarding the position of the portion functioning as a marker acquired by the imaging device; A robot system characterized by:

[0109] (Item 28) In the robot system according to item 1, The portion functioning as the marker is a groove. A robot system characterized by:

[0110] (Item 29) In the robot system according to item 1, The portion functioning as the marker is a reference portion provided in an area where the robot works. A robot system characterized by:

[0111] (Item 30) In the robot system according to item 1, the portion functioning as a marker is an impression imprinted by a stamp controlled by the robot; A robot system characterized by:

[0112] (Item 31) In the robot system according to item 1, the portion functioning as a marker is an engraving made by a stamp controlled by the robot; A robot system characterized by:

[0113] (Item 32) In the robot system according to item 1, The portion functioning as the marker is printed on paper. A robot system characterized by:

[0114] (Item 33) Robots and An imaging device; A robot system including a control device, The control device Acquiring information about a reference position when controlling the robot, and information about the position of the placed part functioning as a marker, which is acquired by the imaging device, by placing a part functioning as a marker by the robot. A robot system characterized by:

[0115] (Item 34) 34. A method for manufacturing an article, comprising the steps of: manufacturing an article using the robot system according to any one of items 1 to 33.

[0116] (Item 35) Robots and An imaging device; A control method for a robot system including a control device, The control device acquiring information regarding the position of the portion functioning as a marker acquired by the robot and information regarding the position of the portion functioning as a marker acquired by the imaging device; A control method comprising:

[0117] (Item 36) A robot controlled by an imaging device, acquiring information regarding the position of the portion functioning as a marker acquired by the robot and information regarding the position of the portion functioning as a marker acquired by the imaging device; A robot characterized by:

[0118] (Item 37) A method for controlling a robot controlled by an imaging device, comprising: acquiring information regarding the position of the portion functioning as a marker acquired by the robot and information regarding the position of the portion functioning as a marker acquired by the imaging device; A control method comprising:

[0119] (Item 38) An information processing device that acquires information about a robot controlled by an imaging device, acquiring information regarding the position of the portion functioning as a marker acquired by the robot and information regarding the position of the portion functioning as a marker acquired by the imaging device; 1. An information processing device comprising:

[0120] (Item 39) An information processing method for acquiring information about a robot controlled by an imaging device, comprising: acquiring information regarding the position of the portion functioning as a marker acquired by the robot and information regarding the position of the portion functioning as a marker acquired by the imaging device; 1. An information processing method comprising:

[0121] (Item 40) A program capable of executing the control method according to item 35 or 37, or the information processing method according to item 39.

[0122] (Item 41) A computer-readable recording medium storing the program described in item 40. [Explanation of symbols]

[0123] 1 marker 2 End Effector 3. Imaging device 4. Work Area 5. Robot 10 Marker gripping part 100 control device 200 PLC 400 Stand 401 Assembly work 402 Workpiece to be assembled 410 Materials 501 Reference workpiece 502 Parallel movement destination workpiece 503 Reference Image 504 misaligned images 600 GUI 601 Placement point specification window 602 Mouse Cursor 603 Placement position 604 Work area coordinate origin input window 605 Work area range specification window 606 Placement position indicator 607 Arrow 608 Execute button 700 Correction Value Table 1000 Robot System

Claims

1. Robots and An imaging device; A robot system including a control device, The control device acquiring information regarding the position of the portion functioning as a marker acquired by the robot and information regarding the position of the portion functioning as a marker acquired by the imaging device; A robot system characterized by:

2. 2. The robot system according to claim 1, acquiring information about a position of a predetermined part of the robot when the part functioning as the marker is placed by the robot as information about the position of the part functioning as the marker; A robot system characterized by:

3. 3. The robot system according to claim 2, The control device acquiring information regarding the position of the predetermined part of the robot when the robot releases the part that functions as the marker; A robot system characterized by:

4. 3. The robot system according to claim 2, The control device When the robot places the portion that functions as the marker, information regarding the position of the predetermined portion of the robot is acquired when it is determined that the robot has settled before releasing the portion that functions as the marker. A robot system characterized by:

5. 3. The robot system according to claim 2, The control device acquiring information about the position of the predetermined part of the robot in a state where the part functioning as a marker is brought into contact with a place where the part functioning as a marker is to be placed; A robot system characterized by:

6. 6. The robot system according to claim 5, The control device acquiring information about the position of the predetermined part of the robot in a state where the part functioning as the marker is in contact with an elastic material; A robot system characterized by:

7. 6. The robot system according to claim 5, The control device and acquiring information about the position of the predetermined part of the robot when information about a force acquired when the part functioning as a marker is brought into contact with a place where the part functioning as a marker is to be placed is in a predetermined state. A robot system characterized by:

8. 3. The robot system according to claim 2, The control device placing the portion functioning as a marker at at least two positions, and acquiring information on the position of the predetermined part of the robot when the portion functioning as a marker is placed by the robot, and information on the position of the placed portion functioning as a marker acquired by the imaging device. A robot system characterized by:

9. 2. The robot system according to claim 1, The position where the portion functioning as the marker is placed can be set by the user. A robot system characterized by:

10. The robot system according to claim 9, The area in which the robot works can be set by the user. A robot system characterized by:

11. The robot system according to claim 9, notifying a user of the execution order at the position where the portion functioning as a marker is placed; A robot system characterized by:

12. The robot system according to claim 11, When a user deletes a position where the portion functioning as a marker is to be placed, the execution order is updated. A robot system characterized by:

13. The robot system according to claim 11, notifying the user of the execution order by arrows or numbers; A robot system characterized by:

14. The robot system according to claim 9, When the user places a cursor in an area where the user is prompted to input a position where the portion functioning as a marker is to be placed, the user is notified of information regarding the position of the cursor in the area. A robot system characterized by:

15. 2. The robot system according to claim 1, The control device Placing the marker two-dimensionally or three-dimensionally; A robot system characterized by:

16. 3. The robot system according to claim 2, The control device acquiring information indicating a correction value for correcting a command value for controlling the robot based on information regarding the position of the predetermined portion of the robot and information regarding the position of the placed portion that functions as the marker, the information being acquired by the imaging device; A robot system characterized by:

17. 17. The robot system according to claim 16, The control device acquiring information indicating the correction value for correcting the command value based on information regarding the position of the predetermined portion of the robot, information regarding the position of the placed portion functioning as the marker acquired by the imaging device, and a reference position when controlling the robot; A robot system characterized by:

18. 17. The robot system according to claim 16, The information indicating the correction value is in the form of at least one of a map, a function, and a table. A robot system characterized by:

19. 17. The robot system according to claim 16, The control device acquiring the correction value corresponding to the command value based on information indicating the command value and the correction value, and controlling the robot; A robot system characterized by:

20. 17. The robot system according to claim 16, The control device obtaining a weighted average of the differences obtained at surrounding positions where the portions functioning as markers are placed, for positions other than the position where the portion functioning as the marker is placed, in the information indicating the correction value; A robot system characterized by:

21. 17. The robot system according to claim 16, The control device The information indicating the correction value is visualized using an arrow that expresses the amount and direction by a size and a direction, respectively, starting from the position where the portion functioning as the marker is placed. A robot system characterized by:

22. 2. The robot system according to claim 1, The imaging device is provided at a position overlooking an area where the robot works, or is provided on the robot, or is a stereo camera. A robot system characterized by:

23. 3. The robot system according to claim 2, The control device acquiring information about the position of the predetermined part of the robot using an encoder mounted on the robot; A robot system characterized by:

24. 3. The robot system according to claim 2, The predetermined portion is an end effector or a flange to which the end effector is attached. A robot system characterized by:

25. 2. The robot system according to claim 1, The control device placing the portion that functions as the marker by the robot; The imaging device captures an image of the portion that functions as the marker. A robot system characterized by:

26. 2. The robot system according to claim 1, The control device The portion functioning as the marker is placed in a grid pattern. A robot system characterized by:

27. 2. The robot system according to claim 1, The control device The robot acquires information about the position of the portion functioning as a marker in a pre-placed state, and information about the position of the portion functioning as a marker acquired by the imaging device. A robot system characterized by:

28. 2. The robot system according to claim 1, The portion functioning as the marker is a groove. A robot system characterized by:

29. 2. The robot system according to claim 1, The portion functioning as the marker is a reference portion provided in an area where the robot works. A robot system characterized by:

30. 2. The robot system according to claim 1, the portion functioning as a marker is an impression imprinted by a stamp controlled by the robot; A robot system characterized by:

31. 2. The robot system according to claim 1, the portion functioning as a marker is an engraving made by a stamp controlled by the robot; A robot system characterized by:

32. 2. The robot system according to claim 1, The portion functioning as the marker is printed on paper. A robot system characterized by:

33. Robots and An imaging device; A robot system including a control device, The control device Acquiring information about a reference position when controlling the robot, and information about the position of the placed part functioning as a marker, which is acquired by the imaging device, by placing a part functioning as a marker by the robot. A robot system characterized by:

34. A method for manufacturing an article, comprising the steps of: manufacturing an article using the robot system according to any one of claims 1 to 33;

35. Robots and An imaging device; A control method for a robot system including a control device, The control device acquiring information regarding the position of the portion functioning as a marker acquired by the robot and information regarding the position of the portion functioning as a marker acquired by the imaging device; A control method comprising:

36. A robot controlled by an imaging device, acquiring information regarding the position of the portion functioning as a marker acquired by the robot and information regarding the position of the portion functioning as a marker acquired by the imaging device; A robot characterized by:

37. A method for controlling a robot controlled by an imaging device, comprising: acquiring information regarding the position of the portion functioning as a marker acquired by the robot and information regarding the position of the portion functioning as a marker acquired by the imaging device; A control method comprising:

38. An information processing device that acquires information about a robot controlled by an imaging device, acquiring information regarding the position of the portion functioning as a marker acquired by the robot and information regarding the position of the portion functioning as a marker acquired by the imaging device; 1. An information processing device comprising:

39. An information processing method for acquiring information about a robot controlled by an imaging device, comprising: acquiring information regarding the position of the portion functioning as a marker acquired by the robot and information regarding the position of the portion functioning as a marker acquired by the imaging device; 1. An information processing method comprising:

40. A program capable of executing the control method according to claim 35 or 37, or the information processing method according to claim 39.

41. A computer-readable recording medium storing the program according to claim 40.

Images