ROBOT SYSTEM AND ROBOT CONTROL METHOD

Abstract

To provide a robot control system that facilitates corrections which are fit to a plurality of robots.SOLUTION: A plurality of robots are arranged to be able to simultaneously work on the same work-piece. Robot controllers are arranged to correspond to the robots respectively. Each of a plurality of visual sensors can obtain visual information about an object to be measured that the work-piece has. The plurality of robot controllers and the plurality of visual sensors respectively can communicate with a visual processing computer in common. The visual processing computer computes a coordinate system of the work-piece, on the basis of a measured result of the object to be measured appearing in the visual information obtained by the plurality of visual sensors. The robot controllers correct motion of the robots corresponding to the robot controllers, on the basis of a required result to the visual processing computer.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to the use of visual sensors in robots. [Background technology]

[0002] When a robot is to perform a specific operation, a program is created in advance by offline teaching work, or a program is created by simulation. The positional relationship between the robot and a workpiece, etc., when offline or simulated may not match the positional relationship when the robot is actually operated. In order to measure and correct this error, a system such as that disclosed in Patent Document 1 has been proposed.

[0003] Patent Document 1 discloses a configuration in which, when there are multiple pairs of one robot and one robot control device, image data from one camera can be shared by the multiple robot control devices. Only one imaging unit (camera) is used for multiple robots (robot control devices). The camera can be attached to the tip of the arm of each robot.

[0004] The camera is replaced each time the combination of robot and measurement object is changed. Specifically, first, the camera is attached to the first robot, and the 3D relative position is measured for the combination of the first robot and measurement object. Next, the camera that was attached to the first robot is attached to the second robot, and the 3D relative position is measured for the combination of the second robot and measurement object.

[0005] The system of Patent Document 1 includes an image processing device. This image processing device performs image processing to extract, for example, three points from an image captured by a camera. Patent Document 1 also discloses an example in which the image processing device is placed outside the robot control device and incorporated into a network. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent No. 4137862 Summary of the Invention [Problem to be solved by the invention]

[0007] In the configuration of Patent Document 1, multiple robots take images of the measurement target while passing one camera to another. This increases the cycle time for correction and requires man-hours to program the robots with the operations for passing the camera to another.

[0008] In Patent Document 1, three points are extracted from an image captured by a camera in each robot, and three-dimensional relative positions are measured. Therefore, it is difficult to achieve consistency in position correction among multiple robots.

[0009] The present invention has been made in consideration of the above circumstances, and an object of the present invention is to provide a robot control system that can easily perform correction for consistency among a plurality of robots. [Means for solving the problem]

[0010] The problem to be solved by the present invention has been described above. Next, the means for solving this problem and the effects thereof will be described.

[0011] According to a first aspect of the present invention, there is provided a robot system having the following configuration. That is, the robot system includes a plurality of robots, a plurality of robot controllers, a plurality of visual sensors, and a visual processing computer. The robots are arranged so that they can work on the same workpiece at the same time. The robot controllers are arranged corresponding to the respective robots. The visual sensor is capable of acquiring visual information. The visual processing computer performs processing based on the information acquired by the visual sensor. Each of the plurality of visual sensors is capable of acquiring visual information on a measurement target of the workpiece. The plurality of robot controllers and the plurality of visual sensors are all capable of communicating with the common visual processing computer. The visual processing computer calculates a workpiece coordinate system based on the result of measuring the measurement target appearing in the visual information acquired by the plurality of visual sensors. Each of the robot controllers corrects the operation of the robot corresponding to the robot controller based on the result of a request made to the visual processing computer. The visual processing computer stores information on the workpiece coordinate system for each of the robots. The visual processing computer initializes the information on the robot corresponding to the robot controller in response to an initialization command transmitted by the robot controller. The robot controller of the robot belonging to a group of the robots to which the workpiece coordinate system is applied transmits an inquiry to the visual processing computer at an irregular timing as to whether or not initialization of information on the workpiece coordinate system has been completed for all of the robots belonging to the group. When the visual processing computer receives an inquiry from the robot controller corresponding to a robot belonging to the group in a state where there is a robot belonging to the group for which initialization of information on the workpiece coordinate system has not been completed, the visual processing computer transmits a standby command to the robot controller that originated the inquiry as a response to the inquiry. When the visual processing computer receives an inquiry from the robot controller corresponding to a robot belonging to the group in a state where initialization of information on the workpiece coordinate system has been completed for all of the robots belonging to the group, the visual processing computer transmits a completion signal, which is a signal indicating that initialization of information has been completed for all of the robots belonging to the group, to the robot controller that originated the inquiry as a response to the inquiry. Each of the robot controllers, upon receiving a standby command, queries the vision processing computer again, but upon receiving a completion signal, does not query the vision processing computer again.

[0012] According to a second aspect of the present invention, there is provided the following robot control method. That is, this robot control method is targeted at a robot system including a plurality of robots, a plurality of robot controllers, a plurality of visual sensors, and a visual processing computer. The robots are arranged so that they can work on the same workpiece simultaneously. The robot controllers are arranged corresponding to the respective robots. The visual sensor is capable of acquiring visual information. The visual processing computer performs processing based on the information acquired by the visual sensor. The robot control method includes a first step, a second step, a third step, and A fourth step, a fifth step, a sixth step, a seventh step, an eighth step, and a ninth step,In the first step, each of the plurality of visual sensors acquires visual information about a measurement target of the workpiece and transmits the visual information to the common visual processing computer. In the second step, the visual processing computer calculates a workpiece coordinate system based on a result of measuring the measurement target appearing in the visual information received from the plurality of visual sensors. In the third step, each of the plurality of robot controllers transmits a request to the common visual processing computer and corrects the operation of the robot corresponding to the robot controller based on a response received from the visual processing computer. In the fourth step, the visual processing computer stores information on the workpiece coordinate system for each of the plurality of robots. In the fifth step, the visual processing computer initializes the information on the robot corresponding to the robot controller in response to an initialization command sent by the robot controller. In the sixth step, each of the robot controllers of the robots belonging to a group of the plurality of robots to which the workpiece coordinate system is applied sends an inquiry to the visual processing computer at an irregular timing as to whether or not initialization of information on the workpiece coordinate system has been completed for all of the robots belonging to the group. In the seventh step, when the visual processing computer receives an inquiry from the robot controller corresponding to a robot belonging to the group while there is a robot belonging to the group for which initialization of information on the workpiece coordinate system has not been completed, the visual processing computer sends a standby command to the robot controller that originated the inquiry as a response to the inquiry. In the eighth step, when the visual processing computer receives an inquiry from a robot controller corresponding to a robot belonging to the group in a state in which initialization of information relating to the work coordinate system for all of the robots belonging to the group has been completed, the visual processing computer transmits a completion signal, which is a signal indicating that initialization of information for all of the robots belonging to the group, as a response to the inquiry to the robot controller that originated the inquiry. In the ninth step, when each of the multiple robot controllers receives a standby command, it again queries the visual processing computer, but when it receives a completion signal, it does not again query the visual processing computer.

[0013] As a result, one or a small number of visual processing computers acquire information from multiple visual sensors and calculate the work coordinate system, so information aggregation can be naturally realized. Therefore, it is suitable for aggregating information from multiple visual sensors to calculate the work coordinate system. Since this work coordinate system is commonly used for the operation correction of multiple robots, the operation correction can be consistent between the multiple robots. In addition, since the server device centrally performs processing corresponding to requests from multiple robot controllers, it is easy to simplify processing and communication. Furthermore, the visual processing computer can realize an interlock control that does not allow the robots to proceed to the next process unless the information of all the robots is initialized. Also, inquiries about the completion of the information initialization process by each robot controller can be made at irregular times. Effect of the Invention

[0014] According to the present invention, it is possible to easily perform corrections that are consistent among a plurality of robots. [Brief description of the drawings]

[0015] [Figure 1] 1 is a schematic diagram showing an overall configuration of a robot system according to a first embodiment of the present invention. [Diagram 2] 11 is a flowchart illustrating an interlock control for initialization. [Diagram 3] 11 is a flowchart illustrating a process of measuring a mark based on an image from a camera. [Figure 4] 11 is a flowchart illustrating an interlock control regarding acquisition of a workpiece coordinate system. [Diagram 5] FIG. 11 is a schematic diagram of a robot system according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Next, an embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a schematic diagram showing an overall configuration of a robot system 100 according to a first embodiment of the present invention.

[0017] 1 is a system that performs operations on a workpiece 1 using multiple robots 11, 12, 13, and 14. The robot system 100 is installed, for example, on a production line in a factory.

[0018] The robot system 100 includes robots 11, 12, 13, and 14, robot controllers 21, 22, 23, and 24, cameras (visual sensors) 31, 32, and 33, and a server device 41 (visual processing computer).

[0019] The robot 11 is an industrial robot. The robot 11 is configured as a vertical articulated robot with six degrees of freedom of movement. However, the configuration of the robot is not limited to this.

[0020] The workpiece 1 transported by the conveyor 2 stops at a predetermined position surrounded by four robots 11, 12, 13, and 14. The four robots 11, 12, 13, and 14 can simultaneously perform work on the workpiece 1. The work performed by the robots is arbitrary, and examples of such work include assembly, welding, and painting. Instead of the conveyor 2, another device capable of moving the workpiece 1 (for example, a supply device) can be used.

[0021] The four robots 11, 12, 13, 14 may have the same or different configurations. The four robots 11, 12, 13, 14 may perform the same or different tasks on the workpiece 1. The number of robots simultaneously performing tasks on the workpiece 1 is not limited to four, and may be two, three, five or more.

[0022] The four robot controllers 21, 22, 23, and 24 are disposed corresponding to the four robots 11, 12, 13, and 14. Each of the robot controllers 21, 22, 23, and 24 is configured as a computer including a CPU, a ROM, a RAM, and the like.

[0023] The robot 11 is electrically connected to a robot controller 21. The robot 11 operates based on commands from the robot controller 21, and can perform a predetermined task on the workpiece 1. The robot 12 operates based on commands from a robot controller 22. The robot 13 operates based on commands from a robot controller 23. The robot 14 operates based on commands from a robot controller 24. In this way, one robot is controlled by one robot controller.

[0024] Each of the robot controllers 21, 22, 23, and 24 is electrically connected to a server device 41. Each of the robot controllers 21, 22, 23, and 24 can transmit and receive information through communication.

[0025] The cameras 31, 32, and 33 are imaging devices capable of acquiring images (visual information). Each of the three cameras 31, 32, and 33 is fixed at an appropriate position where it can capture an image of the workpiece 1 in the above-mentioned predetermined position. The number of cameras can be one, two, or four or more. Some or all of the multiple cameras may be configured as cameras that can be held by a robot instead of fixed cameras. In this case, the robot is operated so that the camera is at the predetermined shooting position, and then shooting is performed.

[0026] The camera 31 can photograph the mark M1 affixed to the workpiece 1. The camera 32 can photograph the mark M2 affixed to the workpiece 1. The camera 33 can photograph the mark M3 affixed to the workpiece 1. The marks (measurement objects) M1, M2, and M3 can be, for example, stickers affixed to the workpiece 1. However, instead of the marks M1, M2, and M3, the cameras 31, 32, and 33 can also photograph, as the measurement objects, features of the shape of the workpiece 1 (for example, convex portions, concave portions, etc.).

[0027] Depending on the conditions, it may be possible to simultaneously photograph the three marks M1, M2, and M3 with one camera. However, depending on the size and shape of the work 1 and the positions of the marks M1, M2, and M3, it may not be possible to include the three marks M1, M2, and M3 in the field of view of one camera. In the configuration of this embodiment, the three marks M1, M2, and M3 are photographed by the three cameras 31, 32, and 33 in a shared manner, so that each of the marks M1, M2, and M3 can be easily photographed.

[0028] The camera 31 is disposed near the robot 11 and is associated with the robot controller 21. The camera 32 is disposed near the robot 12 and is associated with the robot controller 22. The camera 33 is disposed near the robot 13 and is associated with the robot controller 23. In this manner, each of the cameras 31, 32, and 33 belongs to one of the four robots 11, 12, 13, and 14.

[0029] Each of the cameras 31, 32, and 33 is electrically connected to the server device 41. Each of the cameras 31, 32, and 33 can output captured images to the server device 41 through communication.

[0030] The communication between the server device 41 and the robot controllers 21, 22, 23, and 24, and the communication between the server device 41 and the cameras 31, 32, and 33 can be performed using, for example, a known LAN. However, the communication method is not limited to the LAN.

[0031] The server device 41 is configured as a computer equipped with a CPU, a ROM, a RAM, etc. If the server device 41 is equipped with a CPU with higher performance than the above-mentioned robot controller, or if it is equipped with special hardware such as a GPU, it is preferable that the server device 41 can perform image processing described later at high speed.

[0032] The server device 41 performs known image processing (visual processing) on ​​the image data obtained from the multiple cameras 31, 32, 33 to identify the positions of the marks M1, M2, M3 in three-dimensional space. This makes it possible to obtain a three-dimensional orthogonal coordinate system that expresses the actual position and orientation of the workpiece 1. The coordinate system of the workpiece 1 refers to a coordinate system set on the workpiece 1, and changes in conjunction with the movement / rotation of the workpiece 1. Hereinafter, this coordinate system may be referred to as the workpiece coordinate system.

[0033] In the server device 41, a three-dimensional orthogonal coordinate system (hereinafter referred to as a common coordinate system) that is common to the robots 11, 12, 13, and 14 and is immovable with respect to the ground is appropriately defined. In this embodiment, any one of the four robots 11, 12, 13, and 14 is determined as a master robot. The common coordinate system can be, for example, a three-dimensional orthogonal coordinate system (master coordinate system) set in the master robot. The coordinate system set in the robot is, for example, a coordinate system set based on an immovable base part located at the base of the arm part of the robot.

[0034] The workpiece coordinate system can be expressed as the coordinates of its origin expressed in a common coordinate system, and the directions of the three coordinate axes expressed in the common coordinate system. In this specification, the workpiece coordinate system may be referred to as a frame.

[0035] A specific description will be given below. For each of the three cameras 31, 32, and 33, a camera coordinate system corresponding to the actual installation position and posture is determined. In the server device 41, for each camera, a camera coordinate system viewed in a coordinate system specific to the robot to which the camera belongs is stored in advance. The coordinate system specific to each robot will be described later. The server device 41 acquires the position of the mark in the image captured by each camera in the form of two-dimensional orthogonal coordinates by using an appropriate method such as pattern recognition. In three-dimensional space, the position of the mark in the image corresponds to the direction of the mark relative to the camera that captured it. In addition, in the work 1, the relative positional relationship between the three marks M1, M2, and M3 is known. The server device 41 converts the direction of the mark relative to the camera into a common coordinate system (master coordinate system), and then solves the simultaneous equations created based on the above information to obtain the positions of the three marks M1, M2, and M3 in the common coordinate system. Once the positions of the three marks M1, M2, and M3 are determined, the work coordinate system can be uniquely identified.

[0036] In this embodiment, the cameras 31, 32, and 33 are all configured as two-dimensional cameras. However, one or more cameras can be changed to three-dimensional cameras. The configuration of the three-dimensional camera is arbitrary, and can be, for example, a stereo system, a ToF system, or the like. The positions of the marks M1, M2, and M3 can be obtained in three dimensions based on the shooting results of the three-dimensional camera. The positions of the marks are expressed in a three-dimensional coordinate system (camera coordinate system) owned by the three-dimensional camera that shot them. The server device 41 converts the positions of the marks shot by the three-dimensional camera from the camera coordinate system to a common coordinate system, and then solves simultaneous equations to obtain the positions of the three marks M1, M2, and M3 in the common coordinate system. When a visual sensor capable of three-dimensional measurement is used in this way, the three-dimensional shape characteristics of the work 1 can also be measured using known three-dimensional shape recognition instead of the marks M1, M2, and M3.

[0037] The server device 41 returns the obtained workpiece coordinate system in response to a request from the robot controllers 21, 22, 23, 24. The robot controllers 21, 22, 23, 24 acquire the returned workpiece coordinate system as a correction coordinate system. This allows the robots 11, 12, 13, 14 to recognize the actual position and posture of the workpiece 1 and correct their operations accordingly. As a result, it is possible to flexibly respond to deviations in the position and posture of the workpiece 1.

[0038] The server device 41 returns the workpiece coordinate system obtained by integrating and calculating the imaging results of the three cameras 31, 32, and 33 to all the robots (robot controllers) in a uniform manner. Therefore, the corrections of the operations of the four robots 11, 12, 13, and 14 are consistent with each other. In other words, well-coordinated corrections can be made for the four robots 11, 12, 13, and 14. Therefore, this is suitable for the case where one workpiece 1 is worked on simultaneously.

[0039] Among the four robots 11, 12, 13, and 14, each robot other than the master robot also has its own unique coordinate system defined in the same manner as the master coordinate system. In the server device 41, the coordinate system of each robot as viewed from the common coordinate system (master coordinate system) is stored in advance.

[0040] When the server device 41 returns a workpiece coordinate system to a robot other than the master robot, the server device 41 may convert the workpiece coordinate system into a form seen in a coordinate system specific to the robot (robot controller) that made the inquiry, rather than a common coordinate system, and return the conversion result. The above coordinate conversion can be easily performed by using information on the robot coordinate system. For example, if the robot 13 is the master robot, and a vector V pointing from the origin of the master coordinate system, which is a common coordinate system, to the origin of the coordinate system specific to the robot 11 is ms is known. The vector indicating the position of workpiece 1 in the master coordinate system is V mes In this case, a vector V indicating the position of the workpiece 1 in the coordinate system of the robot 11 is mes ' can be expressed by the following formula: V mes '=-V ms +V mes When the coordinate conversion is performed on the robot controller side, the coordinate conversion on the server device 41 side is not necessary.

[0041] However, when the workpiece coordinate system is not available in the server device 41, this means that the robots are unable to obtain information that serves as a reference for correcting their operations. Therefore, in this state, it is not preferable for any of the four robots 11, 12, 13, 14 to perform work on the workpiece 1. In this regard, in this embodiment, interlock control is realized by communicating between the robot controllers 21, 22, 23, 24 and the server device 41, and the robots 11, 12, 13, 14 are configured to wait until the workpiece coordinate system is available.

[0042] The server device 41 stores logs related to operations and various settings. An operator can operate the server device 41 to view logs, change settings, and the like.

[0043] The server device 41 is configured to be able to communicate with the setting computer 51. The communication can be performed using, for example, a known LAN or WAN. The setting computer 51 is located at a location physically separated from the server device 41. The setting computer 51 is configured as a known computer. A remote worker can access the server device 41 by operating the setting computer 51, thereby viewing the logs recorded in the server device 41 and changing the settings related to the creation of the work coordinate system.

[0044] Next, some interlock controls performed in the robot system 100 will be described.

[0045] First, the interlock control of the initialization process will be described with reference to Fig. 2. The processes performed by the four robot controllers 21, 22, 23, and 24 are substantially similar, so here, the process performed by one of them, the robot controller 21, will be described. The left side of Fig. 2 shows the process performed by the robot controller 21, and the right side shows the process performed by the server device 41.

[0046] The left side of Fig. 2 shows the processing performed by the robot controller 21. This processing is started by some kind of trigger in the robot controller 21. The right side of Fig. 2 shows the processing performed by the server device 41.

[0047] First, the robot controller 21 transmits an initialization command to the server device 41 (step S101). When the server device 41 receives this command (step S102), it erases the work coordinate system conversion results and other data stored for the robot controller 21 (which essentially means the robot 11) that transmitted the command (step S103). When the initialization process is completed, the server device 41 transmits a message indicating that initialization is complete to the robot controller 21 that transmitted the command (step S104). The robot controller 21 receives the message indicating that initialization is complete from the server device 41 (step S105).

[0048] Although the above has focused on one robot controller 21 and explained the processing of steps S101 to S105, similar processing is performed in the other robot controllers 22, 23, and 24. Since the start timing of the processing in Fig. 2 in the four robot controllers 21, 22, 23, and 24 is undefined, the timing of sending the initialization command to the server device 41 is also undefined. The server device 41 performs initialization related to the robot at the timing of receiving the initialization command.

[0049] When the robot controller 21 receives the notification of the completion of the initialization from the server device 41, it transmits an inquiry to the server device 41 as to whether or not the initialization has been completed for all the robots 11, 12, 13, and 14 (step S106). The server device 41 receives this inquiry from the robot controller 21 (step S107).

[0050] When the server device 41 receives the inquiry, it judges whether the initialization process of step S103 has been completed for all of the four robots 11, 12, 13, and 14 (step S108). If the initialization has been completed for all the robots, the server device 41 sends a reply of "ok" to the robot controller 21 that originated the inquiry (step S109). If there is a robot for which initialization has not been completed, the server device 41 sends a reply of "wait" (step S110). "Wait" means that waiting is required. Therefore, the reply "wait" can be considered as a kind of wait command from the server device 41 to the robot controller 21 that originated the inquiry.

[0051] Depending on circumstances, a situation may be considered in which, for example, only three of the four robots perform work on the workpiece 1. Possible reasons for such degenerate operation include, but are not limited to, an abnormality occurring in some of the robots. When a setting is made to exclude some of the four robots from the interlock control, in step S108, the server device 41 determines whether or not all initialization has been completed, targeting only the active robots.

[0052] When the robot controller 21 receives a response to the inquiry from the server device 41 (step S111), it judges the content of the response (step S112). If the content of the response is "wait", the process returns to step S106, and the robot controller 21 again transmits the inquiry to the server device 41. If the content of the response is "ok", the series of processes for initialization ends. On the condition that the process shown in FIG. 2 is completed, the robot controller 21 can start the next process.

[0053] Although the above describes the processing of steps S106 to S112 focusing on one robot controller 21, the other robot controllers 22, 23, and 24 also perform similar processing. When the server device 41 receives an inquiry from any of the four robot controllers 21, 22, 23, and 24, if there is a robot for which the initialization of step S103 has not been completed, the server device 41 responds with "wait." The processing shown on the left side of FIG. 2 does not end normally unless the response from the server device 41 is "ok." Therefore, unless the initialization of all the robots 11, 12, 13, and 14 is completed in the server device 41, the processing of FIG. 2 will not be completed in any of the robot controllers 21, 22, 23, and 24. In this manner, interlock control regarding the initialization processing is realized.

[0054] Next, the measurement process of the marks M1, M2, and M3, which is a prerequisite for acquiring the workpiece coordinate system, will be described with reference to Fig. 3. This process is performed with the participation of three robot controllers 21, 22, and 23 associated with cameras 31, 32, and 33. Since the processes performed by the three robot controllers 21, 22, and 23 are substantially the same, the process performed by one of them, the robot controller 21, will be described here. As described above, instead of the marks M1, M2, and M3, appropriate visual features of the workpiece 1 may be measured.

[0055] The left side of Fig. 3 shows the processing performed by the robot controller 21. When a sensor or the like detects that the workpiece 1 has been transported to a predetermined position, this processing is started in the robot controller 21. The right side of Fig. 3 shows the processing performed by the server device 41.

[0056] First, the robot controller 21 transmits a measurement command to the server device 41 (step S201). The server device 41 receives this command (step S202). The server device 41 immediately transmits an image capture command to the camera 31 associated with the robot controller 21 (actually, the robot 11) that transmitted the command, and obtains the position of the mark M1 from the captured image by pattern recognition or the like (step S203). The position of the mark M1 here means the two-dimensional orthogonal coordinates in the image. The process in which the camera 31 transmits the image to the server device 41 in step S203 corresponds to the first process.

[0057] Next, the server device 41 judges whether the measurement of the position of the mark M1 has been completed normally (step S204). If the measurement has been completed normally, the server device 41 converts the position of the mark M1 from the above-mentioned two-dimensional orthogonal coordinate system into a three-dimensional orthogonal coordinate system specific to the robot 11, and if the robot 11 is not a master robot, further converts it into the master coordinate system and stores it (step S205). The conversion between the master coordinate system and the coordinate systems of other robots will be described later. Thereafter, the server device 41 transmits a message indicating that the measurement has been completed normally to the robot controller 21, which is the source of the command (step S206).

[0058] If the position of the mark M1 cannot be acquired due to a failure in pattern recognition or the like, the server device 41 transmits a message to the robot controller 21 indicating that the measurement has ended abnormally (step S207).

[0059] As described above, the camera 31 may be a three-dimensional camera. When the camera 31 is a three-dimensional camera, the coordinates of the mark M1 obtained by the server device 41 in step S203 are three-dimensional coordinates in a three-dimensional coordinate system (camera coordinate system, visual sensor coordinate system) owned by the camera 31. In step S205, the three-dimensional coordinates are converted into a three-dimensional orthogonal coordinate system specific to the robot 11, and further converted into a master coordinate system as necessary. The step in which the camera 31 transmits, for example, three-dimensional point cloud data to the server device 41 in step S203 corresponds to the first step.

[0060] When the robot controller 21 receives a response to the measurement command from the server device 41 (step S208), it judges the measurement result contained in the response (step S209). If the measurement result is normal, the robot controller 21 records in a log or the like that the position of the mark M1 has been measured normally (step S210). If the measurement result is abnormal, the robot controller 21 records in a log or the like that an abnormality has occurred (step S211). In either case, the series of processes ends.

[0061] Although the above has focused on one robot controller 21 and explained the processing of steps S201 to S211, similar processing is performed for the other robot controllers 22 and 23. As described above, the positions of the three marks M1, M2, and M3 can be measured from the images captured by the three cameras 31, 32, and 33, respectively, and the measurement results can be stored on the server device 41 side. The timing at which each of the robot controllers 21, 22, and 23 transmits a measurement command to the server device 41 may be arbitrary. At the timing at which the server device 41 receives the measurement command, it measures the position of the mark from the image captured by the camera associated with the robot controller that transmitted it.

[0062] Next, the interlock control regarding the creation of the work coordinate system will be described with reference to Fig. 4. This process is performed by four robot controllers 21, 22, 23, and 24. Since the processes performed by the four robot controllers 21, 22, 23, and 24 are substantially the same, only the process performed by one of them, the robot controller 21, will be described here. For the sake of simplicity, the work coordinate system is represented as a frame in Fig. 4.

[0063] The left side of Fig. 4 shows the processing performed by the robot controller 21. In the robot controller 21, this processing is started by some kind of trigger. The trigger is, for example, but not limited to, the completion of the measurement processing of the mark M1 described in Fig. 3. The right side of Fig. 4 shows the processing performed by the server device 41.

[0064] First, the robot controller 21 transmits a request for creating a workpiece coordinate system to the server device 41 (step S301). The server device 41 receives this request from the robot controller 21 (step S302).

[0065] When the server device 41 receives the request, it judges whether or not the measurement process described in step S203 of Fig. 3 has been completed for all of the three marks M1, M2, and M3 (step S303). In other words, this judgment means whether or not the measurement process has been completed for all of the robot controllers 21, 22, and 23, and whether or not the measurement process has been completed for all of the three cameras 31, 32, and 33.

[0066] If all the measurement processes have been completed, the server device 41 determines whether or not all the measurement processes have been completed normally (step S304).

[0067] If all the measurement processes have been completed normally, the server device 41 acquires the workpiece coordinate system by calculation using the positions of the marks M1, M2, and M3 stored in step S205 (step S305, second process). After that, the server device 41 calculates the accuracy of the created workpiece coordinate system by calculation, and judges whether or not this accuracy is equal to or higher than a predetermined level (step S306). This accuracy is the accuracy of identifying the position and posture of the workpiece 1. The judgment can be performed, for example, as follows. The server device 41 pre-stores a reference value for each side of a triangle having the three marks M1, M2, and M3 as vertices. If the difference between the reference value and any of the three sides of the triangle is equal to or lower than a predetermined value, the server device 41 judges that the accuracy of the workpiece coordinate system is equal to or higher than a predetermined level. If there is one or more sides whose difference from the reference value exceeds a predetermined value, the server device 41 judges that the accuracy of the workpiece coordinate system is less than a predetermined level. However, the method of judging whether the accuracy is good or not is not limited to the above.

[0068] When it is determined that a predetermined level of accuracy or higher has been obtained for the workpiece coordinate system, the server device 41 stores the obtained workpiece coordinate system (step S307). Before the workpiece coordinate system is stored, it may be converted, as necessary, so that it has a shape seen in a coordinate system specific to the robot controller 21 (actually, the robot 11) that originated the inquiry.

[0069] Thereafter, the server device 41 transmits a response to the request source robot controller 21, notifying the normal end of the creation of the workpiece coordinate system (step S308). The content of this response includes the information on the workpiece coordinate system obtained earlier.

[0070] If it is determined in step S304 that at least one of all the measurement processes has ended abnormally, the server device 41 transmits a response to the request source robot controller 21 to the effect that the creation of the work coordinate system has ended abnormally (step S309). The same applies if the server device 41 determines in step S306 that a predetermined frame accuracy cannot be obtained.

[0071] If it is determined in step S303 that the measurement processes for any of the three marks M1, M2, and M3 have not yet been completed, the server device 41 sends a "wait" as a response to the robot controller 21 that sent the request (step S310).

[0072] When the robot controller 21 receives a response to the request from the server device 41 (step S311), it judges the content of the response (step S312). If the response indicates normal completion, the robot controller 21 records in a log or the like that the work coordinate system was successfully created (step S313). If the response indicates abnormal completion, the robot controller 21 records in a log or the like that an abnormality occurred and the work coordinate system could not be created (step S314). Regardless of whether the response indicates normal completion or abnormal completion, the series of processes in FIG. 4 ends. If the response indicates wait, the process returns to step S301, and the robot controller 21 again sends a request to create a work coordinate system to the server device 41.

[0073] 3, the robot controller 21 transmits a request to create a workpiece coordinate system, and if the response obtained in step S311 indicates normal completion, the robot controller 21 corrects the subsequent operation of the robot 11 based on the obtained workpiece coordinate system so as to match the actual position and posture of the workpiece 1. This series of processes corresponds to the third step in the robot control method.

[0074] Although the above describes the processing of steps S301 to S314 focusing on one robot controller 21, the other robot controllers 22, 23, and 24 also perform similar processing. When the server device 41 receives a request to create a work coordinate system from any of the four robot controllers 21, 22, 23, and 24, the server device 41 responds with "wait" if the conditions for creating the work coordinate system are not satisfied at the time of the command. The processing shown on the left side of FIG. 4 does not end normally unless the response from the server device 41 indicates normal termination. Therefore, the processing of FIG. 4 will not be completed in any of the robot controllers 21, 22, 23, and 24 unless the conditions for creating the work coordinate system are satisfied in the server device 41. In this manner, interlock control regarding the work coordinate system is realized.

[0075] In this embodiment, the server device 41 acquires image data from the three cameras 31, 32, and 33, and measures the marks M1, M2, and M3. Therefore, an inexpensive configuration can be adopted because image processing does not need to be performed on the robot controllers 21, 22, 23, and 24 side. Even if a high-performance CPU is used in the server device 41 to perform image processing in a short time, the increase in cost is limited because there is only one server device 41.

[0076] In the robot system 100 of this embodiment, the server device 41 that performs image processing as described above also calculates a work coordinate system in response to a request for creating a work coordinate system from the multiple robot controllers 21, 22, 23, and 24. Furthermore, if the conditions for calculating the work coordinate system are not satisfied, the server device 41 can realize an interlock by replying "wait" to the request for creating a work coordinate system.

[0077] Next, the range of the robot to which the workpiece coordinate system calculated by the server device 41 is applied will be described.

[0078] In this embodiment, the workpiece coordinate system is shared by four robots 11, 12, 13, and 14 that simultaneously perform one process on a workpiece 1. In other words, it can be considered that one group is formed by the four robots 11, 12, 13, and 14, and one workpiece coordinate system is calculated for this group and commonly used.

[0079] However, the server device 41 may, for example, calculate the workpiece coordinate system for the two robots 11, 12 located on the right side of the conveyor 2, and calculate the workpiece coordinate system for the two robots 13, 14 located on the left side. In this case, it can be considered that a group consisting of two robots is formed on the left and right sides of the conveyor 2, and a workpiece coordinate system is calculated for each group. The above-mentioned initialization interlock control and workpiece coordinate system interlock control can be performed on a group basis.

[0080] As described above, the robot system 100 of this embodiment includes four robots 11, 12, 13, 14, four robot controllers 21, 22, 23, 24, three cameras 31, 32, 33, and a server device 41. The four robots 11, 12, 13, 14 are arranged so that they can work on the same workpiece 1 at the same time. The robot controllers 21, 22, 23, 24 are arranged corresponding to the respective robots 11, 12, 13, 14. The cameras 31, 32, 33 can acquire images. The server device 41 performs processing based on the information acquired by the cameras 31, 32, 33. The camera 31 can acquire an image of the mark M1 of the workpiece 1. Similarly, the camera 32 can acquire an image of the mark M2, and the camera 33 can acquire an image of the mark M3. The four robot controllers 21, 22, 23, 24 and the three cameras 31, 32, 33 can all communicate with a common server device 41. The server device 41 calculates a work coordinate system based on the results of measuring marks M1, M2, M3 that appear in images captured by the three cameras 31, 32, 33. Each robot controller 21, 22, 23, 24 corrects the operation of the robot 11, 12, 13, 14 corresponding to the robot controller based on the results of a request made to the server device 41.

[0081] This allows the server device 41 and the robot controllers 21, 22, 23, 24 to clearly separate the processing targets. Since the robot controllers 21, 22, 23, 24 do not need to perform image processing, low-cost hardware can be used. Since there is only one server device 41 that performs image processing, even if special hardware with enhanced image processing capabilities is used, the increase in cost is limited. Since one server device 41 acquires information from multiple cameras 31, 32, 33 and calculates a work coordinate system, information aggregation can be naturally achieved. Therefore, it is suitable for aggregating information from multiple cameras 31, 32, 33 and calculating a work coordinate system. In addition, since the server device 41 performs processing corresponding to requests from multiple robot controllers 21, 22, 23, 24 in a centralized manner, it is easy to simplify processing and communication.

[0082] In the robot system 100 of this embodiment, when the server device 41 receives a request from, for example, the robot controller 21 while there are cameras 31, 32, 33 that have not yet completed measurement of an object, the server device 41 transmits a "wait" as a response to the request to the requesting robot controller 21. When the robot controller 21 receives a "wait", it makes a request to the server device 41 again.

[0083] This makes it possible to rationally realize interlock control that prevents the robots 11, 12, 13, and 14 from operating unless the workpiece coordinate system is acquired, with the server device 41 that calculates the workpiece coordinate system acting as a contact point for requests.

[0084] In the robot system 100 of this embodiment, the server device 41 stores information on the workpiece coordinate system for each of the robots 11, 12, 13, and 14. The server device 41 initializes information on the robot 11 corresponding to the robot controller 21 in response to an initialization command transmitted by the robot controller 21, for example. The robot controller (e.g., the robot controller 21) of the robot belonging to the group consisting of the robots 11, 12, 13, and 14 to which the workpiece coordinate system is applied transmits an inquiry to the server device 41 as to whether or not the initialization of information on the workpiece coordinate system has been completed for all of the four robots 11, 12, 13, and 14 belonging to the group. When the server device 41 receives an inquiry from a robot controller (e.g., the robot controller 21) corresponding to a robot belonging to the group in a state in which the initialization of information on the workpiece coordinate system has not been completed for a robot belonging to the group, the server device 41 transmits "wait" as a response to the inquiry to the robot controller 21 that originated the inquiry. When the robot controller 21 receives "wait," it makes an inquiry to the server device 41 again.

[0085] This enables the server device 41 to realize an interlock control that does not allow the process to proceed to the next step unless the information of all the robots is initialized.

[0086] In the robot system 100 of this embodiment, the server device 41 can communicate with the setting computer 51. The server device 41 can change settings related to the acquisition of the work coordinate system by transmission from the setting computer 51. There are various possible settings related to the acquisition of the work coordinate system, and one example is settings related to the pattern recognition of the marks M1, M2, and M3.

[0087] This allows remote changes to be made to the settings of the server device 41. This eliminates the need for an operator to travel to the location where the server device 41 is installed, improving convenience.

[0088] Next, the second embodiment will be described. Fig. 5 is a schematic diagram of 100x according to the second embodiment. In the description of this embodiment, the same or similar members as those in the above embodiment are given the same reference numerals in the drawings, and the description may be omitted.

[0089] In a factory where this robot system 100x is applied, a workpiece 1 is subjected to a plurality of work processes as shown in Fig. 5. The server device 41 can calculate a workpiece coordinate system for each work process.

[0090] The workpiece 1 is transported to a first position P1 by the conveyor 2, and then a first work process is performed by four first robots 11, 12, 13, and 14. The workpiece 1 is then transported to a second position P2 by the conveyor 2, and a second work process is performed by four second robots 16, 17, 18, and 19.

[0091] The four robot controllers 21, 22, 23, and 24 are disposed corresponding to the first robots 11, 12, 13, and 14. The four robot controllers 26, 27, 28, and 29 are disposed corresponding to the second robots 16, 17, 18, and 19.

[0092] Four marks M1, M2, M3, and M4 are attached to the workpiece 1. First cameras (first visual sensors) 31, 32, and 33 are installed around the first position P1, and can capture images of the corresponding marks M1, M2, and M3. Second cameras (second visual sensors) 36, 37, and 38 are installed around the second position P2, and can capture images of the corresponding marks M1, M2, and M4.

[0093] The robot controllers 21, 22, 23, 24 of the first robots 11, 12, 13, 14 and the robot controllers 26, 27, 28, 29 of the second robots 16, 17, 18, 19 can all communicate with a server device 41. The first cameras 31, 32, 33 and the second cameras 36, 37, 38 can all communicate with the server device 41.

[0094] The four first robots 11, 12, 13, 14 in charge of the first work process can be considered to form one group. The server device 41 calculates a workpiece coordinate system to which this group is applied. For this calculation, images of the marks M1, M2, M3 captured by the first cameras 31, 32, 33, respectively, when the workpiece 1 is at the first position P1 are used.

[0095] The four second robots 16, 17, 18, 19 in charge of the second work process can be considered to form one group. The server device 41 calculates a work coordinate system to which this group is applied. For this calculation, images of the marks M2, M1, M4 captured by the second cameras 36, 37, 38, respectively, when the work 1 is at the second position P2 are used.

[0096] In this embodiment, the interlock control described in the first embodiment is also performed. However, this interlock control is performed for each group of work processes described above. Therefore, it is possible to prevent the interlock control in one work process from affecting another work process.

[0097] Although the preferred embodiment of the present invention has been described above, the above configuration can be modified, for example, as follows.

[0098] In the above embodiment, the server device 41 calculates the workpiece coordinate system in response to a request from the robot controller 21. However, the present invention is not limited to this, and the server device 41 may be configured so that the server device 41 automatically starts the calculation of the workpiece coordinate system when the server device 41 receives the necessary image data from the cameras 31, 32, and 33.

[0099] As the visual sensor, a stereo camera may be used instead of a camera, or a three-dimensional visual sensor such as a three-dimensional laser sensor may be used.

[0100] One camera may capture two or more marks simultaneously. In this case, two or more marks appearing in one image are measured by the server device 41. Therefore, for example, the positions of three marks M1, M2, and M3 can be measured by two cameras. In this way, the number of cameras in each work process is arbitrary.

[0101] The workpiece coordinate system may be determined from the positions of four or more marks rather than three.

[0102] Two or more computers may be used as the server device 41. In this case, the processing of visual information can be shared among them. [Explanation of symbols]

[0103] 11,12,13,14 Robot 21, 22, 23, 24 Robot Controller 31, 32, 33 Camera (visual sensor) 41 Server device (visual processing computer) 100 Robot System

Claims

1. A plurality of robots arranged so as to be able to work simultaneously on the same workpiece; A robot controller arranged corresponding to each of the robots; A plurality of visual sensors capable of acquiring visual information; a visual processing computer that performs processing based on the information acquired by the visual sensor; Equipped with Each of the plurality of visual sensors is capable of acquiring visual information about a measurement target of the workpiece, the plurality of robot controllers and the plurality of visual sensors are all capable of communicating with the common visual processing computer; the visual processing computer calculates a work coordinate system based on a result of measuring the measurement target appearing in the visual information acquired by the plurality of visual sensors; Each of the robot controllers corrects the movement of the robot corresponding to the robot controller based on the result of the request to the vision processing computer; the visual processing computer stores information regarding the workpiece coordinate system for each of the plurality of robots; the visual processing computer initializes the information on the robot corresponding to the robot controller in response to an initialization command transmitted by the robot controller; the robot controller of the robot belonging to a group of the plurality of robots to which the work coordinate system is applied transmits an inquiry to the vision processing computer at irregular times as to whether or not initialization of information relating to the work coordinate system has been completed for all of the robots belonging to the group; When a robot belonging to the group has not yet completed initialization of information related to the work coordinate system, the visual processing computer, when receiving an inquiry from the robot controller corresponding to the robot belonging to the group, transmits a standby command to the robot controller that originated the inquiry as a response to the inquiry; When the visual processing computer receives an inquiry from the robot controller corresponding to a robot belonging to the group in a state in which initialization of information related to the work coordinate system has been completed for all of the robots belonging to the group, the visual processing computer transmits a completion signal, which is a signal indicating that initialization of information related to all of the robots belonging to the group, to the robot controller that originated the inquiry as a response to the inquiry; A robot system characterized in that each of the robot controllers, when receiving a standby command, queries the visual processing computer again, but, when receiving a completion signal, does not query the visual processing computer again.

2. The robot system according to claim 1 , when the vision processing computer receives a request from the robot controller while the visual sensor has not yet completed measurement of the measurement target, the vision processing computer transmits a standby command to the requesting robot controller as a response to the request; A robot system, wherein each of the robot controllers, when receiving the standby command, again makes a request to the visual processing computer.

3. The robot system according to claim 1 or 2, The plurality of robots include a plurality of first robots arranged so as to be capable of simultaneously working on a workpiece at a first position; a plurality of second robots arranged so as to be capable of simultaneously working on a workpiece transported from the first position to the second position after the first robot has completed its work; Including, the plurality of visual sensors include a plurality of first visual sensors and a plurality of second visual sensors; the robot controller of the first robot, the robot controller of the second robot, the first visual sensor, and the second visual sensor are capable of communicating with the same visual processing computer; the visual processing computer calculates a workpiece coordinate system at the first position based on a result of measuring the measurement object appearing in the visual information acquired by the plurality of first visual sensors, and calculates a workpiece coordinate system at the second position based on a result of measuring the measurement object appearing in the visual information acquired by the plurality of second visual sensors; The robot controller of each of the first robots corrects the movement of the corresponding first robot based on the result of the request to the vision processing computer; A robot system, wherein the robot controller of each of the second robots corrects the operation of the corresponding second robot based on the result of the request to the vision processing computer.

4. The robot system according to claim 3, when the vision processing computer receives a request from the robot controller of the first robot in a state where the first visual sensor has not yet completed measurement of the measurement target, the vision processing computer transmits a standby command to the requesting robot controller as a response to the request; the robot controller of each of the first robots, when receiving the standby command, queries the vision processing computer again; when the vision processing computer receives a request from the robot controller of the second robot in a state where the second visual sensor has not yet completed measurement of the measurement target, the vision processing computer transmits a standby command to the requesting robot controller as a response to the request; The robot system according to claim 1, wherein the robot controller of each of the second robots, when receiving the standby command, again queries the vision processing computer.

5. The robot system according to any one of claims 1 to 4, the visual processing computer is capable of communicating with a setting computer; A robot system according to claim 1, wherein the visual processing computer is capable of changing settings relating to calculation of the work coordinate system by transmission from the setting computer.

6. A plurality of robots arranged so as to be able to work simultaneously on the same workpiece; A robot controller arranged corresponding to each of the robots; A plurality of visual sensors capable of acquiring visual information; a visual processing computer that performs processing based on the information acquired by the visual sensor; A robot control method for a robot system comprising: A first step in which each of the plurality of visual sensors acquires visual information regarding a measurement target of the workpiece and transmits the visual information to the common visual processing computer; a second step of calculating a work coordinate system based on a result of measuring the measurement target appearing in the visual information received from the plurality of visual sensors by the visual processing computer; a third step in which each of the plurality of robot controllers transmits a request to the common vision processing computer and corrects an operation of the robot corresponding to the robot controller based on a response received from the vision processing computer; a fourth step in which the visual processing computer stores information about the workpiece coordinate system for each of the plurality of robots; a fifth step in which the visual processing computer initializes the information on the robot corresponding to the robot controller in response to an initialization command transmitted by the robot controller; a sixth step in which each of the robot controllers of the robots belonging to a group of the robots to which the work coordinate system is applied sends an inquiry to the vision processing computer at irregular timing as to whether or not initialization of information related to the work coordinate system has been completed for all of the robots belonging to the group; a seventh step in which, when the visual processing computer receives a query from the robot controller corresponding to a robot belonging to the group in a state where there is a robot belonging to the group for which initialization of information related to the work coordinate system has not been completed, the visual processing computer transmits a standby command as a response to the query to the robot controller which originated the query; an eighth step of transmitting, when the visual processing computer has received an inquiry from the robot controller corresponding to a robot belonging to the group in a state in which initialization of information related to the work coordinate system has been completed for all of the robots belonging to the group, a completion signal indicating that initialization of information related to all of the robots belonging to the group has been completed to the robot controller that originated the inquiry as a response to the inquiry; a ninth step in which each of the plurality of robot controllers, when receiving a standby command, queries the visual processing computer again, but, when receiving a completion signal, does not query the visual processing computer again; A robot control method comprising:

Images