Robotic system and method of controlling a robot

Abstract

Robots are arranged in a manner capable of simultaneously operating on the same workpiece. A robot controller is arranged corresponding to each of the robots. Each of a plurality of vision sensors is capable of acquiring visual information on a measurement object possessed by the workpiece. The plurality of robot controllers and the plurality of vision sensors are each capable of communicating with a common vision processing computer. The vision processing computer calculates a workpiece coordinate system based on a result obtained from measurement of the measurement object appearing in the visual information acquired by the plurality of vision sensors. Each of the robot controllers corrects the operation of the robot corresponding to the robot controller based on a result requested from the vision processing computer.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the use of a vision sensor of a robot. BACKGROUND

[0002] In the case where a robot is caused to perform a predetermined action, a program is made in advance through an offline teaching work or through simulation. The positional relationship at the time of actual use of the robot can be different from the positional relationship between the robot and a workpiece or the like at the time of offline or simulation. In order to measure the error and make a correction, a system disclosed in, for example, Patent Literature 1 is proposed.

[0003] Patent Literature 1 discloses a structure in which, in the case where a plurality of sets each consisting of one robot and one robot control device are provided, image data of one camera can be shared among a plurality of robot control devices. One photographing section (camera) is used for a plurality of robots (robot control devices). The camera can be attached to the tip end portion of the arm of each robot.

[0004] The camera is replaced each time the combination of the robot and the measurement target object is changed. Specifically, first, the camera is attached to the first robot, and the three-dimensional relative position of the combination of the first robot and the measurement target object is measured. Next, the camera attached to the first robot is attached to the second robot, and the three-dimensional relative position of the combination of the second robot and the measurement target object is measured.

[0005] The system of Patent Literature 1 is provided with an image processing device. The image processing device performs image processing for extracting, for example, three points from the photographed image of the camera. Patent Literature 1 also discloses an example in which the image processing device is provided outside the robot control device and incorporated into a network.

[0006] [Related Art Documents]

[0007] [Patent Literature]

[0008] Patent Literature 1: Japanese Patent No. 4137862 SUMMARY

[0009] Problems to be Solved by the Invention

[0010] In the structure of Patent Literature 1, a plurality of robots photograph a measurement target object while passing one camera. Thus, the cycle time for correction increases, and the time required for programming the action for passing in the robots is needed.

[0011] In Patent Literature 1, three points are extracted from the image photographed by the camera in each robot to measure the three-dimensional relative position. Thus, it becomes difficult to match the positional correction among a plurality of robots.

[0012] The present disclosure was developed in view of the above circumstances, and aims to provide a robot control system capable of easily making a correction of matching between a plurality of robots.

[0013] Means for solving the problem

[0014] The problem to be solved by the present disclosure is as explained above, and the means for solving the problem and its effects are explained below.

[0015] According to a first aspect of the present disclosure, a robot system is provided with the following structure. That is, the robot system includes a plurality of robots, a plurality of robot controllers, a plurality of vision sensors, and a vision processing computer. The robots are configured in a manner capable of simultaneously operating on the same workpiece. The robot controllers are configured corresponding to each of the robots. The vision sensors are capable of acquiring vision information. The vision processing computer processes information acquired by the vision sensors. Each of the plurality of vision sensors is capable of acquiring vision information related to a measurement object possessed by the workpiece. Each of the plurality of robot controllers and the plurality of vision sensors is capable of communicating with the common vision processing computer. The vision processing computer calculates a workpiece coordinate system based on a result of measuring the measurement object appearing in the vision information acquired by the plurality of vision sensors. Each of the robot controllers corrects the action of the robot corresponding to the robot controller based on a result requested to the vision processing computer.

[0016] According to a second aspect of the present disclosure, a robot control method is provided. That is, the robot control method is directed to a robot system provided with a plurality of robots, a plurality of robot controllers, a plurality of vision sensors, and a vision processing computer. The robots are configured in a manner capable of simultaneously operating on the same workpiece. The robot controllers are configured corresponding to each of the robots. The vision sensors are capable of acquiring vision information. The vision processing computer processes information acquired by the vision sensors. The robot control method includes a first step, a second step, and a third step. In the first step, each of the plurality of vision sensors acquires vision information related to a measurement object possessed by the workpiece, and transmits the vision information to the common vision processing computer. In the second step, the vision processing computer calculates a workpiece coordinate system based on a result of measuring the measurement object appearing in the vision information received from the plurality of vision sensors. In the third step, each of the plurality of robot controllers transmits a request to the common vision processing computer, and corrects the action of the robot corresponding to the robot controller based on a reply received from the vision processing computer.

[0017] Thus, since one or a small number of vision processing computers acquire information from a plurality of vision sensors and calculate the workpiece coordinate system structure, the collection of information can be naturally achieved. Therefore, it is suitable to aggregate information from a plurality of vision sensors for the calculation of the workpiece coordinate system. Since the workpiece coordinate system is commonly used for the motion correction of a plurality of robots, it is possible to match the motion correction among the plurality of robots. In addition, since the server device centrally performs processing corresponding to requests from a plurality of robot controllers, it is easy to achieve simplification of processing, simplification of communication, and the like.

[0018] Effects of Invention

[0019] According to the present disclosure, it is possible to easily perform correction that is matched among a plurality of robots. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a schematic diagram showing the overall structure of a robot system of Embodiment 1 of the present disclosure.

[0021] Figure 2 is a flowchart illustrating interlocking control of initialization.

[0022] Figure 3 is a flowchart illustrating a marker measurement process based on a camera image.

[0023] Figure 4 is a flowchart illustrating interlocking control related to acquisition of a workpiece coordinate system.

[0024] Figure 5 is a schematic diagram of a robot system of Embodiment 2. DETAILED DESCRIPTION

[0025] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Figure 1 is a schematic diagram showing the overall structure of a robot system 100 of Embodiment 1 of the present disclosure.

[0026] Figure 1 The robot system 100 shown is a system that operates a workpiece 1 using a plurality of robots 11, 12, 13, 14. The robot system 100 is installed in a production line of a factory, for example.

[0027] The robot system 100 includes the robots 11, 12, 13, 14, robot controllers 21, 22, 23, 24, cameras (vision sensors) 31, 32, 33, and a server device (vision processing computer) 41.

[0028] The robot 11 is an industrial robot. The robot 11 is configured as a vertical multi-joint robot with a degree of freedom of movement of 6. However, the structure of the robot is not limited thereto.

[0029] The workpiece 1 conveyed by the conveyer 2 is stationary at a predetermined position surrounded by the four robots 11, 12, 13, 14. The four robots 11, 12, 13, 14 can simultaneously operate on the workpiece 1. The operation performed by the robots is arbitrary, and for example, assembly, welding, coating, and the like can be considered. Other devices (for example, feeding devices) that enable the workpiece 1 to flow can also be used instead of the conveyer 2.

[0030] The four robots 11, 12, 13, 14 can be identical or different. The operations performed by the four robots 11, 12, 13, 14 on the workpiece 1 can be identical or different. The number of robots that simultaneously operate on the workpiece 1 is not limited to four, and can be two, three, or more than five.

[0031] The four robot controllers 21, 22, 23, 24 are configured in correspondence with the four robots 11, 12, 13, 14. The robot controllers 21, 22, 23, 24 are each configured as a computer having a CPU, a ROM, a RAM, and the like.

[0032] The robot 11 is electrically connected to the robot controller 21. The robot 11 operates in accordance with the instructions of the robot controller 21, and can perform a predetermined operation on the workpiece 1. The robot 12 operates in accordance with the instructions of the robot controller 22. The robot 13 operates in accordance with the instructions of the robot controller 23. The robot 14 operates in accordance with the instructions of the robot controller 24. In this way, one robot is controlled by one robot controller.

[0033] The robot controllers 21, 22, 23, 24 are each electrically connected to the server device 41. Each of the robot controllers 21, 22, 23, 24 can perform transmission and reception of information through communication.

[0034] The cameras 31, 32, 33 are photographing devices that can acquire images (visual information). The three cameras 31, 32, 33 are each fixed at an appropriate position at which the workpiece 1 located at the predetermined position can be photographed. The number of cameras can be one, two, or more than four. Some or all of the plurality of cameras can also be configured by cameras that can be held by a robot, instead of fixed cameras. In this case, the robot is caused to operate so that the camera reaches a predetermined photographing position, and then photographing is performed.

[0035] The camera 31 is capable of capturing the mark Ml attached to the workpiece 1. The camera 32 is capable of capturing the mark M2 attached to the workpiece 1. The camera 33 is capable of capturing the mark M3 attached to the workpiece 1. The marks (measurement targets) Ml, M2, M3 can be, for example, stickers attached to the workpiece 1. However, as the measurement targets, it is also possible to capture, instead of the marks Ml, M2, M3, features (for example, protrusions, recesses, and the like) in the shape of the workpiece 1 by the cameras 31, 32, 33.

[0036] Depending on the conditions, it is also possible to capture the three marks Ml, M2, M3 at the same time using one camera. However, depending on the size and shape of the workpiece 1, the positions of the marks Ml, M2, M3, and the like, it is not always possible to fit the three marks Ml, M2, M3 in the field of view of one camera. In the configuration of the present embodiment, since the three cameras 31, 32, 33 share the capturing of the three marks Ml, M2, M3, it is possible to easily capture each of the marks Ml, M2, M3.

[0037] The camera 31 is disposed in the vicinity of the robot 11 and is associated with the robot controller 21. The camera 32 is disposed in the vicinity of the robot 12 and is associated with the robot controller 22. The camera 33 is disposed in the vicinity of the robot 13 and is associated with the robot controller 23. Thus, the cameras 31, 32, 33 are subordinate to any of the four robots 11, 12, 13, 14, respectively.

[0038] Each of the cameras 31, 32, 33 is electrically connected to the server device 41. Each of the cameras 31, 32, 33 is capable of outputting the captured image to the server device 41 through communication.

[0039] The communication between the server device 41 and the robot controllers 21, 22, 23, 24, and the communication between the server device 41 and the cameras 31, 32, 33 can be performed using, for example, a known LAN. However, the communication method is not limited to the LAN.

[0040] The server device 41 is configured as a computer having a CPU, a ROM, a RAM, and the like. If the server device 41 has a CPU having higher performance than the robot controllers, or has a special hardware such as a GPU, it is possible to perform the image processing described later at high speed, which can be preferable.

[0041] The server device 41 performs known image processing (vision processing) on the image data obtained from the plurality of cameras 31, 32, 33, and determines the positions of the respective markers Ml, M2, M3 in the three-dimensional space. Thereby, a three-dimensional orthogonal coordinate system representing the real position and posture of the workpiece 1 can be obtained. The coordinate system of the workpiece 1 refers to a coordinate system set on the workpiece 1, which changes in conjunction with the movement / rotation of the workpiece 1. Hereinafter, this coordinate system will be sometimes referred to as workpiece coordinate system.

[0042] In the server device 41, a three-dimensional rectangular coordinate system (hereinafter referred to as common coordinate system) common to the robots 11, 12, 13, 14 and fixed with respect to the ground is appropriately defined. In the present embodiment, any one of the four robots 11, 12, 13, 14 is determined as a master robot. The common coordinate system can be, for example, a three-dimensional orthogonal coordinate system (master coordinate system) set on the master robot. The coordinate system set on a robot refers to, for example, a coordinate system set with reference to a fixed base portion located at the arm root portion of the arm of the robot.

[0043] The workpiece coordinate system can be represented as a coordinate representing the origin thereof in the common coordinate system, and a direction representing the directions of the three coordinate axes in the common coordinate system. In the present specification, the workpiece coordinate system will be sometimes referred to as frame.

[0044] The details are as follows. For each of the three cameras 31, 32, 33, a camera coordinate system corresponding to the actual installation position and posture is determined. In the server device 41, a camera coordinate system, which is a coordinate system as seen in the coordinate system inherent to the robot to which the camera belongs, is stored in advance for each camera. The coordinate system inherent to each robot will be described later. The server device 41 acquires the positions of the markers in the images captured by each camera in the form of a two-dimensional orthogonal coordinate by using an appropriate method such as pattern recognition. The marker positions in the images correspond to the directions of the markers with respect to the camera that captured the markers in the three-dimensional space. In addition, in the workpiece 1, the relative positional relationship among the three markers Ml, M2, M3 is a known relationship. The server device 41 solves simultaneous equations made from the information on the basis of the transformation of the directions of the markers with respect to the cameras into the common coordinate system (master coordinate system), and thereby obtains the positions of the three markers Ml, M2, M3 in the common coordinate system. If the positions of the three markers Ml, M2, M3 are determined, the workpiece coordinate system can be uniquely determined.

[0045] In the present embodiment, the cameras 31, 32, 33 are all configured as two-dimensional cameras. However, one or more of the cameras can be changed to three-dimensional cameras. The configuration of the three-dimensional cameras is arbitrary, and can be, for example, a stereo type, a ToF type, or the like. According to the results of photographing by the three-dimensional cameras, the positions of the markers Ml, M2, M3 can be obtained in three dimensions. The position of a marker is represented by a three-dimensional coordinate system (camera coordinate system) possessed by the three-dimensional camera that photographed the marker. The server device 41 obtains the positions of the three markers Ml, M2, M3 in the common coordinate system by solving simultaneous equations on the basis of transforming the positions of the markers photographed by the three-dimensional cameras from the camera coordinate systems to the common coordinate system. In the case of using a vision sensor capable of three-dimensional measurement in this way, a known three-dimensional shape recognition can be used, and the features of the three-dimensional shape of the workpiece 1 can be used as the measurement targets instead of the markers Ml, M2, M3.

[0046] The server device 41 replies to the request from the robot controllers 21, 22, 23, 24 with the obtained workpiece coordinate system. The robot controllers 21, 22, 23, 24 acquire the replied workpiece coordinate system as a correction coordinate system. As a result, the robots 11, 12, 13, 14 can recognize the actual position and posture of the workpiece 1, and correct the movements accordingly. As a result, the robots 11, 12, 13, 14 can flexibly cope with the shift in the position and posture of the workpiece 1.

[0047] The server device 41 also replies to an arbitrary robot (robot controller) with the workpiece coordinate system obtained by integrating the photographing results of the three cameras 31, 32, 33 and performing the calculation. As a result, the movement corrections of the four robots 11, 12, 13, 14 are matched with each other. In other words, the four robots 11, 12, 13, 14 can be controlled to perform the acquisition correction. As a result, the system is suitable for the case where one workpiece 1 is simultaneously handled.

[0048] The coordinate systems unique to the robots other than the master robot among the four robots 11, 12, 13, 14 are also defined in the same manner as the master coordinate system. In the server device 41, the coordinate system of each robot is stored in advance for each robot, and the coordinate system of each robot is a coordinate system as seen in the common coordinate system (master coordinate system).

[0049] In the case where the server device 41 replies to the robots other than the master robot in the workpiece coordinate system, it is also possible to not transform the workpiece coordinate system into the common coordinate system, but to transform into a form seen in the coordinate system inherent to the inquiring robot (robot controller), and to reply to the transformation result. By using information of the robot coordinate system, it is possible to easily perform the coordinate transformation. For example, assume that the robot 13 is the master robot, and a vector Vms from the origin of the master coordinate system as the common coordinate system to the origin of the coordinate system inherent to the robot 11 is known. In the case where a vector Vmes representing the position of the workpiece 1 in the master coordinate system is Vmes, a vector Vmes' representing the position of the workpiece 1 in the coordinate system of the robot 11 can be represented by the following expression.

[0050] Vmes' = -Vms + Vmes

[0051] In the case where the coordinate transformation is performed on the robot controller side, the coordinate transformation on the server device 41 side is not needed.

[0052] However, in the case where the workpiece coordinate system is not obtained in the server device 41, as the robot side, it means that information serving as a reference for the correction operation cannot be obtained. Thus, in this state, the four robots 11, 12, 13, 14 are not suitable for the operation on the workpiece 1. In this regard, in the present embodiment, it becomes a configuration in which interlocking control is realized by performing communication between the robot controllers 21, 22, 23, 24 and the server device 41, and the robots 11, 12, 13, 14 stand by until the workpiece coordinate system is obtained.

[0053] The server device 41 stores logs related to the operation and various settings. An operator can read the logs, change the settings, and the like by operating the server device 41.

[0054] The server device 41 is configured to be able to perform communication with a setting computer 51. For example, communication can be performed using a known LAN or WAN. The setting computer 51 is disposed at a position physically distant from the server device 41. The setting computer 51 is configured as a known computer. A remote operator can read logs recorded in the server device 41 or change settings related to the production of the workpiece coordinate system by operating the setting computer 51 to access the server device 41.

[0055] Next, several interlocking controls performed in the robot system 100 will be described.

[0056] First, the interlocking control of the initialization processing will be described. Figure 2 The processing performed by the four robot controllers 21, 22, 23, 24 is substantially the same, and thus the processing performed by one of the robot controllers 21 will be described here. Figure 2The left side shows the processing by the robot controller 21, and the right side shows the processing by the server device 41.

[0057] Figure 2 The left side shows the processing by the robot controller 21. In the robot controller 21, the processing is started by some trigger. Figure 2 The right side shows the processing by the server device 41.

[0058] The robot controller 21 first transmits an initialization instruction to the server device 41 (step S101). If the server device 41 receives the instruction (step S102), it deletes the storage of the transformation result of the workpiece coordinate system of the robot controller 21 (actually, the robot 11) of the transmission source, and the like (step S103). If the initialization processing is completed, the server device 41 transmits information of completion of initialization to the robot controller 21 of the transmission source (step S104). The robot controller 21 receives the information of completion of initialization from the server device 41 (step S105).

[0059] The above explains the processing of step S101 to S105 focusing on one robot controller 21, but the same processing is performed in the other robot controllers 22, 23, 24. Since the start timing of the processing of steps S101 to S105 in the four robot controllers 21, 22, 23, 24 is not determined, the transmission timing of the initialization instruction to the server device 41 is also not determined. The server device 41 performs initialization related to the robot at the time of reception of the initialization instruction. Figure 2 The processing of steps S101 to S105 is performed in the robot controllers 21, 22, 23, 24. Since the start timing of the processing of steps S101 to S105 in the four robot controllers 21, 22, 23, 24 is not determined, the transmission timing of the initialization instruction to the server device 41 is also not determined. The server device 41 performs initialization related to the robot at the time of reception of the initialization instruction.

[0060] If the robot controller 21 receives the information of completion of initialization from the server device 41, it transmits a query to the server device 41, inquiring whether initialization has been completed for all the robots 11, 12, 13, 14 (step S106). The server device 41 receives the query from the robot controller 21 (step S107).

[0061] If the server device 41 receives the query, it judges whether the initialization processing of step S103 has been completed for all the four robots 11, 12, 13, 14 (step S108). In the case where initialization has been completed for all the robots, the server device 41 transmits ok as a reply to the robot controller 21 of the source of the query (step S109). In the case where there is a robot for which initialization has not been completed, the server device 41 transmits wait as a reply (step S110). Wait indicates that standby is required. Thus, the reply of wait can be considered as a kind of standby instruction from the server device 41 to the robot controller 21 of the source of the query.

[0062] ​Depending on the situation, it is also possible to consider a scenario where only three out of the four robots operate on workpiece 1. This reduced application could be due to the assumption that some robots have malfunctioned, but is not limited to this. If a setting is made to exclude some of the four robots from the interlocked control objects, then in process S108, the server device 41 determines whether all initialization has been completed, focusing only on the active robots.

[0063] If the robot controller 21 receives a response to the query from the server device 41 (step S111), it judges the content of the response (step S112). If the response is "wait", the process returns to step S106, and the robot controller 21 sends the query to the server device 41 again. If the response is "ok", the series of processes used for initialization ends. The robot controller 21 is able to complete... Figure 2 The process shown below is a condition for starting the next process.

[0064] The above description focuses on a robot controller 21 to illustrate the process. The same processing is performed in the other robot controllers 22, 23, and 24. Regardless of which of the four robot controllers 21, 22, 23, and 24 the query is received from, the server device 41 responds with wait if there is a robot that has not completed the initialization of process S103. Figure 2 In the process shown on the left, if the response from server device 41 is not "ok", the process terminates abnormally. Therefore, if initialization is not completed for all robots 11, 12, 13, and 14 in server device 41, initialization will not be completed in any robot controller 21, 22, 23, and 24. Figure 3 This process enables interlocking control regarding the initialization process.

[0065] Next, refer to Figure 3 The measurement process for the prior markings M1, M2, and M3 used to acquire the workpiece coordinate system is explained. This process involves three robot controllers 21, 22, and 23 that are associated with cameras 31, 32, and 33. Since the processes performed by the three robot controllers 21, 22, and 23 are essentially the same, the process performed by one of the robot controllers 21 will be described here. As mentioned above, appropriate visual features of the workpiece 1 can also be measured instead of markings M1, M2, and M3.

[0066] Figure 3 The left side shows the process performed by the robot controller 21. When the workpiece 1 is detected by sensors or the like to have been transported to a predetermined position, the process begins in the robot controller 21. Figure 4 The right side shows the processing performed by server device 41.

[0067] The robot controller 21 first transmits a measurement instruction to the server device 41 (step S201). The server device 41 receives the instruction (step S202). The server device 41 immediately transmits a photographing instruction to the camera 31 associated with the robot controller 21 (in fact, the robot 11) of the transmission source, and acquires the position of the marker Ml from the acquired photographed image by pattern recognition or the like (step S203). The position of the marker Ml in question here refers to a two-dimensional orthogonal coordinate in the image. In step S203, the process of the camera 31 transmitting the image to the server device 41 corresponds to the first process.

[0068] Then, the server device 41 determines whether the measurement of the position of the marker Ml is normally completed (step S204). In the case where the measurement is normally completed, the server device 41 converts the position of the marker Ml from the two-dimensional orthogonal coordinate system into a three-dimensional orthogonal coordinate system inherent to the robot 11, and further into the master coordinate system in the case where the robot 11 is not the master robot, and stores it (step S205). The conversion of the master coordinate system into the coordinate system of the other robot will be described later. Then, the server device 41 transmits information that the measurement is normally completed to the robot controller 21 that is the source of the instruction (step S206).

[0069] In the case where the position of the marker Ml cannot be acquired due to a failure in pattern recognition or the like, the server device 41 transmits information that the measurement is abnormally completed to the robot controller 21 (step S207).

[0070] As described above, the camera 31 can also be a three-dimensional camera. In the case where the camera 31 is a three-dimensional camera, the coordinates of the marker Ml obtained by the server device 41 in step S203 become three-dimensional coordinates in a three-dimensional coordinate system (camera coordinate system, vision sensor coordinate system) possessed by the camera 31. In step S205, the three-dimensional coordinates are converted into a three-dimensional orthogonal coordinate system inherent to the robot 11, and into the master coordinate system as necessary. In step S203, the process of the camera 31 transmitting, for example, three-dimensional point cloud data to the server device 41 corresponds to the first process.

[0071] If the robot controller 21 receives a reply to the measurement instruction from the server device 41 (step S208), it determines the measurement result contained in the reply (step S209). In the case where the measurement result is normal, the robot controller 21 records the content that the position of the marker Ml can be normally measured in a log or the like (step S210). In the case where the measurement result is abnormal, the robot controller 21 records the content that an abnormality has occurred in a log or the like (step S211). In either case, the series of processes is ended.

[0072] In the above description, the processing of the procedure is explained focusing on one robot controller 21, but the same processing is also performed in the other robot controllers 22, 23. Thereby, the positions of the three markers Ml, M2, M3 can be measured from the images captured by the three cameras 31, 32, 33, respectively, and the measurement results are stored on the server device 41 side. The timing at which each robot controller 21, 22, 23 transmits the measurement instruction to the server device 41 can be arbitrary. Upon receipt of the measurement instruction, the server device 41 measures the positions of the markers based on the captured images of the cameras with which the robot controller of the transmission source is associated.

[0073] Next, the interlocking control related to the creation of the workpiece coordinate system is explained with reference to Figure 4 The processing is performed by the four robot controllers 21, 22, 23, 24. Since the processing performed by the four robot controllers 21, 22, 23, 24 is substantially the same, the processing performed by one of the robot controllers 21 is explained here. For simplicity of description, the workpiece coordinate system is represented as a frame in Figure 4

[0074] Figure 3 The left side of Fig. 17 shows the processing performed by the robot controller 21. In the robot controller 21, the processing is started by some trigger. The trigger is, for example, the end of the measurement processing of the marker Ml explained in Figure 4 , but is not limited thereto. Figure 3 The right side of Fig. 17 represents the processing performed by the server device 41.

[0075] First, the robot controller 21 transmits a request for the creation of the workpiece coordinate system to the server device 41 (procedure S301). The server device 41 receives the request from the robot controller 21 (procedure S302).

[0076] Upon receipt of the request, the server device 41 judges whether the measurement processing explained in procedure S203 of Figure 4 has been completed for all three markers Ml, M2, M3 (procedure S303). In other words, this judgment means the judgment of whether the measurement processing has been completed for all the robot controllers 21, 22, 23, or whether the measurement processing has been completed for all the three cameras 31, 32, 33.

[0077] In the case where all the measurement processing has been completed, the server device 41 judges whether all the measurement processing has been normally completed (procedure S304).

[0078] ​In a case where the entire measurement processing is normally completed, the server device 41 acquires the workpiece coordinate system by calculation using the positions of the markers Ml, M2, M3 stored in step S205 (step S305, second step). Then, the server device 41 calculates the accuracy of the workpiece coordinate system created by calculation and determines whether the accuracy is equal to or higher than a predetermined value (step S306). The accuracy is a specific accuracy of the position and posture of the workpiece 1. For example, the determination can be made in the following manner. The server device 41 stores a reference value in advance for each side of a triangle having the three markers Ml, M2, M3 as vertices. In a case where the difference from the reference value for any one of the three sides of the triangle is equal to or lower than a predetermined value, the server device 41 determines that the accuracy of the workpiece coordinate system is equal to or higher than the predetermined value. In a case where there is one or more sides whose difference from the reference value exceeds the predetermined value, the server device 41 determines that the accuracy of the workpiece coordinate system is lower than the predetermined value. However, the method of determining whether the accuracy is good or not is not limited to the above.

[0079] In a case where it is determined that the workpiece coordinate system has acquired accuracy equal to or higher than a predetermined value, the server device 41 stores the acquired workpiece coordinate system (step S307). The workpiece coordinate system can also be transformed into a form seen in a coordinate system inherent to the robot controller 21 (in fact, the robot 11) of the inquiry source before being stored, as necessary.

[0080] Then, the server device 41 transmits a message indicating that the creation of the workpiece coordinate system has been normally completed to the robot controller 21 of the transmission source of the request as a reply (step S308). The content of the reply includes information of the workpiece coordinate system acquired immediately before.

[0081] In the determination of step S304, in a case where it is determined that the measurement processing has been abnormally completed only once among all the measurement processing, the server device 41 transmits information indicating that the creation of the workpiece coordinate system has been abnormally completed to the robot controller 21 of the transmission source of the request as a reply (step S309). The same applies in a case where the server device 41 determines that the predetermined frame accuracy cannot be acquired through the determination of step S306.

[0082] In a case where there is an uncompleted marker among the measurement processing of the three markers Ml, M2, M3 through the determination of step S303, the server device 41 transmits "wait" to the robot controller 21 of the transmission source of the request as a reply (step S310).

[0083] If the robot controller 21 receives a reply to the request from the server device 41 (step S311), the content of the reply is judged (step S312). In the case where the reply is a normal end, the robot controller 21 records the content that the workpiece coordinate system could be normally made in a log or the like (step S313). In the case where the reply is an abnormal end, the robot controller 21 records the content that an abnormality occurred and the workpiece coordinate system could not be made in a log or the like (step S314). Whether the reply is a normal end or an abnormal end, the series of processes ends. In the case where the reply is wait, the process returns to step S301, and the robot controller 21 transmits a request for making of the workpiece coordinate system to the server device 41 again. Figure 3

[0084] The robot controller 21 transmits a request for making of the workpiece coordinate system in step S301, and in the case where the reply obtained in step S311 is a normal end, the subsequent action of the robot 11 is corrected in a manner suitable for the position and posture of the actual workpiece 1 based on the obtained workpiece coordinate system. This series of processes corresponds to the 3rd process in the robot control method. Figure 4 In the above description, the processes of steps S301 to S314 are described with focus on one robot controller 21, but the same processes are performed in the other robot controllers 22, 23, 24. Regardless of which of the four robot controllers 21, 22, 23, 24 receives a request for making of the workpiece coordinate system, the server device 41 replies with wait when the condition for making of the workpiece coordinate system is not satisfied at the instruction time.

[0085] In the processes shown on the left side of FIG. 10, as long as the reply from the server device 41 does not indicate a normal end, it is not a normal end. Thus, as long as the condition that the workpiece coordinate system can be made is not satisfied in the server device 41, the process is an incomplete process in any of the robot controllers 21, 22, 23, 24. In this way, interlocking control related to the workpiece coordinate system is achieved. Figure 4 Figure 5 In the present embodiment, the server device 41 acquires image data from the three cameras 31, 32, 33, and measures the markers M1, M2, M3. Thus, since image processing can not be performed on the robot controller 21, 22, 23, 24 side, a cheap structure can be adopted. In the server device 41, even in the case where a high-performance CPU is used in order to perform image processing in a short time, since there is only one server device 41, the increase in cost is limited.

[0086] In the present embodiment, the server device 41 acquires image data from the three cameras 31, 32, 33, and measures the markers M1, M2, M3. Thus, since image processing can not be performed on the robot controller 21, 22, 23, 24 side, a cheap structure can be adopted. In the server device 41, even in the case where a high-performance CPU is used in order to perform image processing in a short time, since there is only one server device 41, the increase in cost is limited.

[0087] ​​In the robot system 100 of the present embodiment, the server device 41 which performs image processing as described above also performs calculation of the workpiece coordinate system in response to a request for creation of the workpiece coordinate system from the plurality of robot controllers 21, 22, 23, 24. In addition, in a case where a condition for being able to calculate the workpiece coordinate system is not satisfied, the request for creation of the workpiece coordinate system is replied to with "wait" by the server device 41, and interlocking can be achieved.

[0088] Next, the range of the robot for which the workpiece coordinate system calculated by the server device 41 is explained.

[0089] In the present embodiment, the workpiece coordinate system is shared among the four robots 11, 12, 13, 14 which simultaneously perform one process on the workpiece 1. In other words, it can be considered that one group is formed by the four robots 11, 12, 13, 14, and one workpiece coordinate system is calculated for this group and is commonly used.

[0090] However, the server device 41 can also calculate the workpiece coordinate system for the two robots 11, 12 which are located on the right side with respect to the conveyer belt 2, and calculate the workpiece coordinate system for the two robots 13, 14 which are located on the left side, for example. In this case, it can be considered that one group is formed by the two robots on the left side of the conveyer belt 2, and one group is formed by the two robots on the right side, and the workpiece coordinate system is calculated for each group. The interlocking control of initialization and the interlocking control of the workpiece coordinate system can be performed in units of the groups.

[0091] As explained above, the robot system 100 of the present embodiment includes the four robots 11, 12, 13, 14, the four robot controllers 21, 22, 23, 24, the three cameras 31, 32, 33, and the server device 41. The four robots 11, 12, 13, 14 are arranged in a manner that enables simultaneous operation on the same workpiece 1. The robot controllers 21, 22, 23, 24 are arranged in correspondence with the respective robots 11, 12, 13, 14. The cameras 31, 32, 33 are capable of acquiring images. The server device 41 performs processing based on information acquired by the cameras 31, 32, 33. The camera 31 is capable of acquiring an image of the mark Ml possessed by the workpiece 1. Similarly, the camera 32 is capable of acquiring an image of the mark M2, and the camera 33 is capable of acquiring an image of the mark M3. The four robot controllers 21, 22, 23, 24 and the three cameras 31, 32, 33 are each capable of communicating with the common server device 41. The server device 41 calculates the workpiece coordinate system based on a result of measuring the marks Ml, M2, M3 appearing in the images acquired by the three cameras 31, 32, 33. Each of the robot controllers 21, 22, 23, 24 corrects the operation of the robot 11, 12, 13, 14 corresponding to the robot controller based on the result requested to the server device 41.

[0092] Thus, it is possible to clearly distinguish the processing objects in the server device 41 and the robot controllers 21, 22, 23, 24. Since the image processing is not performed on the robot controller 21, 22, 23, 24 side, inexpensive hardware can be employed. Since the server device 41 on the image processing side is one, even if special hardware that improves the image processing capability is employed, the increase in cost is limited. Since the structure in which one server device 41 acquires information from a plurality of cameras 31, 32, 33 and calculates the workpiece coordinate system, the collection of information is naturally achieved. Thus, it is suitable to aggregate the information from a plurality of cameras 31, 32, 33 to calculate the workpiece coordinate system. Further, since the server device 41 centrally performs the processing corresponding to the requests from a plurality of robot controllers 21, 22, 23, 24, it is easy to achieve the simple purification of the processing, the simplification of the communication, and the like.

[0093] Further, in the robot system 100 of the present embodiment, the server device 41 transmits wait to the requesting source robot controller 21 as a reply to the request in a state in which there is a camera 31, 32, 33 for which the measurement of the object is not completed, for example, in a case where the request is received from the robot controller 21. In a case where the wait is received, the robot controller 21 again makes a request to the server device 41.

[0094] Thus, it is possible to rationally achieve the interlock control in which the robot 11, 12, 13, 14 is not caused to act if the workpiece coordinate system is not acquired, in the form in which the server device 41 that calculates the workpiece coordinate system serves as a window that responds to the request.

[0095] Further, in the robot system 100 of the present embodiment, the server device 41 stores the information on the workpiece coordinate system related to a plurality of robots 11, 12, 13, 14 for each robot. The server device 41 initializes the information related to the robot 11 corresponding to the robot controller 21, for example, in accordance with the initialization instruction transmitted by the robot controller 21. The robot controller of the robot that belongs to a group constituted by a plurality of robots 11, 12, 13, 14 that are the application objects of the workpiece coordinate system (for example, the robot controller 21) transmits the following inquiry to the server device 41: that is, whether the initialization of the information related to the workpiece coordinate system has been completed for all of the four robots 11, 12, 13, 14 that belong to the group. In a state in which there is a robot that belongs to the group and for which the initialization of the information related to the workpiece coordinate system is not completed, in a case where the inquiry is received from the robot controller (for example, the robot controller 21) corresponding to the robot that belongs to the group, the server device 41 transmits wait to the inquiring source robot controller 21 as a reply to the inquiry. The robot controller 21 again makes an inquiry to the server device 41 in a case where the wait is received.

[0096] Thus, the interlock control that does not shift to the next process if the information of all the robots is not initialized can be implemented by the server device 41.

[0097] In addition, in the robot system 100 of the present embodiment, the server device 41 can communicate with the setting computer 51. The server device 41 can change the setting related to the acquisition of the workpiece coordinate system by the transmission from the setting computer 51. Various settings related to the acquisition of the workpiece coordinate system can be considered, for example, the setting related to the pattern recognition of the marks M1, M2, M3 can be considered.

[0098] Thus, the remote setting change of the server device 41 can be performed. Therefore, the operator does not need to move to the setting place of the server device 41, and thus the convenience is improved.

[0099] Next, the second embodiment will be described. Figure 5 is a schematic view of the 100x of the second embodiment. Also, in the description of the present embodiment, the same or similar components as those of the above-described embodiment are denoted by the same symbols in the drawings, and the description can be omitted at times.

[0100] In the factory in which the robot system 100x is applied, as shown in ​ , the workpiece 1 is operated across a plurality of operation processes. The server device 41 can calculate the workpiece coordinate system in each operation process.

[0101] The workpiece 1 is conveyed to the first position P1 by the conveyer 2, and the first operation process is performed by the four first robots 11, 12, 13, 14. Then, the workpiece 1 is conveyed to the second position P2 by the conveyer 2, and the second operation process is performed by the four second robots 16, 17, 18, 19.

[0102] The four robot controllers 21, 22, 23, 24 are arranged corresponding to the first robots 11, 12, 13, 14. The four robot controllers 26, 27, 28, 29 are arranged corresponding to the second robots 16, 17, 18, 19.

[0103] The workpiece 1 has four marks M1, M2, M3, M4. The first cameras (first vision sensors) 31, 32, 33 are arranged around the first position P1, and can capture the corresponding marks M1, M2, M3. The second cameras (second vision sensors) 36, 37, 38 are arranged around the second position P2, and can capture the corresponding marks M1, M2, M4.

[0104] The robot controller 21, 22, 23, 24 of the first robot 11, 12, 13, 14 and the robot controller 26, 27, 28, 29 of the second robot 16, 17, 18, 19 are each capable of communicating with the server device 41. The first camera 31, 32, 33 and the second camera 36, 37, 38 are each capable of communicating with the server device 41.

[0105] The four first robots 11, 12, 13, 14 that undertake the first operation process can be considered to form a group. The server device 41 calculates a workpiece coordinate system that is to be applied to this group. In this calculation, the images obtained by the first camera 31, 32, 33 respectively capturing the marks M1, M2, M3 when the workpiece 1 is located at the first position P1 are used.

[0106] The four second robots 16, 17, 18, 19 that undertake the second operation process can be considered to form a group. The server device 41 calculates a workpiece coordinate system that is to be applied to this group. In this calculation, the images obtained by the second camera 36, 37, 38 respectively capturing the marks M2, M1, M4 when the workpiece 1 is located at the second position P2 are used.

[0107] In the present embodiment, the interlock control that has been described in the first embodiment is also performed. However, this interlock control is performed in units of the group of each operation process described above. Thus, it is possible to make the interlock control in a certain operation process not affect other operation processes.

[0108] The preferred embodiments of the present disclosure are described above, but the configuration can be changed, for example, as follows.

[0109] In the present embodiment, the server device 41 performs the calculation of the workpiece coordinate system upon request from the robot controller 21. However, this is not limiting, and the configuration can be such that the server device 41 automatically starts the calculation of the workpiece coordinate system at the time when the server device 41 receives the necessary image data from the camera 31, 32, 33.

[0110] As the vision sensor, a stereo camera can be used instead of the camera, and a three-dimensional vision sensor such as a three-dimensional laser sensor can also be used.

[0111] One camera can also capture two or more marks at the same time. In this case, the two or more marks appearing in one image are measured by the server device 41. Thus, for example, the positions of the three marks M1, M2, M3 can be measured by two cameras. In this way, the number of cameras in each operation process is arbitrary.

[0112] The workpiece coordinate system can also be calculated from the positions of four or more marks instead of three.

[0113] Two or more computers can also be used as the server device 41. In this case, the processing of visual information can be shared.

[0114] The functions of each element disclosed in the present disclosure, including the robot controllers 21, 22, 23, 24, and the server device 41, can be executed using a circuit or processing circuit including a general-purpose processor, a dedicated processor, an integrated circuit, an application-specific integrated circuit (ASIC), a conventional circuit, and / or a combination of these, configured or programmed in a manner to execute the disclosed functions. The processor is considered as a processing circuit or a circuit because it includes transistors or other circuits. In the present disclosure, a circuit, unit, or device is hardware that executes the listed functions or hardware programmed in a manner to execute the listed functions. The hardware can be hardware disclosed in the present specification, or it can be other known hardware programmed or configured in a manner to execute the listed functions. In the case of a processor considered as one of the circuits, the circuit, device, or unit is a combination of hardware and software, and the software is used for the configuration of the hardware and / or the processor.

Claims

1. A robotic system, characterized by, comprise: a plurality of robots configured in a manner that allows simultaneous operation on the same workpiece; a robot controller configured in correspondence with each of the robots; a plurality of vision sensors capable of acquiring visual information; a vision processing computer configured to process information acquired by the vision sensors, each of the plurality of vision sensors is capable of acquiring visual information related to a measurement object possessed by the workpiece, each of the plurality of robot controllers and the plurality of vision sensors is capable of communicating with the common vision processing computer, the vision processing computer is configured to calculate a workpiece coordinate system based on results obtained by measuring the measurement object in the visual information acquired by the plurality of vision sensors, each of the robot controllers is configured to correct the operation of the robot corresponding to the robot controller based on the results requested from the vision processing computer, the vision processing computer is configured to store information related to the workpiece coordinate system for each of the plurality of robots, the vision processing computer is configured to initialize the information related to the robot corresponding to the robot controller based on initialization instructions transmitted from the robot controller, the robot controller of the robot belonging to a group composed of the plurality of robots that are application targets of the workpiece coordinate system is configured to transmit a query to the vision processing computer at an uncertain timing, respectively, to inquire whether initialization of information related to the workpiece coordinate system has been completed for all the robots belonging to the group, the vision processing computer is configured to transmit a standby instruction to the robot controller of the source of the query as a response to the query in a state in which there is a robot belonging to the group for which initialization of information related to the workpiece coordinate system has not been completed, when the query is received from the robot controller corresponding to the robot belonging to the group, the vision processing computer is configured to transmit a signal indicating completion of initialization of information for all the robots belonging to the group, i.e., a completion signal, to the robot controller of the source of the query as a response to the query in a state in which initialization of information related to the workpiece coordinate system has been completed for all the robots belonging to the group, when the query is received from the robot controller corresponding to the robot belonging to the group, each of the robot controllers is configured to re-query the vision processing computer when the standby instruction is received, and on the other hand, not to re-query the vision processing computer when the completion signal is received.

2. The robot system according to claim 1, wherein the vision processing computer is configured to transmit a standby instruction to the robot controller of the source of the request as a response to the request in a state in which there is the vision sensor for which measurement of the measurement object has not been completed, when the request is received from the robot controller, each of the robot controllers is configured to re-request the vision processing computer when the standby instruction is received.

3. The robot system according to claim 1 or 2, wherein the plurality of robots comprise: a plurality of first robots configured to simultaneously operate on workpieces located at a first position; and a plurality of second robots configured to simultaneously operate on workpieces transported from the first position to a second position after operation by the first robots, the plurality of vision sensors include a plurality of first vision sensors and a plurality of second vision sensors, the robot controllers of the first robots, the robot controllers of the second robots, the first vision sensors, and the second vision sensors are communicable with the same vision processing computer, the vision processing computer calculates a workpiece coordinate system at the first position based on results of measuring the measurement object appearing in the vision information acquired by the plurality of first vision sensors, and calculates a workpiece coordinate system at the second position based on results of measuring the measurement object appearing in the vision information acquired by the plurality of second vision sensors, each of the robot controllers of the first robots corrects the operation of the corresponding first robot based on the results requested from the vision processing computer, each of the robot controllers of the second robots corrects the operation of the corresponding second robot based on the results requested from the vision processing computer.

4. The robot system according to claim 3, wherein in a state in which the first vision sensors in which measurement of the measurement object is not completed exist, in a case where a request is received from the robot controllers of the first robots, the vision processing computer transmits a standby instruction to the robot controller of the request source as a reply to the request, in a case where the standby instruction is received, each of the robot controllers of the first robots again inquires the vision processing computer, in a state in which the second vision sensors in which measurement of the measurement object is not completed exist, in a case where a request is received from the robot controllers of the second robots, the vision processing computer transmits a standby instruction to the robot controller of the request source as a reply to the request, in a case where the standby instruction is received, each of the robot controllers of the second robots again inquires the vision processing computer.

5. The robot system according to claim 1 or 2, wherein the vision processing computer is communicable with a setting computer, the vision processing computer is capable of changing settings related to calculation of the workpiece coordinate system by transmission from the setting computer.

6. A robot control method, which is a robot control method in a robot system including: a plurality of robots configured to simultaneously operate on the same workpiece; a robot controller configured corresponding to each of the robots; a plurality of vision sensors capable of acquiring vision information; and a vision processing computer that processes information acquired by the vision sensors, the robot control method including: In a first step, each of a plurality of said vision sensors acquires vision information related to a measurement object possessed by said workpiece and transmits said vision information to a common said vision processing computer; In a second step, said vision processing computer calculates a workpiece coordinate system based on results of measurement of said measurement object appearing in said vision information received from a plurality of said vision sensors; In a third step, each of a plurality of said robot controllers transmits a request to the common said vision processing computer and corrects the action of said robot corresponding to the robot controller based on a reply received from said vision processing computer, Said vision processing computer stores information about said workpiece coordinate system in relation to a plurality of said robots for each robot, Said vision processing computer initializes said information about said workpiece coordinate system in relation to said robot corresponding to said robot controller based on an initialization instruction transmitted from said robot controller, Said robot controllers of said robots belonging to a group composed of a plurality of said robots that are application objects of said workpiece coordinate system respectively transmit inquiries to said vision processing computer at uncertain timings, inquiring whether initialization of information about said workpiece coordinate system has been completed for all said robots belonging to the group, Said vision processing computer, in a state in which there are robots belonging to said group for which initialization of information about said workpiece coordinate system has not been completed, transmits a standby instruction to the said robot controller that is the source of the inquiry as a reply to the inquiry in the case where an inquiry is received from said robot controller corresponding to a robot belonging to the group, Said vision processing computer, in a state in which initialization of information about said workpiece coordinate system has been completed for all said robots belonging to said group, transmits a signal indicating completion of initialization of information about all said robots belonging to said group, i.e., a completion signal, to the said robot controller that is the source of the inquiry as a reply to the inquiry in the case where an inquiry is received from said robot controller corresponding to a robot belonging to the group, Each said robot controller, in the case where a standby instruction is received, again inquires of said vision processing computer, and on the other hand, in the case where a completion signal is received, does not again inquire of said vision processing computer. ​

Images