Robot Control System

Abstract

To restrict an increase in number of required cameras and an increase in cycle time.SOLUTION: A robot control system (10) comprises: a robot (20) that performs work on an object (W) in each of a plurality of work areas; a movable camera (30) that moves separately from the robot and can capture images of respective objects in the plurality of work areas; and an object recognition unit (50) that can recognize a position of each object, based on an image captured from a predetermined position (P1, P2) by the movable camera. The movable camera acquires a predetermined image captured with a reference object (B1) and the object taken in a field of view. The object recognition unit registers in advance a position of the reference object recognized based on an image captured from a predetermined position, as a reference position, calculates an amount of deviation of a position of the movable camera from the predetermined position based on the reference position and the position of the reference object recognized based on the predetermined image, and recognizes a position of the object based on the amount of deviation and the predetermined image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a robot control system including a camera and a robot. [Background technology]

[0002] Conventionally, there is a fixed camera type robot control system in which a camera attached to a fixed structure installed above the work area of ​​the robot captures an image of a workpiece in the work area, and the image captured by the camera is processed to recognize the position of the workpiece (see Patent Document 1). Patent Document 1 also describes an on-hand camera type robot control system in which a camera is attached to the robot. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] WO2020 / 121399 publication Summary of the Invention [Problem to be solved by the invention]

[0004] However, there are cases where a robot performs work on each work in both the supply section and the storage section (multiple work areas), such as when the robot picks up a work (object) from a work supply section and carries it to a work storage section to store it. In this case, a fixed camera-type robot control system requires a camera for each work area. Also, an on-hand camera-type robot control system cannot perform work on a work until the camera attached to the robot captures an image of the work and recognizes its position, which lengthens the cycle time from picking up the work to storing it.

[0005] The present invention has been made to solve the above-mentioned problems, and its main purpose is to prevent an increase in the number of cameras required and to prevent a longer cycle time, even when a robot performs work on objects in multiple work areas. [Means for solving the problem]

[0006] The first means for solving the above problem is: a robot (20) that performs work on a target object (W) in each of a plurality of work areas; a movable camera (30) that moves separately from the robot and can capture images of the objects in each of the plurality of work areas; an object recognition unit (50) capable of recognizing the position of an object based on an image captured by the movable camera from a predetermined position (P1, P2); A robot control system (10) comprising: The movable camera acquires a predetermined image captured by taking a reference object (B1) serving as a position reference and the target object in its field of view, the object recognition unit registers in advance the position of the reference object recognized based on an image captured from the predetermined position as a reference position, calculates a deviation amount of the position of the movable camera from the predetermined position based on the reference position and the position of the reference object recognized based on the predetermined image, and recognizes the position of the target object based on the deviation amount and the predetermined image; The robot performs an operation on the object based on the position of the object recognized by the object recognition unit.

[0007] According to the above configuration, the robot performs work on objects in multiple work areas. Therefore, in order to recognize the position of each object in each work area, it is necessary to capture images of the objects in each work area. In this regard, the movable camera can move separately from the robot and capture images of the objects in each of the work areas. The object recognition unit can recognize the position of the object based on images captured by the movable camera from a predetermined position. Therefore, by moving the movable camera to a predetermined position and capturing images of the objects in each of the work areas, it is possible to recognize the position of the object in each of the work areas. Therefore, even when the robot performs work on objects in each of the work areas, it is possible to prevent an increase in the number of cameras required.

[0008] The movable camera also acquires a predetermined image captured with a reference object serving as a position reference and the target object in its field of view. Therefore, the predetermined image includes both the reference object and the target object. The object recognition unit pre-registers the position of the reference object recognized based on an image captured from a predetermined position as a reference position. If the position of the reference object recognized based on an image captured by the movable camera deviates from the reference position, the amount of deviation is due to the amount of deviation of the movable camera from the predetermined position. Therefore, even if the position of the movable camera deviates from the predetermined position, the object recognition unit can recognize the position of the object based on the captured image by correcting the amount of deviation. In this regard, the object recognition unit calculates the amount of deviation of the movable camera from the predetermined position based on the reference position and the position of the reference object recognized based on the predetermined image, and recognizes the position of the target object based on the amount of deviation and the predetermined image. Therefore, even if the movable camera is unable to accurately return to the predetermined position, the position of the target object can be accurately recognized. The robot can then accurately perform work on the object based on the position of the object recognized by the object recognition unit.

[0009] Furthermore, for example, the position of an object can be recognized by a movable camera in one work area, while a robot can work on the object in another work area where the position for storing the object has already been recognized. That is, the recognition of the position of the object by the movable camera and the work on the object by the robot can be performed in parallel. Therefore, it is possible to prevent the cycle time from starting to recognize the position of the object by the movable camera to finishing the work on the object from becoming long in multiple work areas.

[0010] In a second aspect, a plurality of the robots are provided, the reference object is provided in each of the work areas of the plurality of robots, and the movable camera is capable of capturing images of the target objects in each of the work areas of the plurality of robots. With this configuration, a single movable camera can recognize the positions of the target objects in the work of the plurality of robots. Therefore, the number of cameras required can be further reduced.

[0011] In a third means, when the reference object and the target object are not in the field of view, the movable camera moves to a position where the reference object and the target object are in the field of view and acquires the predetermined image. With this configuration, even when the movable camera has moved to a position where the reference object and the target object are not in the field of view, the movable camera can move and change its position to acquire a predetermined image captured with the reference object and the target object in the field of view.

[0012] Specifically, as in the fourth means, the object recognition unit can perform calibration to correct the correspondence between the coordinate system of the movable camera and the coordinate system of the robot, thereby making it possible to recognize the position of the object in the coordinate system of the robot as the position of the object based on an image captured by the movable camera from the specified position.

[0013] In a fifth aspect, the calibration is performed using a 3D scanner (70) that detects the position of an object with higher accuracy than the movable camera, and the reference position is registered in advance using the 3D scanner. This configuration allows for accurate calibration of the correspondence between the coordinate system of the movable camera and the coordinate system of the robot. Furthermore, the reference position, which is the position of the reference object recognized based on an image captured from the predetermined position, can be more accurately acquired. Therefore, the amount of deviation of the position of the movable camera from the predetermined position can be more accurately calculated, and the position of the target object can be more accurately recognized.

[0014] In a sixth aspect, the robot grasps the object in one of the plurality of work areas, moves the object to another work area, and releases it. With this configuration, even when grasping the object in one of the plurality of work areas and moving the object to another work area to release (place) it, a camera is not required for each work area. Furthermore, for example, the robot can grasp the object in a first work area where the position of the object has already been recognized, and a movable camera can recognize the position where the object should be placed in a second work area where the object is released. As a result, the number of required cameras is prevented from increasing, and the cycle time is also prevented from becoming longer.

[0015] In a seventh aspect, before grasping the object in one of the plurality of work areas, the object recognition unit calculates an amount of deviation of the position of the movable camera from the predetermined position based on the reference position and the position of the reference object recognized based on the predetermined image, and recognizes the position of the object based on the amount of deviation and the predetermined image, and the robot performs the task of grasping the object based on the position of the object recognized by the object recognition unit. With this configuration, the position of the object can be accurately recognized before grasping the object, and the object can be accurately grasped based on the accurately recognized position of the object.

[0016] When a robot grasps an object in one work area, the relative position between the robot and the object varies. In this case, when the grasped object is moved to another work area and then released (placed), the position where the object is placed may vary.

[0017] In this regard, in an eighth aspect of the present invention, after grasping the object in one of the plurality of work areas, the object recognition unit calculates a deviation amount of the position of the movable camera from the predetermined position based on the reference position and the position of the reference object recognized based on the predetermined image, recognizes the position of the object based on the deviation amount and the predetermined image, and the robot performs an operation of moving the object from one work area to another work area and releasing it based on the position of the object recognized by the object recognition unit. With this configuration, the position of the object can be accurately recognized after grasping, and based on the accurately recognized position of the object, the object can be moved from one work area to another work area and released at an accurate position (placed in an accurate position).

[0018] In a ninth aspect, the reference object is a block made of an asymmetric polyhedron arranged around the target object. With this configuration, the object recognition unit can easily capture the features of the reference object, and therefore can easily recognize the position of the reference object.

[0019] In a tenth aspect, the reference object is a part of a facility existing around the target object. With this configuration, a part of the facility existing around the target object can be used as the reference object, eliminating the need to place a special block or the like as the reference object. [Brief explanation of the drawings]

[0020] [Figure 1] Schematic diagram of the robot control system. [Figure 2] 10 is a flowchart showing the procedure for initial setting. [Figure 3]10 is a flowchart showing a control procedure when the robot is operating. DETAILED DESCRIPTION OF THE INVENTION

[0021] Hereinafter, an embodiment of a robot control system that takes out a workpiece from a supply unit, processes it, and stores it in a storage unit will be described with reference to the drawings.

[0022] As shown in FIG. 1, the robot control system 10 includes a robot 20, a camera 30, a slider 40, an object recognition unit 50, a robot control device 60, and the like.

[0023] The robot 20 is, for example, a single-arm, vertically articulated robot. The robot 20 grasps (grabs) the workpiece W in the supply section and moves it to the processing section, where it processes the workpiece W using a cutting machine or the like, and then moves the workpiece W to the storage section and places the workpiece W on each placement section Sn of the storage case H (releases the workpiece W). In other words, the robot 20 performs work on each of the objects in a plurality of work areas. A grasping tool 22 (work tool) is attached to the tip of the arm section 21 of the robot 20.

[0024] The workpieces W (objects) are three-dimensional objects of a predetermined shape. Multiple workpieces W of the same shape are accommodated in a returnable box Tb in a misaligned state. In other words, the position and orientation of each workpiece W are unspecified.

[0025] The camera 30 (movable camera) is a binocular camera that captures three-dimensional images. The camera 30 is attached to the movable part 41 of the slider 40. The slider 40 moves the movable part 41 back and forth linearly and can stop the movable part 41 at any position on the line. The driving state of the slider 40 is controlled, for example, by the object recognition unit 50. The camera 30 is attached to the movable part 41 in an orientation that allows it to move together with the movable part 41 and capture images of the supply part, processing part, and storage part (work area). The field of view of the camera 30 is such that, in the supply part, all of the workpieces W and reference blocks B1 in the return box Tb can be seen. Furthermore, in the storage part, the field of view of the camera 30 is such that all of the placement parts Sn and reference blocks B3 in the storage case H can be seen. The reference blocks B1 and B3 (reference objects) serve as positional references and are, for example, blocks (three-dimensional objects) made of asymmetric polyhedrons (shapes with distinctive features) arranged near (around) the workpiece W and the placement section Sn, respectively. The placement section Sn (the position for accommodating the object) is formed, for example, as a recess corresponding to the shape of the workpiece W after machining. In other words, the camera 30 moves separately from the robot 20 and can capture images of each of the objects in the multiple work areas. The camera 30 has a function for automatically adjusting the focus position when capturing an image.

[0026] The camera 30 is moved between a first imaging position P1 and a second imaging position P2. At the first imaging position P1, the camera 30 captures a three-dimensional image of all the workpieces W and the reference block B1 in the return box Tb in the same imaging field of view. At the second imaging position P2, the camera 30 captures a three-dimensional image of all the placement sections Sn and the reference block B3 in the storage case H in the same imaging field of view. The first imaging position P1 and the second imaging position P2 can be set in advance as positions where the camera 30 can capture images of each work area, or can be set each time based on the three-dimensional image.

[0027] The object recognition unit 50 includes an image processing unit, a storage unit, an input / output interface, and the like. The object recognition unit 50 performs calibration to calibrate the correspondence between the coordinate system of the camera 30 and the coordinate system of the robot 20. The object recognition unit 50 performs calibration while moving the camera 30 to a first image capture position P1 and a second image capture position P2 (predetermined positions). For example, a predetermined marker B2 is attached to the tip of the arm unit 21 of the robot 20, and the marker B2 is captured by the camera 30 while the arm unit 21 is operating. The marker B2 may be a calibration board printed with a predetermined dot pattern or a block similar to the reference blocks B1 and B3. As a result, the object recognition unit 50 acquires parameters for converting the position and orientation (position) of an object in the coordinate system of the camera 30 recognized based on an image captured by the camera 30 from the image capture position during calibration into the position and orientation (position) of the object in the coordinate system of the robot 20. The parameters include information on the first image capture position P1 and the second image capture position P2. That is, by performing calibration, the object recognition unit 50 is able to recognize the position and orientation of an object in the coordinate system of the robot 20 as the position and orientation of the object based on an image captured by the camera 30 from a predetermined position. Hereinafter, the position and orientation of an object in the coordinate system of the robot 20 may be simply referred to as the position and orientation of the object. The object recognition unit 50 recognizes (calculates) the position and orientation of the object using a three-dimensional image (three-dimensional image data) acquired by the camera 30. Specifically, the object recognition unit 50 calculates the position, orientation (direction), etc. of the object from the three-dimensional image acquired by the camera 30. The object recognition unit 50 recognizes the positions and orientations of the workpiece W, reference blocks B1 and B3, and the placement unit Sn in the storage case H based on the three-dimensional image.

[0028] The robot control device 60 is equipped with a CPU, ROM, RAM, an input / output interface, etc. The robot control device 60 controls the swing and rotation of the arm unit 21 of the robot 20, the operation of the gripping tool 22, etc. The robot control device 60 controls each operation of the robot 20 based on the position and orientation of the workpiece W recognized by the object recognition unit 50. Note that each operation of the robot 20 is taught in advance and stored in the robot control device 60, so the robot control device 60 automatically corrects the teaching data based on the position and orientation of the workpiece W calculated by the object recognition unit 50.

[0029] Next, the procedure for initial setting performed before operating the robot 20 will be described with reference to the flowchart in Fig. 2. This series of processes is executed by the object recognition unit 50.

[0030] First, calibration of the robot 20 and the camera 30 is performed (S10). Specifically, the above calibration is performed in a state in which the camera 30 is moved to the first imaging position P1 by the slider 40. At this time, the three-dimensional image captured by the camera 30 includes all of the workpieces W and reference blocks B1 inside the returnable box Tb. Note that the returnable box Tb and reference blocks B1 may be positioned after the calibration so that the three-dimensional image captured by the camera 30 includes all of the workpieces W and reference blocks B1 inside the returnable box Tb.

[0031] Next, the shape data of the reference block B1 is registered (S11). Specifically, the three-dimensional shape data of the reference block B1 is acquired and registered by processing the three-dimensional image captured by the camera 30. The three-dimensional shape data is acquired as point cloud data representing the positions of each point of the object.

[0032] Next, the positional relationship between the reference block B1 and the camera 30 is acquired (S12). Specifically, the position and orientation of the reference block B1 in the coordinate system of the camera 30 is recognized based on a three-dimensional image of the reference block B1 captured by the camera 30 from the first imaging position P1. The position and orientation (position) of the reference block B1 in the coordinate system of the camera 30 is converted to the position and orientation (position) of the reference block B1 in the coordinate system of the robot 20 using the above parameters. Then, the position and orientation (reference position) of the reference block B1 in the coordinate system of the robot 20 is acquired as the positional relationship between the reference block B1 and the camera 30. In other words, the position and orientation of the reference block B1 recognized based on the three-dimensional image of the reference block B1 captured by the camera 30 from the first imaging position P1 is acquired as the positional relationship between the reference block B1 and the camera 30.

[0033] Next, the shape data of the target object is registered (S13). Specifically, the three-dimensional shape data of the workpiece W is acquired and registered by processing the three-dimensional image captured by the camera 30. Note that, since multiple workpieces W have the same shape, it is sufficient to acquire the three-dimensional shape data of one representative workpiece W.

[0034] Next, the master data is saved (S14). Specifically, the three-dimensional shape data of the reference block B1, the position and orientation of the reference block B1 in the coordinate system of the robot 20, and the three-dimensional shape data of the workpiece W are saved as master data to be used when the robot 20 is in operation. Then, this series of processes is ended (END). Furthermore, the object recognition unit 50 performs the same processes as S10 to S14 described above with the camera 30 moved to the second imaging position P2 by the slider 40. Here, the reference block B3 is used as the reference block, and the placement section Sn in the storage case H is treated as an object.

[0035] Next, the control procedure when the robot 20 is in operation will be described with reference to the flowchart in Fig. 3. In this series of processes, the processes of S20 to S26 are executed by the object recognition unit 50, and the process of S27 is executed by the robot control device 60. Here, an example will be described in which, while the robot 20 is working in the storage unit, the camera 30 is moved to the first image capturing position P1 by the slider 40 and this series of processes is performed. When the camera 30 is moved from the first image capturing position P1 to another position by the slider 40 and then moved back to the first image capturing position P1, a deviation (error) occurs between the first image capturing position P1 and the actual position of the camera 30.

[0036] First, the master data is read (S20). Specifically, the master data saved in the initial setting is read.

[0037] Next, it is determined whether the reference block and the target object are in the same imaging field of view (S21). Specifically, it is determined whether the reference block B1 and all of the workpieces W are in the same imaging field of view (one imaging field of view). This determination can be made by so-called matching based on the 3D image captured by the camera 30 and the master data (3D shape data of the reference block B1 and the 3D shape data of the workpieces W). If it is determined in this determination that the reference block B1 and all of the workpieces W are not in the same imaging field of view (S21: NO), the camera 30 is moved to another imaging position by the slider 40 (S22). Specifically, the movable part 41 of the slider 40 is moved a predetermined amount along one of the linear trajectories of the slider 40. Note that the direction and amount of movement of the movable part 41 to bring the reference block and the target object into the same imaging field of view can also be determined based on the 3D image captured in the above determination. Thereafter, the process is executed again from S21. That is, when the reference block B1 and the workpiece W are not in the same field of view, the camera 30 moves to a position where the reference block B1 and the workpiece W are in the same field of view and acquires a three-dimensional image (predetermined image).

[0038] If it is determined in S21 that the reference block B1 and all the workpieces W are within the same imaging field of view (S21: YES), the amount of deviation of the movable camera is calculated (S23). Specifically, the amount of deviation of the current position (position) of the camera 30 from the first imaging position P1 is calculated. Here, if the position of the reference block B1 recognized based on the three-dimensional image (predetermined image) captured by the camera 30 at the current position is deviated from the position (reference position) of the reference block B1 in the master data, the amount of deviation corresponds to (is caused by) the amount of deviation of the current position of the camera 30 from the first imaging position P1. Therefore, the amount of deviation of the current position of the camera 30 from the first imaging position P1 is calculated based on the position of the reference block B1 recognized based on the three-dimensional image captured by the camera 30 at the current position and the position of the reference block B1 in the master data.

[0039] Next, the amount of deviation of the movable camera is corrected (S24). Specifically, the parameters for converting the position and orientation (position) of the object in the coordinate system of camera 30 into the position and orientation (position) of the object in the coordinate system of robot 20 are corrected based on the amount of deviation (the amount of deviation is reflected in the parameters).

[0040] Next, the position and orientation (position) of the object is recognized (S25). Specifically, the position and orientation of the workpiece W in the coordinate system of the camera 30 is recognized based on a three-dimensional image of the workpiece W captured by the camera 30 from its current position. At this time, the workpiece W to be grasped is selected from among the multiple workpieces W, and the position and orientation of the selected workpiece W is recognized. Then, the position and orientation of the workpiece W in the coordinate system of the camera 30 is converted to the position and orientation of the workpiece W in the coordinate system of the robot 20 using the corrected parameters. That is, based on the position (reference position) of the reference block B1 in the master data and the position of the reference block B1 recognized based on a three-dimensional image (predetermined image) captured with the reference block B1 and the workpiece W in view, the amount of deviation of the current position (position) of the camera 30 from the first imaging position P1 (predetermined position) is calculated, and the position and orientation (position) of the workpiece W is recognized based on the amount of deviation and the predetermined image.

[0041] Next, the position and orientation (position) of the target object is transmitted to the robot control device 60 (S26). Specifically, the object recognition unit 50 transmits the recognized position and orientation of the workpiece W to the robot control device 60.

[0042] Next, the operation of the robot 20 is controlled (S27). Specifically, the robot control device 60 operates the arm unit 21 and the gripping tool 22 to grasp the workpiece W based on the received position and orientation of the workpiece W (performs work on the workpiece W). Meanwhile, while the robot 20 is working in the supply unit, the object recognition unit 50 moves the camera 30 to the second imaging position P2 and executes the same processes as in S20 to S26 for the reference block B3 and each placement portion Sn of the storage case H. After grasping the workpiece W in the supply unit, the workpiece W is moved to the processing unit and processed by a cutting machine or the like. After processing the workpiece W, the workpiece W is moved to the storage unit and placed on each placement portion Sn of the storage case H. At this time, the robot control device 60 places the workpiece W on the target placement portion Sn based on the position and orientation of each placement portion Sn of the storage case H that the robot 20 recognized in advance when working in the supply unit. Then, while the robot 20 is working in the storage section, the object recognition unit 50 moves the camera 30 back to the first imaging position P1 and executes the processes of S20 to S26 again. Thereafter, the above processes are repeated until work on all the workpieces W is completed.

[0043] The present embodiment described above in detail has the following advantages.

[0044] The camera 30 moves separately from the robot 20 and is capable of capturing images of the workpieces W and the placement portions Sn of the storage cases H in each of the multiple work areas. The object recognition unit 50 is then able to recognize the position of the object based on images captured by the camera 30 from the first imaging position P1 and the second imaging position P2. Therefore, by moving the camera 30 to the first imaging position P1 and the second imaging position P2 to capture images of the workpieces W and the placement portions Sn of the storage cases H in each of the multiple work areas, the positions of the workpieces W and the placement portions Sn of the storage cases H in each of the multiple work areas can be recognized. Therefore, even when the robot 20 performs work on the workpieces W in each of the multiple work areas, it is possible to prevent an increase in the number of cameras required.

[0045] The object recognition unit 50 calculates the amount of deviation of the position of the camera 30 from the first imaging position P1 based on the reference position of the reference block B1 recognized based on a three-dimensional image captured from the first imaging position P1 and the position of the reference block B1 recognized based on a predetermined image captured with the reference block B1 and the workpiece W in the same field of view. The object recognition unit 50 then recognizes the position of the workpiece W based on the amount of deviation and the predetermined image. Therefore, even if the camera 30 cannot accurately return to the first imaging position P1, the position of the workpiece W can be accurately recognized. The robot 20 can then accurately perform work on the workpiece W based on the position of the workpiece W recognized by the object recognition unit 50. The same applies to the second imaging position P2, the reference block B3, and the placement portion Sn of the storage case H.

[0046] The position of the workpiece W can be recognized by the camera 30 in the supply section, and the robot 20 can perform work on the workpiece W in the storage section where the position of the placement section Sn of the storage case H has already been recognized. In other words, the recognition of the position of the workpiece W by the camera 30 and the work on the workpiece W on the placement section Sn by the robot 20 can be performed in parallel. Therefore, in multiple work areas, it is possible to prevent the cycle time from when the camera 30 starts to recognize the position of the workpiece W to when the work on the workpiece W is completed from becoming long. Furthermore, the position of the placement section Sn of the storage case H can be recognized by the camera 30 in the storage section, and the robot 20 can perform work on the workpiece W in the supply section where the position of the workpiece W has already been recognized.

[0047] When the reference block B1 and the workpiece W are not in the same imaging field of view, the camera 30 moves to a position where the reference block B1 and the workpiece W are in the same imaging field of view and acquires a predetermined image. With this configuration, even if the camera 30 has moved to a position where the reference block B1 and the workpiece W are not in the same imaging field of view, the camera 30 can move and change its position to acquire a predetermined image in which the reference block B1 and the workpiece W are in the same imaging field of view.

[0048] The robot 20 grasps the workpiece W in the supply section of the multiple work areas, moves the workpiece W to the storage section, and releases it. With this configuration, even when grasping the workpiece W in one of the multiple work areas and moving the workpiece W to another work area to release (place) it, a camera is not required for each work area. Furthermore, the robot 20 grasps the workpiece W in the supply section, which has already recognized the position of the workpiece W, and the camera 30 in the storage section, where the workpiece W is released, can recognize the position of the placement section Sn of the storage case H. As a result, the number of required cameras is prevented from increasing, and the cycle time is also prevented from becoming longer.

[0049] Before the robot 20 grasps the workpiece W in the supply section of the multiple work areas, the object recognition unit 50 calculates the amount of deviation of the current position of the camera 30 from the first imaging position P1 based on the reference position and the position of the reference block B1 recognized based on a predetermined image, and recognizes the position and orientation of the workpiece W based on the amount of deviation and the predetermined image. The robot 20 then performs the task of grasping the workpiece W based on the position and orientation of the workpiece W recognized by the object recognition unit 50. With this configuration, the position and orientation of the workpiece W can be accurately recognized before grasping the workpiece W, and the workpiece W can be accurately grasped based on the accurately recognized position and orientation of the workpiece W.

[0050] The reference block B1 is an asymmetric polyhedron block arranged around the workpiece W. This configuration makes it easier for the object recognition unit 50 to capture the characteristics of the reference block B1, and therefore makes it easier to recognize the position of the reference block B1.

[0051] The above embodiment can be modified as follows: The same parts as those in the above embodiment are denoted by the same reference numerals and the description thereof will be omitted.

[0052] The three-dimensional shape data of the reference blocks B1 and B3 can be formed by combining three-dimensional image data captured by the camera 30 at the first imaging position P1 and the second imaging position P2, respectively, with three-dimensional CAD data of the reference blocks B1 and B3. Also, the three-dimensional shape data of the workpiece W and the mounting portion Sn of the storage case H can be formed by combining three-dimensional image data captured by the camera 30 at the first imaging position P1 and the second imaging position P2, respectively, with three-dimensional CAD data of the workpiece W and the mounting portion Sn.

[0053] 1, the object recognition unit 50 can also perform calibration using a 3D scanner 70, which has higher accuracy in detecting the position of an object than the camera 30. In this case, the object recognition unit 50 performs calibration with the 3D scanner 70 placed at the first imaging position P1 and the second imaging position P2 (predetermined positions). Then, the position and orientation (reference positions) of the reference blocks B1 and B3 recognized based on the 3D images captured by the 3D scanner 70 from the first imaging position P1 and the second imaging position P2, respectively, are acquired and registered as the positional relationship between the reference blocks B1 and B3 and the camera 30.

[0054] This configuration allows for accurate calibration of the correspondence between the coordinate system of the camera 30 and the coordinate system of the robot 20. Furthermore, it is possible to more accurately acquire the reference positions, which are the positions of the reference blocks B1 and B3 recognized based on the images captured from the first imaging position P1 and the second imaging position P2, respectively. Therefore, it is possible to more accurately calculate the amount of deviation of the position of the camera 30 from the first imaging position P1 and the second imaging position P2, and therefore it is possible to more accurately recognize the positions of the workpiece W and the placement portion Sn of the storage case H.

[0055] When the robot 20 grasps the workpiece W in the supply section (one work area), variations occur in the relative position between the robot 20 and the workpiece W. In that case, when the grasped workpiece W is moved to the storage section (another work area) and released (placed), there is a risk that variations will occur in the position where the workpiece W is placed.

[0056] Therefore, the object recognition unit 50 may, after grasping the workpiece W in a supply unit among the plurality of work areas, calculate the amount of deviation of the current position of the camera 30 from the first imaging position P1 based on the reference position and the position of the reference block B1 recognized based on the predetermined image, and recognize the position and orientation (position) of the workpiece W based on the amount of deviation and the predetermined image. Then, after grasping the workpiece W, the robot 20 may perform an operation of moving the workpiece W from the supply unit to the storage unit and releasing it based on the position and orientation of the workpiece W recognized by the object recognition unit 50.

[0057] According to this configuration, the position and orientation of the workpiece W can be accurately recognized after the workpiece W is grasped, and based on the accurately recognized position and orientation of the workpiece W, the workpiece W can be moved from the supply unit to the storage unit and released at an accurate position (placed at an accurate position). Note that after the workpiece W is grasped in the supply unit, it is also possible to move the workpiece W from the supply unit to the processing unit and perform processing based on the position and orientation of the workpiece W recognized by the object recognition unit 50.

[0058] The amount of deviation of the current position of the camera 30 from the first image capturing position P1 is calculated based on the position of the reference block B1 recognized based on the 3D image captured by the camera 30 at its current position and the position of the reference block B1 in the master data. The current position of the camera 30 is calculated based on this amount of deviation and the information on the first image capturing position P1 and the second image capturing position P2 included in the parameters for converting the coordinates. Then, the position and orientation (location) of the workpiece W can also be recognized based on the current position of the camera 30 and the specified image.

[0059] As shown by the dashed line in FIG. 1, the robot control system 10 may include multiple robots 20. In this case, reference blocks B1 and B3 (reference objects) are provided in multiple work areas of the multiple robots 20, respectively. The camera 30 can capture images of the workpieces W (objects) and the placement portions Sn of the storage cases H in the multiple work areas of the multiple robots 20. Specifically, the movable portion 41 of the slider 40 can move to the multiple work areas of the multiple robots 20. With this configuration, a single camera 30 can recognize the positions of objects in the work of the multiple robots 20. Therefore, the number of required cameras can be further reduced.

[0060] In the above case, when one reference block is imaged by the camera 30, if the placement section Sn for the workpiece W and the storage case H is within the imaging field of view, one reference block may be arranged in common for the supply section and the storage section. One reference block may also be arranged for each robot 20.

[0061] A part of the equipment (for example, a part of a device or a part of a support structure) existing around the placement section Sn of the workpiece W and the storage case H can also be used as a reference object that serves as a positional reference. The part of the equipment used as the reference object is preferably an object whose position and shape do not change and whose features are easy to recognize by the camera 30. With this configuration, a part of the equipment existing around the placement section Sn of the workpiece W and the storage case H can be used as the reference object, eliminating the need to place a special block or the like as the reference object.

[0062] The object recognition unit 50 may be included in the robot control device 60 or the camera 30.

[0063] The robot 20 may be a single-arm horizontal articulated robot, or a double-arm articulated robot.

[0064] The above-described embodiments and their modifications can be combined within the scope of possible combinations.

[0065] Characteristic configurations extracted from the above-described embodiment and modified examples will be described below. [Configuration 1] a robot (20) that performs work on a target object (W) in each of a plurality of work areas; a movable camera (30) that moves separately from the robot and can capture images of the objects in each of the plurality of work areas; an object recognition unit (50) capable of recognizing the position of an object based on an image captured by the movable camera from a predetermined position (P1, P2); A robot control system (10) comprising: The movable camera acquires a predetermined image captured by taking a reference object (B1) serving as a position reference and the target object in its field of view, the object recognition unit registers in advance the position of the reference object recognized based on an image captured from the predetermined position as a reference position, calculates a deviation amount of the position of the movable camera from the predetermined position based on the reference position and the position of the reference object recognized based on the predetermined image, and recognizes the position of the target object based on the deviation amount and the predetermined image; A robot control system in which the robot performs an operation on the object based on the position of the object recognized by the object recognition unit. [Configuration 2] A plurality of the robots are provided, the reference objects are provided in the plurality of work areas of the plurality of robots, respectively; 2. The robot control system of claim 1, wherein the movable camera is capable of capturing images of the objects in the work areas of the robots. [Configuration 3] The robot control system according to configuration 1 or 2, wherein when the reference object and the target object are not in the field of view, the movable camera moves to a position where the reference object and the target object are in the field of view and acquires the specified image. [Configuration 4] The robot control system of any one of configurations 1 to 3, wherein the object recognition unit performs calibration to calibrate the correspondence between the coordinate system of the movable camera and the coordinate system of the robot, thereby enabling the position of the object to be recognized as the position of the object in the coordinate system of the robot based on an image captured by the movable camera from the predetermined position. [Configuration 5] The calibration is performed using a 3D scanner (70) that has higher accuracy in detecting the position of an object than the movable camera; 5. The robot control system of claim 4, wherein the reference position is registered in advance using the 3D scanner. [Configuration 6] The robot control system according to any one of configurations 1 to 5, wherein the robot grasps the object in one of the plurality of work areas, moves the object to another work area, and releases it. [Configuration 7] the object recognition unit calculates a deviation amount of a position of the movable camera from a predetermined position based on the reference position and a position of the reference object recognized based on the predetermined image, before grasping the object in one of the plurality of work areas, and recognizes the position of the object based on the deviation amount and the predetermined image; 7. The robot control system according to configuration 6, wherein the robot performs a task of grasping the object based on the position of the object recognized by the object recognition unit. [Configuration 8] the object recognition unit, after grasping the object in one of the plurality of work areas, calculates an amount of deviation of a position of the movable camera from the predetermined position based on the reference position and a position of the reference object recognized based on the predetermined image, and recognizes the position of the object based on the amount of deviation and the predetermined image; The robot control system according to configuration 6 or 7, wherein the robot performs a task of moving the object from one work area to another work area and separating the object based on the position of the object recognized by the object recognition unit. [Configuration 9] 9. The robot control system according to any one of configurations 1 to 8, wherein the reference object is a block made of an asymmetric polyhedron arranged around the target object. [Configuration 10] 9. The robot control system according to any one of configurations 1 to 8, wherein the reference object is a part of a facility existing around the target object. [Explanation of symbols]

[0066] 10...robot control system, 20...robot, 30...camera, 50...object recognition unit, B1...reference block, B3...reference block, Sn...mounting unit, W...workpiece.

Claims

1. a robot (20) that performs work on a target object (W) in each of a plurality of work areas; a movable camera (30) that moves separately from the robot and is capable of capturing images of the objects in each of the plurality of work areas; an object recognition unit (50) capable of recognizing the position of an object based on an image captured by the movable camera from a predetermined position (P1, P2); A robot control system (10) comprising: The movable camera acquires a predetermined image captured by taking a reference object (B1) serving as a position reference and the target object in its field of view, the object recognition unit registers in advance the position of the reference object recognized based on an image captured from the predetermined position as a reference position, calculates a deviation amount of the position of the movable camera from the predetermined position based on the reference position and the position of the reference object recognized based on the predetermined image, and recognizes the position of the target object based on the deviation amount and the predetermined image; A robot control system in which the robot performs an operation on the object based on the position of the object recognized by the object recognition unit.

2. A plurality of the robots are provided, the reference objects are provided in the plurality of work areas of the plurality of robots, respectively; The robot control system according to claim 1 , wherein the movable camera is capable of capturing images of the objects in the plurality of work areas of the plurality of robots, respectively.

3. 3. The robot control system according to claim 1, wherein when the reference object and the target object are not within the field of view, the movable camera moves to a position where the reference object and the target object are within the field of view and acquires the specified image.

4. 3. The robot control system according to claim 1, wherein the object recognition unit performs calibration to calibrate the correspondence between the coordinate system of the movable camera and the coordinate system of the robot, thereby enabling the recognition of the position of the object in the coordinate system of the robot as the position of the object based on an image captured by the movable camera from the predetermined position.

5. The calibration is performed using a 3D scanner (70) that has higher accuracy in detecting the position of an object than the movable camera; The robot control system according to claim 4 , wherein the reference position is registered in advance using the 3D scanner.

6. The robot control system according to claim 1 or 2, wherein the robot performs an operation of grasping the object in one of the plurality of work areas, moving the object to another work area, and releasing the object.

7. the object recognition unit calculates a deviation amount of a position of the movable camera from a predetermined position based on the reference position and a position of the reference object recognized based on the predetermined image, before grasping the object in one of the plurality of work areas, and recognizes the position of the object based on the deviation amount and the predetermined image; The robot control system according to claim 6 , wherein the robot performs an operation of grasping the object based on the position of the object recognized by the object recognition unit.

8. the object recognition unit, after grasping the object in one of the plurality of work areas, calculates an amount of deviation of a position of the movable camera from the predetermined position based on the reference position and a position of the reference object recognized based on the predetermined image, and recognizes the position of the object based on the amount of deviation and the predetermined image; 7. The robot control system according to claim 6, wherein the robot performs an operation of moving the object from one work area to another work area and separating the object based on the position of the object recognized by the object recognition unit.

9. 3. The robot control system according to claim 1, wherein the reference object is a block made of an asymmetric polyhedron arranged around the target object.

10. The robot control system according to claim 1 or 2, wherein the reference object is a part of a facility existing around the target object.

Images