Robot control system

Abstract

The invention relates to a robot control system. A robot control system (10) is provided with: a robot (20) that performs work on an object (W) in each of a plurality of work regions; a movable camera (30) that moves while being separated from the robot, and that is capable of imaging objects in each of the plurality of work regions; and an object recognition unit (50) capable of recognizing the position of the object on the basis of images captured from the predetermined positions (P1, P2) by the movable camera. The movable camera acquires a predetermined image captured by a reference object (B1) and an object entering a field of view. The object recognition unit pre-registers, as a reference position, the position of a reference object recognized on the basis of an image captured from a prescribed position, and calculates the amount of displacement of the position of the movable camera from the prescribed position on the basis of the reference position and the position of the reference object recognized on the basis of the prescribed image. And recognizes the position of the object on the basis of the offset amount and the predetermined image.

Description

[0001] Cross - reference to related applications

[0002] This application is based on Japanese Application No. 2022 - 206736 filed on December 23, 2022, the content of which is hereby incorporated by reference. Technical field

[0003] The present disclosure relates to a robot control system including a camera and a robot. Background art

[0004] Conventionally, there has been a fixed - camera - type robot control system that uses a camera installed on a fixed structure above the work area of a robot to photograph a workpiece in the work area and processes the image captured by the camera to identify the position of the workpiece (see Patent Document 1). In addition, Patent Document 1 also describes a hand - held camera - type robot control system in which a camera is mounted on a robot.

[0005] Patent Document 1: WO2020 / 121399

[0006] However, there are cases where a robot takes out a workpiece from a supply unit of a workpiece (object), transports the workpiece to a storage unit of the workpiece and stores it, etc., and performs operations on the workpiece in the supply unit and the storage unit (multiple work areas) respectively. In this case, in a fixed - camera - type robot control system, a camera is required for each work area. In addition, in a hand - held camera - type robot control system, after photographing the workpiece with the camera mounted on the robot and identifying the position of the workpiece, only the operation on the workpiece is performed, and the cycle time from taking out the workpiece to storing it becomes long. Summary of the invention

[0007] The present disclosure has been made to solve the above problems, and its main object is to suppress an increase in the number of required cameras and suppress an increase in the cycle time even when a robot performs operations on an object in multiple work areas respectively.

[0008] A first aspect for solving the above problems is a robot control system, including:

[0009] A robot that performs operations on an object in multiple work areas respectively;

[0010] A movable camera that is separated from the robot and moves, and can photograph the object in the multiple work areas respectively; and

[0011] An object recognition unit that can recognize the position of an object based on an image captured from a specified position by the movable camera,

[0012] The above movable camera captures a prescribed image in which a reference object serving as a position reference and the above object enter the field of view.

[0013] The above object recognition unit previously registers the position of the above reference object recognized from the image captured from the above prescribed position as a reference position, calculates the offset of the position of the above movable camera from the above prescribed position based on the above reference position and the position of the above reference object recognized from the above prescribed image, and recognizes the position of the above object based on the above offset and the above prescribed image.

[0014] The above robot performs an operation on the above object based on the position of the above object recognized by the above object recognition unit.

[0015] According to the above structure, the robot performs operations on objects in multiple work areas respectively. Therefore, in order to recognize the positions of objects in multiple work areas respectively, it is necessary to capture the objects in multiple work areas separately. In this regard, the movable camera moves separately from the above robot and can capture the above objects in the above multiple work areas. Moreover, the object recognition unit can recognize the position of an object based on the image captured from a prescribed position by the above movable camera. Therefore, by moving the movable camera to a prescribed position and separately capturing the above objects in the above multiple work areas, it is possible to recognize the positions of objects in multiple work areas respectively. Therefore, even when the robot performs operations on objects in multiple work areas respectively, an increase in the number of required cameras can be suppressed.

[0016] In addition, the above movable camera captures a prescribed image in which a reference object serving as a position reference and the above object enter the field of view. Therefore, the prescribed image includes the reference object and the object. The above object recognition unit previously registers the position of the above reference object recognized from the image captured from a prescribed position as a reference position. Here, if the position of the reference object recognized from the image captured by the above movable camera is offset from the reference position, this offset is due to the offset of the position of the above movable camera from the above prescribed position. Therefore, even when the position of the movable camera is offset from the prescribed position, the object recognition unit can recognize the position of the object based on the captured image by using this offset for correction. In this regard, the above object recognition unit calculates the offset of the position of the above movable camera from the above prescribed position based on the above reference position and the position of the above reference object recognized from the above prescribed image, and recognizes the position of the above object based on the above offset and the above prescribed image. Therefore, even when the movable camera cannot accurately return to the prescribed position, the position of the object can be accurately recognized. Moreover, the above robot can accurately perform an operation on the above object based on the position of the above object recognized by the above object recognition unit.

[0017] Further, for example, it is possible to identify the position of an object using a movable camera in one work area, and perform an operation on the object using a robot in another work area where the position of the object to be accommodated has already been identified. That is, it is possible to concurrently perform the identification of the position of the object based on the movable camera and the operation on the object based on the robot. Therefore, in multiple work areas, it is possible to suppress an increase in the cycle time from the start of identifying the position of the object using the movable camera to the end of the operation on the object.

[0018] In the second aspect, a plurality of the above-described robots are provided, the above-described reference objects are respectively provided in the above-described plurality of work areas of the plurality of the above-described robots, and the movable camera can respectively photograph the above-described objects in the above-described plurality of work areas of the plurality of the above-described robots. According to such a configuration, it is possible to identify the position of the object in the operations of a plurality of robots using one movable camera. Therefore, it is possible to further reduce the number of cameras required.

[0019] In the third aspect, when the above-described reference object and the above-described object have not entered the field of view, the movable camera moves to a position where the above-described reference object and the above-described object enter the field of view and acquires the above-described specified image. According to such a configuration, even when the movable camera moves to a position where the above-described reference object and the above-described object have not entered the field of view, by changing the position by moving the movable camera, it is possible to acquire a specified image obtained by photographing when the reference object and the above-described object enter the field of view.

[0020] Specifically, as in the fourth aspect, the above-described object recognition unit performs calibration for correcting the correspondence between the coordinate system of the movable camera and the coordinate system of the robot, so that it is possible to identify the position of the object in the coordinate system of the robot as the position of the object based on the image captured from the above-described specified position using the movable camera.

[0021] In the fifth aspect, a 3D scanner with a higher accuracy in detecting the position of an object than the above-described movable camera is used for the above-described calibration, and the above-described reference position is pre-registered using the above-described 3D scanner. According to such a configuration, it is possible to accurately correct the correspondence between the coordinate system of the movable camera and the coordinate system of the robot. And it is possible to more accurately obtain the position of the above-described reference object, that is, the reference position, identified based on the image captured from the above-described specified position. Therefore, it is possible to more accurately calculate the offset of the position of the movable camera from the specified position, and further, it is possible to more accurately identify the position of the object.

[0022] In the sixth aspect, the above-described robot performs an operation of grasping the above-described object in one of the above-described multiple work areas, moving the above-described object to another work area, and releasing it. With such a configuration, even when performing the operation of grasping the above-described object in one of the above-described multiple work areas, moving the above-described object to another work area, and releasing (placing) it, cameras are not required in each work area. Additionally, for example, the operation of grasping the object using the robot can be performed in the first work area where the position of the object has already been recognized, and the position of the receptacle for the object can be recognized using the movable camera in the second work area where the object is released. As a result, an increase in the number of required cameras is suppressed, and an increase in the cycle time is suppressed.

[0023] In the seventh aspect, before grasping the above-described object in one of the above-described multiple work areas, the above-described object recognition unit calculates the offset of the position of the above-described movable camera from the above-described specified position based on the above-described reference position and the position of the above-described reference object recognized based on the above-described specified image, and recognizes the position of the above-described object based on the above-described offset and the above-described specified image. The above-described robot performs the operation of grasping the above-described object based on the position of the above-described object recognized by the above-described object recognition unit. With such a configuration, the position of the object can be accurately recognized before grasping the object, and the object can be accurately grasped based on the accurately recognized position of the object.

[0024] When the robot grasps an object in a work area, a deviation occurs in the relative position between the robot and the object. In this case, in the operation of moving the grasped object to another work area and releasing it, a deviation may occur in the position where the object is placed.

[0025] In this regard, in the eighth aspect, after grasping the above-described object in one of the above-described multiple work areas, the above-described object recognition unit calculates the offset of the position of the above-described movable camera from the above-described specified position based on the above-described reference position and the position of the above-described reference object recognized based on the above-described specified image, and recognizes the position of the above-described object based on the above-described offset and the above-described specified image. The above-described robot performs the operation of moving the above-described object from the above-described one work area to another work area and releasing it based on the position of the above-described object recognized by the above-described object recognition unit. With such a configuration, the position of the object can be accurately recognized after grasping the object, and based on the accurately recognized position of the object, the object can be moved from one work area to another work area and released at an accurate position (placed at an accurate position).

[0026] In the ninth aspect, the reference object is a block formed by an asymmetric polyhedron arranged around the object. With such a structure, the object recognition unit can easily capture the features of the reference object, and thus can easily recognize the position of the reference object.

[0027] In the tenth aspect, the reference object is a part of a device existing around the object. With such a structure, a part of the device existing around the object can be used as the reference object, so there is no need to arrange a special block or the like as the reference object. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] With reference to the accompanying drawings and the following detailed description, the above and other objects, features, and advantages of the present disclosure will become more apparent. The accompanying drawings are:

[0029] Figure 1 It is a schematic diagram of a robot control system,

[0030] Figure 2 It is a flowchart showing the steps of initial setting,

[0031] Figure 3 It is a flowchart showing the steps of control when the robot is working. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] Hereinafter, with reference to the accompanying drawings, an embodiment of a robot control system that takes out a workpiece from a supply unit, processes it, and stores it in a storage unit will be described.

[0033] As Figure 1 shown, the robot control system 10 includes a robot 20, a camera 30, a slider 40, an object recognition unit 50, a robot control device 60, and the like.

[0034] The robot 20 is, for example, a single-arm vertical multi-joint robot. The robot 20 holds (grasps) the workpiece W at the supply unit and moves it to the processing unit. After processing the workpiece W using a cutting machine or the like at the processing unit, the robot 20 moves the workpiece W to the storage unit and places the workpiece W (releases the workpiece W) on each placement unit Sn of the storage box H. That is, the robot 20 performs operations on the object in multiple work areas. A gripping tool 22 (working tool) is attached to the front end of the arm 21 of the robot 20.

[0035] The workpiece W (object) is a three-dimensional object with a specified shape. A plurality of workpieces W with the same shape are stored in the transport box Tb in a state where they are not neatly arranged. That is, the positions and orientations of the respective workpieces W are not determined.

[0036] The camera 30 (movable camera) is a binocular camera that captures three-dimensional images. The camera 30 is mounted on the movable part 41 of the slider 40. The slider 40 reciprocates the movable part 41 linearly and can stop the movable part 41 at any position on the straight line. The driving state of the slider 40 is controlled by, for example, the object recognition unit 50. The camera 30 moves together with the movable part 41 and is thus mounted on the movable part 41 in a direction capable of capturing the supply unit, the processing unit, and the storage unit (working area). The viewing angle of the camera 30 is such that in the supply unit, all the workpieces W and the reference block B1 within the transport box Tb can enter the viewing field. Also, the viewing angle of the camera 30 is such that in the storage unit, all the placement parts Sn and the reference block B3 within the storage box H can enter the viewing field. The reference blocks B1, B3 (reference objects) are objects serving as position references and are, for example, blocks (three-dimensional objects) formed of asymmetric polyhedra (shapes with characteristic parts) respectively arranged near (around) the workpiece W and the placement part Sn. The placement part Sn (position of the object to be stored) is formed, for example, as a concave part corresponding to the shape of the processed workpiece W. That is, the camera 30 moves separately from the robot 20 and can capture the objects in multiple working areas respectively. The camera 30 has a function of automatically adjusting the focus position during shooting.

[0037] The camera 30 moves between the first shooting position P1 and the second shooting position P2. At the first shooting position P1, the camera 30 captures a three-dimensional image by bringing all the workpieces W and the reference block B1 within the transport box Tb into the same shooting field of view. Additionally, at the second shooting position P2, the camera 30 captures a three-dimensional image by bringing all the placement parts Sn and the reference block B3 within the storage box H into the same shooting field of view. Furthermore, the first shooting position P1 and the second shooting position P2, as positions where the camera 30 can capture each working area, can be set in advance or set each time based on the three-dimensional image.

[0038] The object recognition unit 50 includes an image processing unit, a storage unit, an input / output interface, and the like. The object recognition unit 50 performs calibration to correct the correspondence between the coordinate system of the camera 30 and the coordinate system of the robot 20. The object recognition unit 50 performs calibration in a state where the camera 30 is moved to the first shooting position P1 and the second shooting position P2 (predetermined positions), respectively. For example, a predetermined marker B2 is attached to the front end of the arm 21 of the robot 20, and the marker B2 is photographed by the camera 30 while the arm 21 is moved. As the marker B2, a calibration plate printed with a predetermined dot pattern or the like, a block similar to the reference blocks B1 and B3 described above, or the like can be used. Thus, the object recognition unit 50 obtains parameters for converting the position and posture (position) of an object in the coordinate system of the camera 30 recognized from an image photographed from the shooting position at the time of calibration by the camera 30 into the position and posture (position) of the object in the coordinate system of the robot 20. This parameter includes information on the first shooting position P1 and the second shooting position P2. That is, the object recognition unit 50 can, by performing calibration, recognize the position and posture of an object in the coordinate system of the robot 20 as the position and posture of the object based on an image photographed by the camera 30 from a predetermined position. Hereinafter, the position and posture of an object in the coordinate system of the robot 20 may be simply referred to as the position and posture of the object. The object recognition unit 50 uses the three-dimensional image (three-dimensional image data) obtained by the camera 30 to recognize (calculate) the position and posture of the object. Specifically, the object recognition unit 50 calculates the position, posture (orientation), etc. of the object based on the three-dimensional image obtained by the camera 30. The object recognition unit 50 recognizes the position and posture of the workpiece W, the reference blocks B1 and B3, and the placement part Sn in the housing box H based on the three-dimensional image.

[0039] The robot control device 60 includes a CPU, a ROM, a RAM, an input / output interface, and the like. The robot control device 60 controls the swinging, rotation, and operation of the gripping tool 22 of the arm 21 of the robot 20. The robot control device 60 controls each operation of the robot 20 based on the position and posture of the workpiece W recognized by the object recognition unit 50. In addition, since each operation of the robot 20 is previously taught and stored in the robot control device 60, the robot control device 60 automatically corrects the teaching data based on the position and posture of the workpiece W calculated by the object recognition unit 50.

[0040] Next, with reference to Figure 2 the flowchart, the steps of the initial setting performed before operating the robot 20 will be described. This series of processes is executed by the object recognition unit 50.

[0041] First, calibrate the robot 20 and the camera 30 (S10). Specifically, perform the above calibration while moving the camera 30 to the first shooting position P1 using the slider 40. At this time, all the workpieces W and the reference block B1 inside the transport box Tb are included in the three-dimensional image captured by the camera 30. In addition, the transport box Tb and the reference block B1 can also be arranged after calibration, and all the workpieces W and the reference block B1 inside the transport box Tb are included in the three-dimensional image captured by the camera 30.

[0042] Next, register the shape data of the reference block B1 (S11). Specifically, obtain the three-dimensional shape data of the reference block B1 by processing the three-dimensional image captured by the camera 30 and register it. In addition, the three-dimensional shape data is obtained as point cloud data representing the positions of the points of the object.

[0043] Next, obtain the positional relationship between the reference block B1 and the camera 30 (S12). Specifically, based on the three-dimensional image obtained by shooting the reference block B1 from the first shooting position P1 using the camera 30, identify the position and orientation of the reference block B1 in the coordinate system of the camera 30. Convert the position and orientation (position) of the reference block B1 in the coordinate system of the camera 30 to the position and orientation (position) of the reference block B1 in the coordinate system of the robot 20 using the above parameters. Then, obtain the positional relationship between the robot and the camera 30. That is, obtain the position and orientation of the reference block B1 identified based on the three-dimensional image obtained by shooting the reference block B1 from the first shooting position P1 using the camera 30 as the positional relationship between the reference block B1 and the camera 30.

[0044] Next, register the shape data of the object (S13). Specifically, obtain the three-dimensional shape data of the workpiece W by processing the three-dimensional image captured by the camera 30 and register it. In addition, since multiple workpieces W have the same shape, it is sufficient to obtain the three-dimensional shape data of one representative workpiece W.

[0045] Next, save the main data (S14). Specifically, save the three-dimensional shape data of the above reference block B1, the position and orientation of the reference block B1 in the coordinate system of the robot 20, and the three-dimensional shape data of the workpiece W as the main data used when the robot 20 works. Then, end this series of processes (END).

[0046] In addition, the object recognition unit 50 performs the same processes as S10 to S14 above while moving the camera 30 to the second shooting position P2 using the slider 40. Here, the reference block B3 is used as the reference block, and the placement part Sn inside the storage box H is regarded as the object.

[0047] Next, refer to Figure 3The flowchart illustrates the control steps of the robot 20 during operation. In this series of processes, the processes of S20 to S26 are executed by the object recognition unit 50, and the process of S27 is executed by the robot control device 60. Here, an example is described where, when the robot 20 performs an operation in the housing unit, the slider 40 is used to move the camera 30 to the first shooting position P1 for this series of processes. When the camera 30 is moved from the first shooting position P1 to another position by the slider 40 and then moved back to the first shooting position P1, an offset (error) occurs between the first shooting position P1 and the actual position of the camera 30.

[0048] First, the main data is read (S20). Specifically, the main data saved in the above initial setting is read.

[0049] Next, it is determined whether the reference block and the object enter the same shooting field of view (S21). Specifically, it is determined whether the reference block B1 and all the workpieces W enter the same shooting field of view (one shooting field of view). This determination can be made by so-called matching based on the three-dimensional image captured by the camera 30 and the above main data (the three-dimensional shape data of the reference block B1 and the three-dimensional shape data of the workpiece W). In this determination, if it is determined that the reference block B1 and all the workpieces W have not entered the same shooting field of view (S21: No), the camera 30 is moved to another shooting position by the slider 40 (S22). Specifically, the movable part 41 of the slider 40 is moved a predetermined amount in either direction of the linear track of the slider 40. In addition, based on the three-dimensional image captured in the above determination, the moving direction and the moving amount of the movable part 41 for making the reference block and the object enter the same shooting field of view can also be determined. Then, the process starts again from S21. That is, when the reference block B1 and the workpiece W have not entered the same field of view, the camera 30 is moved to a position where the reference block B1 and the workpiece W enter the same field of view, and a three-dimensional image (predetermined image) is acquired.

[0050] In the determination of S21, if it is determined that the reference block B1 and all the workpieces W enter the same shooting field of view (S21: Yes), the offset amount of the movable camera is calculated (S23). Specifically, the offset amount of the current position (position) of the camera 30 from the first shooting position P1 is calculated. Here, if the position of the reference block B1 recognized based on the three-dimensional image (predetermined image) captured by the camera 30 at the current position is offset from the position of the reference block B1 in the main data (reference position), this offset amount corresponds to (results from) the offset of the current position of the camera 30 from the first shooting position P1. Therefore, based on the position of the reference block B1 recognized from the three-dimensional image captured by the camera 30 at the current position and the position of the reference block B1 in the main data, the offset amount by which the current position of the camera 30 is offset from the first shooting position P1 is calculated.

[0051] Next, correct the offset of the movable camera (S24). Specifically, correct the above parameters (make the parameters reflect the offset) for converting the position and orientation (position) of an object in the coordinate system of camera 30 to the position and orientation (position) of the object in the coordinate system of robot 20 based on the above offset.

[0052] Next, identify the position and orientation (position) of the object (S25). Specifically, based on the three-dimensional image obtained by photographing workpiece W from the current position using camera 30, identify the position and orientation of workpiece W in the coordinate system of camera 30. At this time, select workpiece W as the object to be grasped from among multiple workpieces W, and identify the position and orientation of the selected workpiece W. Then, use the above corrected parameters to convert the position and orientation of workpiece W in the coordinate system of camera 30 to the position and orientation of workpiece W in the coordinate system of robot 20. That is, calculate the offset of the current position (position) of camera 30 from the first shooting position P1 (specified position) based on the position (reference position) of reference block B1 in the master data and the position of reference block B1 identified based on the three-dimensional image (specified image) in which reference block B1 and workpiece W enter the field of view, and identify the position and orientation (position) of workpiece W based on the offset and the specified image.

[0053] Next, send the position and orientation (position) of the object to the robot control device 60 (S26). Specifically, the object recognition unit 50 sends the identified position and orientation of workpiece W to the robot control device 60.

[0054] Next, control the operation of robot 20 (S27). Specifically, based on the received position and orientation of workpiece W, the robot control device 60 operates the arm 21 and the gripping tool 22 to grasp workpiece W (perform an operation on workpiece W). On the other hand, when robot 20 performs an operation at the supply unit, the object recognition unit 50 moves camera 30 to the second shooting position P2, and performs the same processing as S20 to S26 on each placement part Sn of reference block B3 and storage box H. After grasping workpiece W at the supply unit, move workpiece W to the processing unit, and process workpiece W using a cutting machine or the like. After processing workpiece W, move workpiece W to the storage unit, and place workpiece W on each placement part Sn of storage box H. At this time, the robot control device 60 places workpiece W on the target placement part Sn based on the position and orientation of each placement part Sn of storage box H pre-identified when robot 20 performs an operation at the supply unit. Then, when robot 20 performs an operation at the storage unit, the object recognition unit 50 moves camera 30 back to the first shooting position P1, and performs the processing of S20 to S26 again. Hereinafter, the above processing is repeated until the operation on all workpieces W is completed.

[0055] The embodiment described in detail above has the following advantages.

[0056] · The camera 30 moves separately from the robot 20 and can photograph the workpiece W in multiple work areas and the placement portion Sn of the storage box H. Then, the object recognition unit 50 can recognize the position of the object based on the images captured by the camera 30 from the first shooting position P1 and the second shooting position P2. Therefore, by moving the camera 30 to the first shooting position P1 and the second shooting position P2 to photograph the workpiece W in multiple work areas and the placement portion Sn of the storage box H respectively, the positions of the workpiece W and the placement portion Sn of the storage box H can be recognized in multiple work areas respectively. Therefore, even when the robot 20 operates on the workpiece W in multiple work areas respectively, an increase in the number of cameras required can be suppressed.

[0057] · The object recognition unit 50 calculates the offset of the position of the camera 30 from the first shooting position P1 based on the reference position of the reference block B1 recognized from the three-dimensional image captured from the first shooting position P1 and the position of the reference block B1 recognized from the specified image in which the reference block B1 and the workpiece W enter the same field of view. Then, the position of the workpiece W is recognized based on the offset and the specified image. Therefore, even when the camera 30 cannot accurately return to the first shooting position P1, the position of the workpiece W can be accurately recognized. Moreover, the robot 20 can accurately operate on the workpiece W based on the position of the workpiece W recognized by the object recognition unit 50. The same applies to the second shooting position P2, the reference block B3, and the placement portion Sn of the storage box H.

[0058] · The position of the workpiece W can be recognized by the camera 30 in the supply unit, and the workpiece W can be operated by the robot 20 in the storage unit where the position of the placement portion Sn of the storage box H has been recognized. That is, the recognition of the position of the workpiece W based on the camera 30 and the operation on the workpiece W to the placement portion Sn based on the robot 20 can be performed in parallel. Therefore, an increase in the cycle time from starting to recognize the position of the workpiece W by the camera 30 to ending the operation on the workpiece W in multiple work areas can be suppressed. In addition, the position of the placement portion Sn of the storage box H can be recognized by the camera 30 in the storage unit, and the workpiece W can be operated by the robot 20 in the supply unit where the position of the workpiece W has been recognized.

[0059] · When the reference block B1 and the workpiece W have not entered the same shooting field of view, the camera 30 moves to a position where the reference block B1 and the workpiece W enter the same shooting field of view and acquires a specified image. With such a structure, even when the camera 30 moves to a position where the reference block B1 and the workpiece W have not entered the same shooting field of view, by changing the position through the movement of the camera 30, it is possible to acquire a specified image captured when the reference block B1 and the workpiece W enter the same shooting field of view.

[0060] · The robot 20 performs an operation of grasping the workpiece W in the supply unit in a plurality of work areas, moving the workpiece W to the housing unit, and releasing it. With such a structure, even when performing an operation of grasping the workpiece W in one work area among a plurality of work areas, moving the workpiece W to another work area, and releasing (placing) it, it is not necessary to provide a camera in each work area. In addition, the operation of grasping the workpiece W using the robot 20 can be performed in the supply unit where the position of the workpiece W has already been recognized, and the position of the placement portion Sn of the storage box H can be recognized using the camera 30 in the housing unit where the workpiece W is released. As a result, an increase in the number of required cameras is suppressed, and an increase in the cycle time is suppressed.

[0061] · Before the robot 20 grasps the workpiece W in the supply unit in a plurality of work areas, the object recognition unit 50 calculates the offset of the current position of the camera 30 from the first shooting position P1 based on the reference position and the position of the reference block B1 recognized from the specified image, and recognizes the position and orientation of the workpiece W based on the offset and the specified image. Then, the robot 20 performs the operation of grasping the workpiece W based on the position and orientation of the workpiece W recognized by the object recognition unit 50. With such a structure, it is possible to accurately recognize the position and orientation of the workpiece W before grasping the workpiece W, and to accurately grasp the workpiece W based on the accurately recognized position and orientation of the workpiece W.

[0062] · The reference block B1 is a block formed by an asymmetric polyhedron arranged around the workpiece W. With such a structure, the object recognition unit 50 can easily capture the features of the reference block B1, and thus can easily recognize the position of the reference block B1.

[0063] In addition, the above-described embodiment can also be implemented by making the following changes. The same reference numerals are given to the same parts as those in the above-described embodiment, and the description thereof is omitted.

[0064] · The three-dimensional shape data of the reference blocks B1 and B3 can also be formed by synthesizing the three-dimensional image data captured by the camera 30 at the first shooting position P1 and the second shooting position P2 respectively and the three-dimensional CAD data of the reference blocks B1 and B3. In addition, the three-dimensional shape data of the workpiece W and the placement portion Sn of the storage box H can also be formed by synthesizing the three-dimensional image data captured by the camera 30 at the first shooting position P1 and the second shooting position P2 respectively and the three-dimensional CAD data of the workpiece W and the placement portion Sn.

[0065] · As Figure 1 shown, the object recognition unit 50 can also be calibrated using a 3D scanner 70 with a higher accuracy in detecting the position of the object than the camera 30. In this case, the object recognition unit 50 performs calibration respectively in a state where the 3D scanner 70 is disposed at the first shooting position P1 and the second shooting position P2 (predetermined positions). Then, the position postures (reference positions) of the reference blocks B1 and B3 recognized from the three-dimensional images obtained by shooting the reference blocks B1 and B3 from the first shooting position P1 and the second shooting position P2 using the 3D scanner 70 are acquired and registered as the positional relationship between the reference blocks B1 and B3 and the camera 30.

[0066] With such a configuration, the correspondence relationship between the coordinate system of the camera 30 and the coordinate system of the robot 20 can be accurately corrected. And the positions of the reference blocks B1 and B3, that is, the reference positions, recognized based on the images captured from the first shooting position P1 and the second shooting position P2 respectively can be obtained more accurately. Therefore, the offset amount of the position of the camera 30 from the first shooting position P1 and the second shooting position P2 can be calculated more accurately, and furthermore, the positions of the workpiece W and the placement portion Sn of the storage box H can be recognized more accurately.

[0067] · When the robot 20 grasps the workpiece W in the supply unit (one working area), a deviation occurs in the relative position between the robot 20 and the workpiece W. In this case, in the operation of moving the grasped workpiece W to the storage unit (other working area) and releasing (placing) it, a deviation may occur in the position where the workpiece W is placed.

[0068] Therefore, after the object recognition unit 50 grasps the workpiece W in the supply unit among the plurality of working areas, the offset amount of the current position of the camera 30 from the first shooting position P1 can be calculated based on the reference position and the position of the reference block B1 recognized from a specified image, and the position posture (position) of the workpiece W can be recognized based on the offset amount and the specified image. Then, after grasping the workpiece W, the robot 20 can perform an operation of moving the workpiece W from the supply unit to the storage unit and releasing it based on the position posture of the workpiece W recognized by the object recognition unit 50.

[0069] According to such a structure, after grasping the workpiece W, the position and posture of the workpiece W can be accurately recognized, and based on the accurately recognized position and posture of the workpiece W, the workpiece W can be moved from the supply unit to the storage unit and released at an accurate position (placed at an accurate position). In addition, after grasping the workpiece W in the supply unit, based on the position and posture of the workpiece W recognized by the object recognition unit 50, an operation of moving the workpiece W from the supply unit to the processing unit and performing processing can be carried out.

[0070] · Based on the position of the reference block B1 recognized from the three-dimensional image captured by the camera 30 at the current position and the position of the reference block B1 in the main data, calculate the offset amount by which the current position of the camera 30 deviates from the first shooting position P1. Based on this offset amount and the information of the first shooting position P1 and the second shooting position P2 included in the above parameters for converting coordinates, calculate the current position of the camera 30. Then, based on the current position of the camera 30 and the above specified image, the position and posture (position) of the workpiece W can also be recognized.

[0071] · As Figure 1 As shown by the dashed line in the figure, the robot control system 10 may also include multiple robots 20. In this case, the reference blocks B1 and B3 (reference objects) are respectively arranged in the multiple working areas of the multiple robots 20. Moreover, the camera 30 can respectively capture the workpiece W (object) and the placement part Sn of the storage box H in the multiple working areas of the multiple robots 20. Specifically, the movable part 41 of the slider 40 can move to the multiple working areas of the multiple robots 20. According to such a structure, the position of the object in the operations of the multiple robots 20 can be recognized by using one camera 30. Therefore, the number of cameras required can be further reduced.

[0072] In the above case, when the camera 30 is capturing one reference block, if the workpiece W and the placement part Sn of the storage box H enter the shooting field of view, a common reference block can also be arranged in the supply unit and the storage unit. Moreover, one reference block can also be arranged for each robot 20.

[0073] · As the reference object serving as the position reference, a part of the equipment existing around the workpiece W and the placement part Sn of the storage box H (for example, a part of the device, a part of the support structure) can also be adopted. A part of the equipment adopted as the reference object is preferably an object whose position and shape do not change and is an object whose feature part is easily recognized by the camera 30. According to such a structure, a part of the equipment existing around the workpiece W and the placement part Sn of the storage box H can be used as the reference object, so there is no need to arrange a special block or the like as the reference object.

[0074] · The object recognition unit 50 may be included in the robot control device 60 or may be included in the camera 30.

[0075] · The robot 20 can also be a single-arm horizontal multi-joint robot. Additionally, the robot 20 can also be a multi-joint robot with two arms.

[0076] Furthermore, the above-described embodiments and their modification examples can be combined and implemented within the combinable range.

[0077] Hereinafter, characteristic structures extracted from the above-described embodiments and modification examples are described.

[0078] [Structure 1] A robot control system (10) includes:

[0079] A robot (20) that performs operations on an object (W) in multiple work areas;

[0080] A movable camera (30) that moves separately from the above-described robot and can capture the above-described object in the above-described multiple work areas; and

[0081] An object recognition unit (50) that can recognize the position of an object based on images captured by the above-described movable camera from specified positions (P1, P2),

[0082] The above-described movable camera acquires a reference object (B1) serving as a position reference and a specified image captured when the above-described object enters the field of view,

[0083] The above-described object recognition unit pre-registers the position of the above-described reference object recognized from the image captured from the above-described specified position as a reference position, calculates the offset of the position of the above-described movable camera from the above-described specified position based on the above-described reference position and the position of the above-described reference object recognized from the above-described specified image, and recognizes the position of the above-described object based on the above-described offset and the above-described specified image.

[0084] The above-described robot performs operations on the above-described object based on the position of the above-described object recognized by the above-described object recognition unit.

[0085] [Structure 2] The robot control system according to Structure 1, wherein

[0086] A plurality of the above-described robots are provided,

[0087] The above-described reference objects are respectively provided in the above-described multiple work areas of the above-described plurality of robots,

[0088] The above-described movable camera can respectively capture the above-described objects in the above-described multiple work areas of the above-described plurality of robots.

[0089] [Structure 3] The robot control system according to Structure 1 or 2, wherein

[0090] In the case where the above reference object and the above target object have not entered the field of view, the above movable camera moves to the position where the above reference object and the above target object enter the field of view and acquires the above specified image.

[0091] [Structure 4] The robot control system according to any one of Structures 1 to 3, wherein

[0092] The above object recognition unit can, by calibrating the correspondence between the coordinate system of the above movable camera and the coordinate system of the above robot, identify the position of the object in the coordinate system of the above robot as the position of the above object based on the image captured from the above specified position using the above movable camera.

[0093] [Structure 5] The robot control system according to Structure 4, wherein

[0094] The above calibration is performed using a 3D scanner (70) with a higher accuracy in detecting the position of an object than the above movable camera,

[0095] The above reference position is registered in advance using the above 3D scanner.

[0096] [Structure 6] The robot control system according to any one of Structures 1 to 5, wherein

[0097] The above robot performs an operation of grasping the above target object in one of the above multiple work areas, moving the above target object to another work area, and releasing it.

[0098] [Structure 7] The robot control system according to Structure 6, wherein

[0099] Before grasping the above target object in one of the above multiple work areas, the above object recognition unit calculates the offset of the position of the above movable camera from the above specified position based on the above reference position and the position of the above reference object recognized based on the above specified image, and identifies the position of the above target object based on the above offset and the above specified image,

[0100] The above robot performs an operation of grasping the above target object based on the position of the above target object recognized by the above object recognition unit.

[0101] [Structure 8] The robot control system according to Structure 6 or 7, wherein

[0102] After grasping the above target object in one of the above multiple work areas, the above object recognition unit calculates the offset of the position of the above movable camera from the above specified position based on the above reference position and the position of the above reference object recognized based on the above specified image, and identifies the position of the above target object based on the above offset and the above specified image,

[0103] Based on the position of the object identified by the object recognition unit, the robot performs an operation of moving the object from the one work area to another work area and releasing it.

[0104] [Structure 9] The robot control system according to any one of Structures 1 to 8, wherein

[0105] The reference object is a block formed of an asymmetric polyhedron arranged around the object.

[0106] [Structure 10] The robot control system according to any one of Structures 1 to 8, wherein

[0107] The reference object is a part of a device existing around the object.

[0108] The present disclosure has been described based on the embodiments, but it should be understood that the present disclosure is not limited to the embodiments and configurations. The present disclosure also includes various modifications and modifications within the equivalent scope. In addition, various combinations and modes, further including only one element thereof, more than one, or other combinations and modes less than one thereof are also included in the scope and thinking range of the present disclosure.

Claims

1. A robot control system (10) comprising: A robot (20) that performs operations on an object (W) in multiple work areas respectively; A movable camera (30) that moves separately from the above-mentioned robot and can respectively photograph the above-mentioned object in the above-mentioned multiple work areas; and An object recognition unit (50) that can recognize the position of an object based on images taken by the above-mentioned movable camera from specified positions (P1, P2), wherein, The above-mentioned movable camera obtains a specified image in which a reference object (B1) serving as a position reference and the above-mentioned object enter the field of view, The above-mentioned object recognition unit pre-registers the position of the above-mentioned reference object recognized based on the image taken from the above-mentioned specified position as a reference position, calculates the offset of the position of the above-mentioned movable camera from the above-mentioned specified position based on the above-mentioned reference position and the position of the above-mentioned reference object recognized based on the above-mentioned specified image, and recognizes the position of the above-mentioned object based on the above-mentioned offset and the above-mentioned specified image, The above-mentioned robot performs operations on the above-mentioned object based on the position of the above-mentioned object recognized by the above-mentioned object recognition unit.

2. The robot control system according to claim 1, wherein, There are multiple above-mentioned robots, The above-mentioned reference objects are respectively arranged in the above-mentioned multiple work areas of the multiple above-mentioned robots, The above-mentioned movable camera can respectively photograph the above-mentioned objects in the above-mentioned multiple work areas of the multiple above-mentioned robots.

3. The robot control system according to claim 1 or 2, wherein, When the above-mentioned reference object and the above-mentioned object have not entered the field of view, the above-mentioned movable camera moves to a position where the above-mentioned reference object and the above-mentioned object enter the field of view and obtains the above-mentioned specified image.

4. The robot control system according to claim 1 or 2, wherein, The above-mentioned object recognition unit can recognize the position of an object in the coordinate system of the above-mentioned robot as the above-mentioned position of the object based on an image taken by the above-mentioned movable camera from the above-mentioned specified position by calibrating the correspondence between the coordinate system of the above-mentioned movable camera and the coordinate system of the above-mentioned robot.

5. The robot control system according to claim 4, wherein, The above-mentioned calibration is performed using a 3D scanner (70) with a higher accuracy in detecting the position of an object than the above-mentioned movable camera, The above-mentioned reference position is pre-registered using the above-mentioned 3D scanner.

6. The robot control system according to claim 1 or 2, wherein, The above-mentioned robot performs an operation of grasping the above-mentioned object in one of the above-mentioned multiple work areas, moving the above-mentioned object to another work area, and releasing it.

7. The robot control system according to claim 6, wherein, Before grasping the above-mentioned object in one of the above-mentioned multiple work areas, the above-mentioned object recognition unit calculates the offset of the position of the above-mentioned movable camera from the above-mentioned specified position based on the above-mentioned reference position and the position of the above-mentioned reference object recognized based on the above-mentioned specified image, and recognizes the position of the above-mentioned object based on the above-mentioned offset and the above-mentioned specified image, The above-mentioned robot performs an operation of grasping the above-mentioned object based on the position of the above-mentioned object recognized by the above-mentioned object recognition unit.

8. The robot control system according to claim 6, wherein after grasping the above-mentioned object in one of the above-mentioned multiple work areas, the above-mentioned object recognition unit calculates an offset amount of the position of the above-mentioned movable camera from the above-mentioned specified position based on the above-mentioned reference position and the position of the above-mentioned reference object recognized from the above-mentioned specified image, and recognizes the position of the above-mentioned object based on the above-mentioned offset amount and the above-mentioned specified image. The above-mentioned robot performs an operation of moving the above-mentioned object from the above-mentioned one work area to another work area and releasing it based on the position of the above-mentioned object recognized by the above-mentioned object recognition unit.

9. The robot control system according to claim 1 or 2, wherein the above-mentioned reference object is a block formed of an asymmetric polyhedron arranged around the above-mentioned object.

10. The robot control system according to claim 1 or 2, wherein the above-mentioned reference object is a part of a device existing around the above-mentioned object.

Images