Method for controlling conveyance of object, conveyance device for conveying object, and work system provided with conveyance device

Abstract

The control method is a method for controlling conveyance of an object to a target position by a first moving body (232) using a captured image obtained by an imaging unit (234). The control method includes the steps of: performing proximity control for causing the imaging unit (234) to relatively approach a feature part indicating a reference position different from the target position; causing an imaging unit (234) that relatively approaches the feature part by the approach control to acquire a first image obtained by imaging the feature part; and causing the first moving body (232) to convey the object to the target position on the basis of the first image.

Description

Technical Field

[0001] The present disclosure relates to a control method for controlling the conveyance of an object, a conveyance device for conveying the object to be conveyed, and a work system including the conveyance device. Background Art

[0002] In recent years, efforts have been made to automate the operation of conveying an object such as a sample to a target position of a work device by a robot. In a scenario where such automation is attempted, usually, a teaching operation must be performed every time the relative position between the robot and the work device changes due to system relocation, reorganization, maintenance during operation, or the like. The teaching operation refers to an operation in which a person sets at which timing the conveyance robot should perform what action. The action that the robot should adopt changes according to the relative position between the robot and the work device. Therefore, a teaching operation is required every time the position of the work device changes. The teaching operation is a laborious and time-consuming operation, and sometimes takes about several hours. Therefore, a technique is known in which the action of the robot is corrected to match the recognized target position during conveyance to save the labor and time of the teaching operation.

[0003] The following is disclosed in the non-patent document titled "Benchmark for Bimanual Robotic Manipulation of Semi-Deformable Objects": When assembling a clock using a robot, a feature part (for example, a mark or the like) provided around the clock is photographed by a camera to identify the position of the dial of the clock.

[0004] Prior Art Documents

[0005] Non-Patent Documents

[0006] Non-Patent Document 1: Konstantinos Chatzilygeroudis et al. (six others), "Benchmark for Bimanual Robotic Manipulation of Semi-Deformable Objects", [online], IEEE ROBOTICS AND AUTOMATION LETTERS, VOL.5, NO.2, APRIL 2020, [searched on July 12, 2022], Internet, <URL: https: / / ieeexplore.ieee.org / abstract / document / <8989777> Summary of the Invention

[0007] ​Problem that the invention aims to solve

[0008] In order to identify the target position in the conveying operation performed by the robot, it is considered to apply the position recognition technology disclosed in the above non-patent document. If the position recognition technology disclosed in the above non-patent document is applied, the target position can be identified by using a camera to capture a feature portion set at a position different from the target position in the conveying operation performed by the robot. In the case of conveying an object by identifying the target position in this way, a technology for conveying the object to the target position more reliably is sought.

[0009] The present disclosure has been made in view of this actual situation, and one object thereof is to provide a technology capable of reliably transporting an object to a target position.

[0010] Solutions for solving problems

[0011] A control method according to one aspect of the present disclosure is a method for controlling the transport of an object to a target position by a first moving body using a camera image obtained by a camera unit. The control method includes the following steps: performing a proximity control to make the camera unit relatively close to a feature portion indicating a reference position different from the target position; causing the camera unit that has relatively approached the feature portion through the proximity control to acquire a first image obtained by photographing the feature portion; and causing the first moving body to transport the object to the target position based on the first image.

[0012] A conveying device according to an aspect of the present disclosure includes: an imaging unit; a first moving body that conveys an object to a target position; and a control unit that performs approach control to make the imaging unit relatively close to a feature portion indicating a reference position different from the target position. The imaging unit that has relatively approached the feature portion through the approach control acquires a first image obtained by photographing the feature portion. The first moving body conveys the object to the target position based on the first image.

[0013] A working system according to one aspect of the present disclosure includes the above-mentioned conveying device and a working device for performing a work using an object conveyed by the conveying device.

[0014] Effects of the Invention

[0015] According to the present disclosure, the camera unit can capture the feature from the vicinity of the feature, so that the posture of the feature can be determined more accurately. The target position is determined based on the relative positional relationship with the reference position represented by the feature. Therefore, if the posture of the feature is accurately determined, the target position is more accurately determined, and thus the object is reliably transported to the target position. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1This is a diagram for explaining the outline of the operating system in the embodiment.

[0017] Figure 2 This is a diagram showing an example of the hardware configuration of the operating system in the embodiment.

[0018] Figure 3 This is a diagram for explaining the outline of the processing of the transfer device in the embodiment.

[0019] Figure 4 This is a flowchart showing the process of the processing related to the control of the transfer robot in the processing of the management device in the embodiment.

[0020] Figure 5 This is a diagram for explaining the second mark in the embodiment.

[0021] Figure 6 This is a flowchart showing the process of the registration process in the embodiment.

[0022] Figure 7 This is a flowchart showing the process of the motion planning creation process in the embodiment.

[0023] Figure 8 This is a flowchart showing the process of the transfer process in the embodiment.

[0024] Fig. 9 This is a diagram showing the transfer robot in Modification 1. Detailed Embodiment

[0025] Hereinafter, embodiments and modifications of the present disclosure will be described in detail with reference to the drawings. In addition, the same or corresponding parts in the drawings are given the same reference numerals, and their descriptions will not be repeated. The embodiments and modifications described below can also be selectively combined as appropriate.

[0026] <A. Outline of the Operating System>

[0027] Figure 1 This is a diagram for explaining the outline of the operating system in the embodiment. The operating system 1 in the embodiment includes a controlled device group 200 and a management device 100. The controlled device group 200 includes a transfer robot 230, a working device 220, and a working device 240.

[0028] The transport robot 230 transports the well plate 5 from the working device 220 to the target position Tp in the working device 240. The well plate 5 contains samples used by the working devices 220 and 240. The well plate 5 containing the sample is an example of the "object" in the present disclosure. The working device 220 and the working device 240 are each an example of a device for performing work using the sample transported by the transport robot 230. The working device 220 is a rack for holding the well plate 5. The working device 240 is a culture device for culturing bacteria contained in the sample contained in the well plate 5.

[0029] In this embodiment, as an example of the operation system 1, a system for performing a bacterial culture experiment is used, as an example of the operation on the object, an experiment is used, and as the operation device, a device for performing the experiment is used. In addition, the operation on the object is not limited to the experiment. As the operation on the object, any type of operation such as metal processing can also be used. Therefore, as the operation device, in addition to the device for performing the experiment, any type of device for performing the operation on the object can also be used.

[0030] The management device 100 controls the operations of the transfer robot 230 , the work device 220 , and the work device 240 . The transfer device 300 is a device including the management device 100 and the transfer robot 230 .

[0031] The transport robot 230 is a six-axis vertical multi-joint robot with one arm. The transport robot 230 includes a main body 231, an arm 232, a gripper 233, and a camera 234. The main body 231 is used to hold the arm 232. The arm 232 is used to transport the orifice plate 5 from the working device 220 to the target position Tp in the working device 240. The arm 232 is an example of the "first moving body" in the present disclosure. The gripper 233 is provided at the front end of the arm 232 and is used to hold the orifice plate 5. The arm 232 includes one or more movable parts 8. By moving one or more movable parts 8, the arm 232 can move up and down, left and right, and front and back. Thus, the orifice plate 5 is transported from the working device 220 to the target position Tp in the working device 240 by the arm 232.

[0032] exist Figure 1 In the example, the controlled device group 200 includes a plurality of working devices, and the transport robot 230 transports the object (orifice plate) from one of the plurality of working devices to another working device. In addition, the controlled device group 200 may also be composed of a single working device, and the transport robot 230 may also transport the object from a certain position in the single working device to another position.

[0033] The imaging unit 234 is provided on the arm 232 and is configured to acquire an image of the object to be imaged. The operations of the arm 232, the jig 233, and the imaging unit 234 are controlled by the management device 100.

[0034] First markers Rm1, Rm2, and Rm3 are attached to objects around the transfer robot 230. The first marker Rm1 indicates a reference position Rp1, the first marker Rm2 indicates a reference position Rp2, and the first marker Rm3 indicates a reference position Rp3.

[0035] In the following description, the first markers Rm1, Rm2, and Rm3 are regarded as having a common property (indicating a reference position) and are referred to as "first marker Rm" when there is no need to distinguish them from each other. The first marker Rm is an example of the "feature part" in the present disclosure. The first marker Rm is provided at a position different from the target position Tp. The first marker Rm may be any pattern that can determine the pose of the first marker Rm, such as an AR (Augmented Reality) marker, a QR code (registered trademark), etc. In the present embodiment, "pose" includes position and orientation. The first markers Rm1, Rm2, and Rm3 can be distinguished from each other. For example, when the first markers Rm1, Rm2, and Rm3 are implemented as QR codes, they are implemented as QR codes with different information.

[0036] In the following description, the reference positions Rp1, Rp2, and Rp3 are regarded as having a common property (being reference positions) and are referred to as "reference position Rp" when there is no need to distinguish them from each other. The reference position Rp is located at a position different from the target position Tp. In the present embodiment, the target position Tp is identified based on the reference position Rp. Through the registration process described later, the position information indicating the relative position relationship between the target position Tp and the reference position Rp is stored in the management device 100.

[0037] The management device 100 performs an approach control to relatively approach the first marker Rm with the imaging unit 234, so that the imaging unit 234 captures the first marker Rm from nearby. In the following description, the image obtained in this way is referred to as the "first image". The first image includes pixels representing the first marker Rm. The management device 100 corrects the motion plan of the arm 232 created in advance through a teaching operation based on the first image, and controls the operation of the transfer robot 230 based on the corrected motion plan. Thus, in the present embodiment, the object can be reliably transferred to the target position.

[0038] <B. Hardware Structure of the Operating System>

[0039] Figure 21 is a diagram showing an example of a hardware configuration of a work system in the embodiment. The work system 1 includes a transport device 300 , a work device 220 , and a work device 240 . The transport device 300 includes a transport robot 230 and a management device 100 .

[0040] The management device 100 is composed of, for example, a general-purpose computer. Figure 2 In the example of FIG. 1 , the management device 100 includes a processor 101 , a memory 102 , a storage 103 , an interface 104 , a display 105 , and an input device 106 .

[0041] The processor 101 executes various programs to cause the management device 100 to perform various processes. The processor 101 is composed of hardware elements such as a CPU (Central Processing Unit) and an MPU (Micro-Processing Unit). The memory 102 functions as a main storage device and is composed of a volatile storage device such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory).

[0042] The storage 103 stores programs executed by the processor 101 and various data required for executing the programs, and is composed of a non-volatile storage device such as an SSD (Solid State Drive) or a flash memory.

[0043] In addition, the program may be provided not as a single program but embedded in a part of any program. In this case, the processing involved in this embodiment is implemented in cooperation with any program. Even if such a program does not include a part of the module, it does not deviate from the main purpose of the management device 100 involved in this embodiment. In addition, part or all of the functions provided by the program may also be implemented by dedicated hardware.

[0044] The storage 103 stores position information 132 and teaching information 133. The position information 132 is stored in the storage 103 by a registration process described later, and the teaching information 133 is stored in the storage 103 by an action plan creation process described later.

[0045] The interface 104 relays communication between the management device 100 and external devices (for example, the transfer robot 230 , the working device 220 , the working device 240 , etc.).

[0046] The display 105 displays the result of the arithmetic processing of the processor 101, and the input device 106 (such as a mouse, a keyboard, a touch sensor, etc.) accepts the operation of inputting data to the processor 101.

[0047] The transfer robot 230 includes a camera unit 234, an interface 235, a motor unit 236, and a driver unit 237. The interface 235 relays the communication between the transfer robot 230 and the management device 100. The motor unit 236 includes motors respectively corresponding to one or more movable parts 8. The driver unit 237 includes drivers that drive the one or more motors included in the motor unit 236 respectively.

[0048] <C. Processing Outline of Transfer Device>

[0049] Figure 3 This is a diagram for explaining the processing outline of the transfer device in the embodiment. The transfer device 300 in the embodiment includes a camera unit 234, a management device 100, and an arm 232. The management device 100 includes a registration unit 151, a storage 103, a motion plan creation unit 152, a motion plan correction unit 153, and a control unit 154. The registration unit 151, the motion plan creation unit 152, the motion plan correction unit 153, and the control unit 154 are implemented by the processor 101 executing the above program.

[0050] The camera unit 234 acquires an image of the imaging object according to the instruction of the control unit 154. In the registration process described later, the camera unit 234 acquires a second image. In the motion plan creation process and the transfer process described later, the camera unit 234 acquires a first image. The second image includes pixels representing a second marker and pixels representing a first marker Rm. Regarding the second marker, it will be described later, but the second marker is pasted by the user at the target position Tp to represent the target position Tp.

[0051] When the registration unit 151 acquires the second image from the camera unit 234, it first determines the relative positional relationship between the target position Tp and the reference position Rp based on the second image. Then, the registration unit 151 saves the position information 132 representing the relative positional relationship between the target position Tp and the reference position Rp in the storage 103. Saving the position information 132 in the storage 103 is an example of "registering position information" in the present disclosure.

[0052] When the motion plan creation unit 152 acquires the first image from the camera unit 234 in the motion plan creation process described later, it first determines the pose (position and orientation) of the first marker Rm. In the following description, the pose of the first marker Rm determined by the motion plan creation unit 152 is referred to as the "first pose".

[0053] Next, the motion plan creation unit 152 creates a motion plan for the arm 232 when the conveying robot 230 conveys the orifice plate 5 to the target position Tp in the device configuration during the teaching operation according to the user's setting operation (so-called teaching operation). As an example, the motion plan creation unit 152 displays a simulation screen for planning the motion of the arm 232 on the display 105. The device configuration during the teaching operation and the target position Tp are displayed on the simulation screen. The target position Tp displayed on the simulation screen is the position determined by the motion plan creation unit 152 based on the first posture and the position information 132. The user plans the motion of the arm 232 on the simulation screen.

[0054] The motion planning of the arm 232 includes the direction in which the arm 232 moves and the amount by which the arm 232 moves. In more detail, the motion planning of the arm 232 includes information for specifying the motion of the arm 232 (for example, the distance and / or speed of movement in each direction of the three-axis translation direction, the angle and / or angular velocity of rotation in each direction of the three-axis rotation direction). In addition, the motion planning of the arm 232 can include information for specifying motions that contribute to the motion of the arm 232 for conveying an object, and the motions that contribute to the motion of the arm 232 for conveying an object are motions of elements other than the arm 232 (for example, the clamp 233, a motor for rotating the wheels of the main body 231 mounted on the conveying robot 230, etc.).

[0055] The so-called device configuration refers to the configuration of multiple devices included in the controlled device group 200. Figure 1 In the example of , the device configuration refers to the configuration of the working device 220 , the transport robot 230 , and the working device 240 .

[0056] Finally, the motion plan creation unit 152 stores the teaching information 133 in the storage 103, and the teaching information 133 is obtained by associating the first posture with the motion plan of the arm 232. In addition, the motion plan creation unit 152 can also create a motion plan for the arm 232 when the conveying robot 230 conveys the orifice plate 5 to the target position Tp under the device configuration during the teaching operation based on the position information 132.

[0057] When the motion plan correction unit 153 acquires the first image from the camera unit 234 in the transport process described later, it first determines the posture (position and orientation) of the first marker Rm. In the following description, the posture of the first marker Rm determined by the motion plan correction unit 153 is referred to as the "second posture". Next, the motion plan correction unit 153 corrects the motion plan included in the teaching information 133 based on the first posture and the second posture.

[0058] The control unit 154 controls the operation of the imaging unit 234. In a certain situation, the control unit 154 commands the imaging unit 234 to start and end imaging. In another situation, the control unit 154 commands the imaging unit 234 to perform imaging.

[0059] In addition, the control unit 154 controls the operation of the arm 232. More specifically, the control unit 154 outputs a drive command to the drive motor corresponding to the movable part 8 that is the object. As a result, the movable part 8 that is the object operates, and thus the arm 232 moves up and down, left and right, and / or forward and backward. In a certain situation, the control unit 154 performs approach control to relatively approach the imaging unit 234 to the first marker Rm by controlling the operation of the arm 232. In another situation, the control unit 154 controls the operation of the arm 232 based on the corrected motion plan.

[0060] <D. Processing procedure of the management device>

[0061] Refer to Figures 4 to 8 The processing related to the control of the transfer robot 230 in the processing of the management device 100 in the embodiment will be described.

[0062] (D1: Process of the processing related to the control of the transfer robot 230)

[0063] Refer to Figure 4 The process of the processing related to the control of the transfer robot 230 in the processing of the management device 100 in the embodiment will be described. Figure 4 It is a flowchart showing the process of the processing related to the control of the transfer robot in the processing of the management device in the embodiment. Figure 4 The series of processes shown are performed by the processor 101 of the management device 100.

[0064] In step S1, the processor 101 performs registration processing. The registration processing is the processing of registering the position information 132 indicating the relative position relationship between the target position Tp and the reference position Rp.

[0065] In step S2, the processor 101 performs motion plan creation processing. The motion plan creation processing is the processing of creating a motion plan of the arm 232 when the transfer robot 230 transports the orifice plate 5 to the target position Tp in the device configuration during the teaching operation according to the user's setting operation (so-called teaching operation).

[0066] In step S3, the processor 101 performs transfer processing. The transfer processing is the following processing: correcting the motion plan of the arm 232 created in the motion plan creation processing based on the first image, and controlling the operation of the transfer robot 230 based on the corrected motion plan. After step S3, Figure 4 The series of processes shown end.

[0067] (D2: Registration Process)

[0068] Reference Figure 5 and Figure 6 The processing in step S1, that is, the registration processing, will be described.

[0069] Reference Figure 5 The second marker affixed to the target position Tp will be described. Figure 5 2 is a diagram for explaining the second mark in the embodiment. Figure 5 The working device 240 is shown in FIG. Figure 1 As described above, the first mark Rm indicating the reference position Rp is attached to the reference position Rp. Before the registration process starts, the second mark Tm indicating the target position Tp is attached to the target position Tp by the user. After the registration process is completed, the second mark Tm is peeled off from the target position Tp by the user.

[0070] The second mark Tm can be any pattern that can determine the posture (position and orientation) of the second mark Tm, such as an AR mark, a QR code, etc. It is preferred that the shape of the second mark Tm is the same as the shape of the object (for example, the orifice plate 5, etc.) transported by the transport robot 230. More specifically, it is preferred that the shape of the surface of the second mark Tm is the same as the shape of the bottom surface of the object transported by the transport robot 230. In the case where the shape of the surface of the second mark Tm is the same as the shape of the bottom surface of the object transported by the transport robot 230, it is easy for the management device 100 to grasp the position (target position Tp) where the object is transported during the registration process, and thus more accurate position information can be calculated.

[0071] Figure 6 1 is a flowchart showing the process of the registration process in the embodiment. In step S11, the processor 101 instructs the transport robot 230 to start searching for the first mark Rm and the second mark Tm. More specifically, the processor 101 instructs the camera unit 234 to start shooting, and instructs the drivers corresponding to one or more movable parts 8 to start driving the motors. Thus, the camera unit 234 starts shooting, and the arm 232 moves up and down, left and right, and front and back. The camera unit 234 sends the acquired image to the processor 101.

[0072] In step S12, the processor 101 determines whether the first marker Rm and the second marker Tm are found. When the image sent from the imaging unit 234 contains both pixels representing the first marker Rm and pixels representing the second marker Tm, the processor 101 determines that the first marker Rm and the second marker Tm are found. When the first marker Rm and the second marker Tm are found ("Yes" in step S12), the processor 101 advances the process to step S13.

[0073] In step S13, the processor 101 instructs the transport robot 230 to end the search for the first mark Rm and the second mark Tm. More specifically, the processor 101 instructs the imaging unit 234 to end the image capture, and instructs the actuators corresponding to the one or more movable parts 8 to stop driving the motors. As a result, the arm 232 stops in the posture (position and orientation) at the time when the first mark Rm and the second mark Tm are found.

[0074] In step S14, the processor 101 instructs the imaging unit 234 to perform imaging. Thus, the imaging unit 234 acquires a second image including pixels representing the first marker Rm and pixels representing the second marker Tm. The imaging unit 234 sends the second image to the processor 101.

[0075] In step S15 , the processor 101 acquires a second image from the imaging unit 234 .

[0076] In step S16, the processor 101 determines the relative positional relationship between the target position Tp and the reference position Rp based on the second image. More specifically, the processor 101 determines the relative positional relationship between the reference position Rp1 and the target position Tp, the relative positional relationship between the reference position Rp2 and the target position Tp, and the relative positional relationship between the reference position Rp3 and the target position Tp based on the second image.

[0077] In step S17, the processor 101 stores the position information 132 indicating the relative positional relationship between the target position Tp and the reference position Rp in the memory 103. After step S17, the process returns to step S2.

[0078] In addition, Figure 6In the example shown, the relative positional relationship between the target position Tp and the reference position Rp is determined based on an image including both pixels representing the first marker Rm and pixels representing the second marker Tm, that is, an image in which the first marker Rm and the second marker Tm are captured. However, it is also possible to obtain an image in which the first marker Rm and an image in which the second marker Tm are captured, respectively, and determine the relative positional relationship between the target position Tp and the reference position Rp based on these images. As an example, the processor 101 may estimate the reference position Rp based on an image obtained by capturing the first marker Rm, and estimate the target position Tp based on an image obtained by capturing the second marker Tm, thereby determining the relative positional relationship between the target position Tp and the reference position Rp.

[0079] In addition, at the time point when step S13 is completed, the second image including the pixel representing the first mark Rm and the pixel representing the second mark Tm is acquired by the camera unit 234, so the processing of step S14 can also be omitted. Figure 6 As in the example shown, by performing step S14, the first mark Rm and the second mark Tm can be imaged while the arm 232 is stopped, so blurring of the image can be suppressed.

[0080] (D3: Action plan creation process)

[0081] Reference Figure 7 The processing in step S2, that is, the action plan creation processing, will be described. Figure 7 Detailed description of the invention is a flowchart showing the procedure of action plan creation processing in the embodiment.

[0082] In step S21, the processor 101 instructs the transport robot 230 to start searching for the first mark Rm. More specifically, the processor 101 instructs the camera unit 234 to start shooting, and instructs the drivers corresponding to one or more movable parts 8 to start driving the motors. As a result, the camera unit 234 starts shooting, and the arm 232 moves up and down, left and right, and forward and backward. The camera unit 234 sends the acquired image to the processor 101.

[0083] In step S22, the processor 101 determines whether the first marker Rm is found. If the image sent from the imaging unit 234 contains pixels representing the first marker Rm, the processor 101 determines that the first marker Rm is found. If the first marker Rm is found ("Yes" in step S22), the processor 101 advances the process to step S23.

[0084] In step S23, the processor 101 instructs the transport robot 230 to end the search for the first mark Rm. More specifically, the processor 101 instructs the camera unit 234 to end the image capture and instructs the drivers corresponding to the one or more movable parts 8 to stop driving the motors. As a result, the arm 232 stops in the posture (position and orientation) at the time when the first mark Rm is found.

[0085] In step S24, the processor 101 instructs the transport robot 230 to start the approaching action. More specifically, the processor 101 instructs the camera unit 234 to start shooting, and instructs the driver corresponding to the movable part for moving the arm 232 forward and backward to start driving the motor to move the arm 232 forward. Thus, the transport robot 230 starts to perform the approaching action. The so-called approaching action means that the arm 232 moves forward while the camera unit 234 is shooting. Thus, the camera unit 234 gradually approaches the first mark Rm. The camera unit 234 sends the acquired image to the processor 101.

[0086] In step S25, the processor 101 determines whether the distance between the imaging unit 234 and the first marker Rm has become smaller than a threshold value. The processor 101 determines whether the distance between the imaging unit 234 and the first marker Rm has become smaller than a threshold value based on the size of the area occupied by the pixel representing the first marker Rm in the image sent from the imaging unit 234. When the distance between the imaging unit 234 and the first marker Rm has become smaller than the threshold value ("Yes" in step S25), the processor 101 advances the process to step S26.

[0087] In step S26, the processor 101 instructs the transport robot 230 to end the approaching action. More specifically, the processor 101 instructs the imaging unit 234 to end the photographing, and instructs the driver corresponding to the movable part for moving the arm 232 forward and backward to stop driving the motor, so that the arm 232 stops moving forward. As a result, the arm 232 stops at a position where the distance between the imaging unit 234 and the first mark Rm becomes smaller than the threshold value.

[0088] In step S27, the processor 101 instructs the imaging unit 234 to perform imaging. Thus, the imaging unit 234 located at a position where the distance from the first marker Rm is less than the threshold value acquires a first image including pixels representing the first marker Rm. That is, through the processing of step S27, the imaging unit 234 captures the first marker Rm from a position close to the first marker Rm. The imaging unit 234 sends the first image to the processor 101.

[0089] In step S28 , the processor 101 acquires a first image from the imaging unit 234 .

[0090] In step S29, the processor 101 determines the posture of the first marker Rm. The posture of the first marker Rm determined in step S29 of the action plan creation process (step S2) is the above-mentioned "first posture".

[0091] In step S30 , the processor 101 creates a motion plan for the arm 232 when the transport robot 230 transports the well plate 5 to the target position Tp in the device configuration during the teaching operation according to the user's setting operation (so-called teaching operation).

[0092] In step S31, the processor 101 stores the teaching information 133 obtained by associating the first posture with the motion plan of the arm 232 created in step S30 in the memory 103. After step S31, the process returns to step S3.

[0093] (D4: Transport processing)

[0094] Reference Figure 8 The process in step S3, that is, the transport process, will be described. Figure 8 FIG. 1 is a flowchart showing the process of the transport process in the embodiment. Figure 8 The processing shown is Figure 7 The same processing as shown is assigned the same step number, and the description thereof will not be repeated.

[0095] After performing steps S21 to S29, the processor 101 advances the process to step S41. The posture of the first marker Rm specified in step S29 of the transport process (step S3) is the above-mentioned "second posture".

[0096] In step S41, the processor 101 corrects the motion plan included in the teaching information 133 based on the first posture determined in step S29 of the motion plan creation process (step S2) and the second posture determined in step S29 of the transport process (step S3). As an example, when the processor 101 can determine that the first mark Rm is offset to the right by 1 cm compared with the motion plan creation process during the transport process by comparing the first posture with the second posture, the processor 101 corrects the motion plan in such a way that the trajectory of the arm 232 is offset to the right by 1 cm. In this way, a motion plan that matches the device configuration during the transport process is created.

[0097] In step S42, the processor 101 controls the movement of the arm 232 based on the corrected movement plan. More specifically, the processor 101 outputs a motor drive instruction to the driver corresponding to the movable part 8 as the object among the drivers included in the driver unit 237 based on the corrected movement plan. As a result, the movable part 8 as the object moves, so that the arm 232 moves up and down, left and right, and / or forward and backward, and the orifice plate 5 can be reliably transported to the target position Tp. After step S42, the transport process ends. Figure 4 The series of processes shown ends.

[0098] Thus, in the present embodiment, the management device 100 performs approach control to bring the camera unit 234 relatively close to the first mark Rm, so that the camera unit 234 can shoot the first mark Rm from nearby. Since the camera unit 234 shoots the first mark Rm from a closer position, the recognition accuracy of the first mark Rm is improved, and thus the posture of the first mark Rm is more accurately determined. Since the target position Tp is determined based on the relative positional relationship with the reference position Rp represented by the first mark Rm, being able to more accurately determine the posture of the first mark Rm means being able to more accurately determine the target position Tp. Therefore, the object can be reliably transported to the target position Tp.

[0099] Furthermore, even if the device configuration is changed during teaching or when transporting an object, the motion plan can be corrected, so the user does not need to perform the teaching operation again. Therefore, the user's labor and time can be reduced.

[0100] In addition, even when the device configuration is slightly changed during the teaching operation and when the object is transported, the posture of the first marker Rm can be more accurately determined by making the camera unit 234 shoot the first marker Rm from nearby. Therefore, the deviation amount between the posture of the first marker Rm during the teaching operation and the posture of the first marker Rm when the object is transported can be more accurately determined. Therefore, the correction amount of the movement of the arm 232 can be calculated more accurately, so the object can be reliably transported to the target position Tp.

[0101] In addition, the first marker Rm is set at a position different from the target position Tp. If the first marker Rm is set at the target position Tp, the first marker Rm is blocked by the object after the object is moved in. Therefore, when the object is to be moved out using the transport robot 230, the first marker Rm cannot be recognized and the object cannot be moved out correctly. However, in the present embodiment, the first marker Rm is set at a position different from the target position Tp, so when the object is to be moved out using the transport robot 230, the object can be moved out correctly from the target position Tp.

[0102] In addition, if the first marker Rm is set at the target position Tp, it may hinder the device at the destination of the object (for example, Figure 1 For example, when the working device 240 transmits light through the aperture plate 5 to perform measurement, if a mark that prevents light from being transmitted is provided at the setting position of the aperture plate 5, that is, the target position Tp, measurement cannot be performed correctly. However, in this embodiment, the first mark Rm is provided at a position different from the target position Tp, so that the operation of the device at the transport destination of the object is not hindered.

[0103] [Modification 1]

[0104] In the above-described embodiment, a transfer robot having one arm is used as the transfer robot. However, a transfer robot having two arms may be used instead of the transfer robot having one arm.

[0105] Fig. 9 2 is a diagram showing a transfer robot in Modification 1. The transfer robot 230A in Modification 1 is a six-axis vertical articulated robot having two arms. The transfer robot 230A includes a main body 231A, an arm 232A, an arm 238A, a gripper 233 , and an imaging unit 234 .

[0106] The main body 231A holds the arm 232A and the arm 238A. The arm 232A and the arm 238A share the three responsibilities of the arm 232 in the embodiment. The first responsibility is to search for the imaging object (for example, the first mark Rm, the second mark Tm, etc.). The second responsibility is to make the imaging unit 234 close to the first mark Rm. The third responsibility is to transport the orifice plate 5. The arm 232A has the third responsibility, and the arm 238A has the first responsibility and the second responsibility.

[0107] The arm 232A transports the orifice plate 5 from the working device 220 to the target position Tp in the working device 240. The arm 232A is an example of the "first moving body" of the present disclosure. The clamp 233 is provided at the front end of the arm 232A and is used to hold the orifice plate 5. The arm 232A includes one or more movable parts 8A. By moving one or more movable parts 8A, the arm 232A can move up and down, left and right, and front and back. Therefore, the orifice plate 5 is transported from the working device 220 to the target position Tp in the working device 240 by the arm 232A.

[0108] The arm 238A is an example of the "second moving body" of the present disclosure. The camera unit 234 is provided on the arm 238A and is used to obtain an image of the object to be imaged. The arm 238A includes one or more movable parts 9A. By moving one or more movable parts 9A, the arm 238A can move up and down, left and right, and front and back. Therefore, the object to be imaged can be searched and the camera unit 234 can be brought close to the first mark Rm.

[0109] The operations of the arm 232A, the arm 238A, the clamp 233, and the camera unit 234 are controlled by the management device 100. When the transport robot 230A is used as the transport robot, Figure 2 The illustrated motor unit 236 includes motors corresponding to one or more movable parts 8A, respectively, and motors corresponding to one or more movable parts 9A, respectively.

[0110] In addition, when the transport robot 230A is used as the transport robot, the management device 100 performs the control related to the first and second duties of the control of the arm 232 described above on the arm 238A, and performs the control related to the third duty on the arm 232A. In addition, when the transport robot 230A is used as the transport robot, the motion plan of the arm 232 described above is replaced by the motion plan of the arm 232A. As a result, the object can be reliably transported to the target position Tp.

[0111] In addition, the main body 231A may be configured to move relative to the target position Tp. As an example of a structure in which the main body 231A moves relative to the target position Tp, wheels are provided on the main body 231A. By driving the wheels, the main body 231A moves left and right and front and back. As a result, the transfer robot 230A moves left and right and front and back. In the case where the main body 231A is configured to move relative to the target position Tp, the movement of the main body 231A is controlled by the management device 100.

[0112] In the case where the main body 231A is configured to move relative to the target position Tp, in the above-mentioned conveying process (step S3), the main body 231A may be operated without operating the arm 232A and the arm 238A until the distance between the imaging unit 234 and the first mark Rm becomes less than the threshold value. Thus, the arm 232A and the arm 238A can be moved by the same amount at the same time. Therefore, at the time point when the distance between the imaging unit 234 and the first mark Rm becomes less than the threshold value, the arm 232A is operated from the position where the arm 232A is located to convey the orifice plate 5 to the target position Tp, so that the movement amount of the arm 232A can be suppressed. Therefore, the conveying accuracy of conveying the object to the target position Tp is improved.

[0113] [Modification 2]

[0114] In the above embodiment, three first marks Rm attached to the working device 240 are cited as an example as the "characteristic part", but the characteristic part is not limited to this. The "characteristic part" only needs to include one or more first marks Rm. In the case where multiple first marks Rm are provided, the target position Tp can be determined more accurately compared to the case where only one first mark Rm is provided. Therefore, the conveying accuracy of the object to the target position Tp is improved.

[0115] In addition, the characteristic portion may also be a part of the working device included in the working system 1 (for example, the working device 240, etc.), instead of one or more first marks Rm. The part of the working device included in the working system 1 is, for example, a mark engraved on the working device, a corner of the working device, etc. In the case of using a part of the working device included in the working system 1 as the characteristic portion, multiple parts of the working device may be used as the characteristic portion, or only one part of the working device may be used as the characteristic portion. In the case of using multiple parts of the working device as the characteristic portion, the target position Tp can be determined more accurately than in the case of using only one part of the working device as the characteristic portion. Therefore, the conveying accuracy of the object to the target position Tp is improved.

[0116] In addition, the characteristic portion may include one or more first marks Rm and one or more parts of the working device (for example, the working device 240 , etc.) included in the working system 1 .

[0117] [Variation 3]

[0118] In the above embodiment, as the approach control, an example of bringing the camera unit 234 close to the first mark Rm is cited, but the present invention is not limited thereto. When the working device 240 with the first mark Rm attached is configured to be movable under the control of the management device 100, the management device 100 may also perform control to bring the working device 240 close to the camera unit 234 as the approach control. In addition, when the working device 240 with the first mark Rm attached is configured to be movable under the control of the management device 100, the management device 100 may also perform control to bring the working device 240 and the camera unit 234 close to each other as the approach control.

[0119] [Variation 4]

[0120] The transfer robot 230 in the above-described embodiment may be configured to be able to travel on rails provided on the floor, or may be configured to be able to autonomously travel on the floor in an arbitrary pattern.

[0121] [Variation 5]

[0122] In the above-mentioned embodiment, the processing of registering the position information indicating the relative positional relationship between the target position Tp and the reference position Rp is performed using the transport robot 230, but the position information may be measured by the user and registered in the management device 100. In the case where the user measures the position information and registers it in the management device 100, it is preferable that the shape of the second mark Tm is the same as the shape of the object (for example, the orifice plate 5, etc.) transported by the transport robot 230. More specifically, it is preferable that the shape of the surface of the second mark Tm is the same as the shape of the bottom surface of the object transported by the transport robot 230. In the case where the shape of the surface of the second mark Tm is the same as the shape of the bottom surface of the object transported by the transport robot 230, it is easy for the user to grasp the position (target position Tp) where the object is transported, and thus the efficiency of the position information measurement operation is improved.

[0123] [Variation 6]

[0124] In the above embodiment, a case where the object is carried into the working device 240 using the transfer robot 230 is described. However, the above embodiment can also be applied to a case where the object is carried out from the working device 240 using the transfer robot 230.

[0125] [Way]

[0126] It will be understood by those skilled in the art that the above-described exemplary embodiments are specific examples of the following aspects.

[0127] (Item 1) A control method according to one embodiment is a method for controlling the transport of an object to a target position by a first moving body using a captured image obtained by a camera unit, the control method comprising the following steps:

[0128] performing approach control to make the imaging unit relatively approach a feature portion indicating a reference position different from the target position;

[0129] causing the imaging unit that has come relatively close to the characteristic portion through the approach control to acquire a first image obtained by photographing the characteristic portion; and

[0130] The first moving body is caused to transport the object to the target position based on the first image.

[0131] According to the control method described in the first item, the camera unit can capture the feature from the vicinity of the feature, so that the posture of the feature can be determined more accurately. The target position is determined based on the relative position relationship with the reference position represented by the feature. Therefore, if the posture of the feature is accurately determined, the target position is more accurately determined, thereby reliably transporting the object to the target position.

[0132] (Item 2) In the control method described in Item 1, the characteristic portion includes a plurality of elements that can be distinguished from each other.

[0133] According to the control method described in the second aspect, since a plurality of elements are provided, the target position can be determined more accurately than when only one element is provided. Therefore, the conveyance accuracy of the object to the target position is improved.

[0134] (Item 3) In the control method described in Item 1 or Item 2, the characteristic portion includes a first mark added to indicate the reference position.

[0135] According to the control method described in the third aspect, since the first mark is added to the reference position, the reference position can be clearly seen at a glance.

[0136] (Item 4) In the control method described in Item 1 or Item 2, the characteristic portion includes a portion of a working device for performing work using the object.

[0137] According to the control method described in the fourth aspect, since it is not necessary to intentionally add the first mark as the characteristic portion, it is possible to save the user's effort and time.

[0138] (Item 5) In the control method described in any one of Items 1 to 4, the imaging unit is provided on the first moving body.

[0139] According to the control method described in the fifth aspect, the number of moving bodies provided in the transfer robot can be suppressed.

[0140] (Item 6) In the control method described in any one of Items 1 to 4, the imaging unit is provided on a second moving body different from the first moving body.

[0141] According to the control method described in the sixth aspect, since the first moving body for conveying the object to the target position is not provided with an imaging unit, the first moving body becomes compact and can therefore move in a narrower space.

[0142] (Item 7) In the control method described in Item 6, a main body portion for holding the first moving body and the second moving body is configured to move relative to the target position.

[0143] According to the control method described in the seventh aspect, the first moving body and the second moving body can be moved simultaneously by the same amount.

[0144] (Item 8) In the control method described in any one of Items 1 to 7, in the registration process of registering the position information indicating the relative positional relationship between the target position and the reference position, a second mark indicating the target position is added to the target position,

[0145] The control method further comprises the following steps:

[0146] causing the imaging unit to acquire a second image obtained by photographing the second mark and the characteristic portion; and

[0147] The position information is determined based on the second image, and the position information is registered.

[0148] According to the control method described in the eighth aspect, since the user does not need to measure and register the position information, the user's effort and time can be saved.

[0149] (Item 9) Another embodiment of the transport device includes:

[0150] Camera Department;

[0151] A first moving body that transports the object to a target position; and

[0152] a control unit configured to perform approach control to make the imaging unit relatively approach a feature portion indicating a reference position different from the target position,

[0153] wherein the camera unit that is relatively close to the feature portion through the approach control acquires a first image obtained by photographing the feature portion,

[0154] The first moving body transports the object to the target position based on the first image.

[0155] According to the conveying device described in the ninth item, since the camera unit can capture the feature portion from the vicinity of the feature portion, the posture of the feature portion can be determined more accurately. The target position is determined based on the relative positional relationship with the reference position represented by the feature portion. Therefore, if the posture of the feature portion is accurately determined, the target position is determined more accurately, thereby reliably conveying the object to the target position.

[0156] (Item 10) In the conveying device described in Item 9, the characteristic portion includes a plurality of elements that can be distinguished from each other.

[0157] According to the conveying device described in the tenth item, since a plurality of elements are provided, the target position can be determined more accurately than when only one element is provided. Therefore, the conveying accuracy of conveying the object to the target position is improved.

[0158] (Item 11) In the conveying device described in Item 9 or 10, the characteristic portion includes a first mark added to indicate the reference position.

[0159] According to the conveying device described in the eleventh aspect, since the first mark is added to the reference position, the reference position can be clearly seen at a glance.

[0160] (Item 12) In the conveying device described in Item 9 or 10, the characteristic portion includes a portion of a working device for performing work using the object.

[0161] According to the conveying device described in the twelfth item, since it is not necessary to specially add the first mark as the characteristic portion, the user's labor and time can be saved.

[0162] (Item 13) In the conveying device described in any one of Items 9 to 12, the imaging unit is provided on the first movable body.

[0163] According to the transfer device described in the thirteenth item, the number of moving bodies provided in the transfer robot can be reduced.

[0164] (Item 14) In the conveying device described in any one of Items 9 to 12,

[0165] The conveying device further includes a second moving body different from the first moving body,

[0166] The imaging unit is provided on the second moving body.

[0167] According to the conveying device described in the fourteenth aspect, since the first movable body for conveying the object to the target position is not provided with an imaging unit, the first movable body becomes compact and can therefore move in a narrower space.

[0168] (Item 15) In the conveying device described in Item 14,

[0169] The conveying device further includes a main body for holding the first moving body and the second moving body.

[0170] The main body is configured to move relative to the target position.

[0171] According to the conveying device described in the fifteenth item, the first movable body and the second movable body can be moved simultaneously by the same amount.

[0172] (Item 16) In the conveying device described in any one of Items 9 to 15,

[0173] In a registration process of registering position information indicating a relative positional relationship between the target position and the reference position, a second mark indicating the target position is added to the target position.

[0174] The camera unit acquires a second image obtained by photographing the second mark and the characteristic portion,

[0175] The transport device further includes a registration unit configured to identify the position information based on the second image and register the position information.

[0176] According to the conveying device described in the sixteenth item, since it is not necessary for the user to measure and register the position information, the user's labor and time can be saved.

[0177] (Item 17) A work system according to another aspect comprises: the conveying device according to any one of Items 9 to 16; and a work device for performing work using an object conveyed by the conveying device.

[0178] According to the operation system described in item 17, the camera unit can capture the feature from the vicinity of the feature, so that the posture of the feature can be determined more accurately. The target position is determined based on the relative position relationship with the reference position represented by the feature. Therefore, if the posture of the feature is accurately determined, the target position is more accurately determined, thereby reliably transporting the object to the target position.

[0179] The embodiments disclosed herein are illustrative in all aspects and should not be considered restrictive. The scope of the present disclosure is indicated by the claims rather than the above description, and is intended to include all modifications within the meaning and scope equivalent to the claims.

[0180] Description of Reference Numerals

[0181] 1: operating system; 5: orifice plate; 8, 8A, 9A: movable part; 100: management device; 101: processor; 102: memory; 103: storage; 104, 235: interface; 105: display; 106: input device; 132: position information; 133: teaching information; 151: registration unit; 152: action plan creation unit; 153: action plan correction unit; 154: control unit; 200: controlled device group; 2 20, 240: working device; 230, 230A: transport robot; 231, 231A: main body; 232, 232A, 238A: arm; 233: clamp; 234: camera unit; 236: motor unit; 237: driver unit; 300: transport device; Rm, Rm1, Rm2, Rm3: first mark; Rp, Rp1, Rp2, Rp3: reference position; Tm: second mark; Tp: target position.

Claims

1. A control method for controlling the transport of an object to a target position by a first moving body using a captured image obtained by a camera unit, the control method comprising the following steps: performing approach control to make the imaging unit relatively approach a feature portion indicating a reference position different from the target position; causing the imaging unit, which has come relatively close to the characteristic portion through the approach control, to acquire a first image obtained by photographing the characteristic portion; as well as The first moving body is caused to transport the object to the target position based on the first image.

2. The control method according to claim 1, wherein: The characteristic portion includes a plurality of elements that can be distinguished from each other.

3. The control method according to claim 1, wherein: The characteristic portion includes a first mark added to indicate the reference position.

4. The control method according to claim 1, wherein: The characteristic portion includes a portion of a working device for performing work using the object.

5. The control method according to claim 1, wherein: The imaging unit is provided on the first moving body.

6. The control method according to claim 1, wherein: The imaging unit is provided on a second moving body different from the first moving body.

7. The control method according to claim 6, wherein: The main body portion for holding the first moving body and the second moving body is configured to move relative to the target position.

8. The control method according to claim 1, wherein: In a registration process of registering position information indicating a relative positional relationship between the target position and the reference position, a second mark indicating the target position is added to the target position. The control method further comprises the following steps: causing the imaging unit to acquire a second image obtained by photographing the second mark and the characteristic portion; and The position information is determined based on the second image, and the position information is registered.

9. A conveying device comprising: Camera Department; A first moving body that transports the object to a target position; and a control unit configured to perform approach control to make the imaging unit relatively approach a feature portion indicating a reference position different from the target position, in, The imaging unit that has come relatively close to the feature portion through the approach control acquires a first image obtained by photographing the feature portion, The first moving body transports the object to the target position based on the first image.

10. The conveying device according to claim 9, wherein: The characteristic portion includes a plurality of elements that can be distinguished from each other.

11. The conveying device according to claim 9, wherein: The characteristic portion includes a first mark added to indicate the reference position.

12. The conveying device according to claim 9, wherein: The characteristic portion includes a portion of a working device for performing work using the object.

13. The conveying device according to claim 9, wherein: The imaging unit is provided on the first moving body.

14. The conveying device according to claim 9, wherein: The conveying device further includes a second moving body different from the first moving body, The imaging unit is provided on the second moving body.

15. The conveying device according to claim 14, wherein: The conveying device further includes a main body for holding the first moving body and the second moving body. The main body is configured to move relative to the target position.

16. The conveying device according to claim 9, wherein: In a registration process of registering position information indicating a relative positional relationship between the target position and the reference position, a second mark indicating the target position is added to the target position. The camera unit acquires a second image obtained by photographing the second mark and the characteristic portion, The transport device further includes a registration unit configured to identify the position information based on the second image and register the position information.

17. An operating system comprising: The conveying device according to claim 9; and A working device is used to perform work using the object conveyed by the conveying device.

Images