Picking device

Abstract

To facilitate alignment of a position at which a robot is to be arranged.SOLUTION: A picking device comprises: a robot having a photographing device and picking a work-piece from a container storing the work-pieces and conveying the work-piece to a predetermined conveyance destination; a user interface for inputting and outputting information between a user and the user interface; and a control device that receives an operation from the user through the user interface, identifies the container and the conveyance destination on the basis of images photographed by the photographing device, and controls the motion of the robot. Further, the control device superimposes a guide showing a position of an object which is to be photographed from a position where the picking device should be arranged, on the images photographed by the photographing device, and displays the images on the user interface.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to a picking device.

Background Art

[0002] Conventionally, a production system has been proposed in which a connection mechanism includes a positioning means between adjacent units (for example, Patent Document 1). Also, a bonding device has been proposed in which a guide image indicating with a frame the position where a component is to be arranged with respect to the display area of a display unit is synthesized with an image captured by a photographing unit and displayed (for example, Patent Document 2).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In cases where the layout of a production line is frequently changed, such as in a food factory, or in cases where the position where a robot is arranged is changed in a flow operation, there has been a problem that it takes time to align the position of the place where the robot is arranged. An object of the present invention is to provide a technique for facilitating the alignment of the position of the place where a robot is arranged.

Means for Solving the Problems

[0005] The picking device comprises an imaging device, a robot that picks workpieces from a container and transports them to a predetermined destination, a user interface for inputting and outputting information with the user, and a control device that accepts operations from the user via the user interface, identifies the container and destination based on the image captured by the imaging device, and controls the robot's movement. The control device also overlays a guide indicating the position of the object being photographed from the position where the picking device should be placed onto the image captured by the imaging device and displays it on the user interface.

[0006] Furthermore, the object may be the container described above, and the guide may represent the position, size, and orientation of the container. Alternatively, the guide may be a figure representing the outer shape of the container, and may be dynamically generated based on the position, size, and orientation of the container.

[0007] Furthermore, the control device may display a guide stored in the storage device, associated with the type of container, on the user interface, depending on the type of container input via the user interface.

[0008] The system may also include a means of moving the robot and a stopper to restrict the robot's movement. Furthermore, the workpiece may be food, and the container may be a traditional Japanese food box.

[0009] Furthermore, the contents described in the means for solving the problem can be combined as much as possible without departing from the problem or technical idea of ​​the present invention. In addition, the contents of the means for solving the problem can be provided as a device such as a computer or a system including multiple devices, a method executed by a computer, or a program to be executed by a computer. Furthermore, a recording medium for holding the program may also be provided. [Effects of the Invention]

[0010] This technology can provide a way to easily align the robot to its placement location. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 is a perspective view showing an example of the configuration of a picking device. [Figure 2] Figure 2 is a functional block diagram showing an example of the functional configuration of a picking device. [Figure 3] Figure 3 is a side view showing an example of the picking device in use. [Figure 4] Figure 4 is a process flow diagram showing an example of the alignment process performed by a picking device. [Figure 5] Figure 5 shows an example of information regarding the arrangement of the first container. [Figure 6] Figure 6 shows an example of a guide displayed on a monitor. [Modes for carrying out the invention]

[0012] <Embodiment 1> Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

[0013] <Device configuration> Figure 1 is a perspective view showing an example of the configuration of the picking device 100. Figure 2 is a functional block diagram showing an example of the functional configuration of the picking device 100. Figure 3 is a schematic side view showing an example of the picking device in use. The picking device 100 can be used in kitchens such as central kitchens and in production lines of food processing plants. For example, the picking device 100 picks up food items such as fried chicken that are loosely packed in a first container such as a food box 4 (Figure 3) and places them in a second container such as a bento box 31 (Figure 3) that is transported by a conveying device such as a belt conveyor 3 (Figure 3). The picking device 100 also displays a guide to the user indicating the position where the device should be placed. The position where the picking device 100 should be placed is determined relative to the first or second container based on its range of motion, etc.

[0014] As shown in Figure 1, the picking device 100 comprises a robot 1 and a trolley 2 on which the robot 1 is fixed. The robot 1 also has robot arms 11 (11A, 11B), end effectors 12 (12A, 12B), and imaging devices 13 (13A, 13B). The trolley 2 comprises a user interface (UI) 21, casters 22, and a handle 23, and has a control device 24 (Figure 2) within its housing that controls the operation of the picking device 100. The UI 21 also includes buttons 211, a monitor 212, and a speaker 213. As shown in Figure 3, the bottom of the trolley 2 may be equipped with a fixing device (stopper) 25 for fixing the trolley 2 to the installation location.

[0015] <Robot> Robot 1 is a multi-joint robot that mimics the upper body of a human, equipped with, for example, robot arms 11 having 7 degrees of freedom on both arms. Robot 1 is also equipped with motors in its torso, allowing it to twist. The robot arms 11 are so-called vertical multi-joint robots, alternately comprising links, which are skeletal members that displace as a whole, and joints that connect the links and displace the angles between them. Note that the multi-joint robot shown in Figure 1 is an example of Robot 1 according to the present invention, and Robot 1 may have three or more robot arms. Furthermore, at least some of the robot arms of Robot 1 do not have to be vertical multi-joint robots. That is, Robot 1 may be a system that includes known robots such as horizontal multi-joint robots, Cartesian robots, or parallel link robots.

[0016] Furthermore, the materials used for the links and other components of Robot 1 are not particularly limited and can be resin, metal, carbon, etc. The molding method is also not particularly limited and can be manufactured by plastic deformation or injection molding, or it can be fabricated using a 3D printer.

[0017] The robot arm 11 is provided with an end effector 12 including an arbitrary mechanism at its tip. The end effector 12 shown in FIG. 1 is a two-finger gripper that detachably holds a pair of two tongs with the fulcrum part cut off. Note that the end effector 12 may be modeled after cooking utensils other than tongs, such as a ladle or a ball. Also, it may be a multi-finger gripper with three or more fingers, a multi-finger gripper in which the fingers rotate, a multi-finger gripper in which the fingers perform linear motion, a multi-finger gripper in which some of these fingers are fixed, a multi-finger gripper using a parallel link mechanism, a underactuated hand, a tendon-driven hand, a suction (vacuum) gripper, etc.

[0018] The robot arm 11 has a built-in servo motor at the joint and is connected to a control device 24 that controls the rotation angle of the servo motor via a signal line. Also, the end effector 12 also has a servo motor and is electrically connected to the control device 24 via a predetermined connector, and the rotation of the servo motor is controlled by the control device 24. Note that the control device 24 may be built into the robot 1 or may be externally attached. Also, the control device 24 may be a computer that is connected via a communication network such as the Internet or a LAN (Local Area Network) and provides services on so-called clouds.

[0019] Also, the robot 1 is provided with two imaging devices 13. That is, the robot 1 is provided with an imaging device 13A on the head and an imaging device 13B on the chest. The imaging device 13 may be a digital camera provided with an image sensor using a CCD (Charge-Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), etc., or may be an RGB-D camera capable of measuring depth. By providing two imaging devices, when the robot 1 handles some workpiece, the dead angle of the robot arm can be reduced.

[0020] <Cart> As shown in Figure 1, the trolley 2 has a rectangular parallelepiped housing, and the robot 1 is fixed to one end of its upper surface (for convenience, referred to as the "front" or "front side"). On the other end of the upper surface of the trolley 2 (for convenience, referred to as the "rear" or "(back side)"), there are several buttons 211 and a monitor 212. The monitor 212 is assumed to be an input / output device such as a touch panel. A speaker 213 is also located on the side of the trolley 2. The trolley 2 also has casters 22 on its bottom and a handle 23 on its rear side, allowing the user to easily move the picking device 100. As shown in Figure 3, the trolley 2 may be equipped with a fixing device 25 on its bottom (rear in the example in Figure 3). The fixing device 25 is a support mechanism that, for example, pushes up and supports (jacks up) the trolley 2, and can separate at least some (for example, the rear) of the casters 22 from the floor surface. Furthermore, in place of the fixing device 25, or in addition to the fixing device 25, at least some (for example, the rear) casters 22 may be equipped with stoppers to restrict the rotation of the wheels. In the example in Figure 3, the rear casters 22 are swivel casters equipped with stoppers that rotate around a main shaft extending vertically, while the front casters 22 are fixed casters whose direction of travel does not change. Also, in Figure 3, the fixing device 25 and the rear casters 22 are shown offset front to back for convenience, but the fixing device 25 and the rear casters 22 may be arranged so that they overlap in a side view. In addition, as shown in Figure 2, the control device 24 is housed inside the housing of the trolley 2.

[0021] The control device 24 is a computer and includes a processor 241 and a storage device 242. The control device 24 is also connected to the robot 1 and UI 21 via an input / output interface including predetermined connectors and cables.

[0022] Processor 241 is a processing unit such as a CPU (Central Processing Unit), The program according to this embodiment performs predetermined processing. Note that the processor 241 is not limited to a single processor, but may be a multi-processor configuration. Furthermore, the processor 241 may be a single processor connected via a single socket, and may have a multi-core configuration. In addition, at least a portion of the processing of the picking device 100 may be provided by a dedicated processor such as a Digital Signal Processor (DSP), Graphics Processing Unit (GPU), numerical processing processor, vector processor, image processing processor, or Application Specific Integrated Circuit (ASIC). At least a portion of the King device 100 is a Field-Programmable Gate Array (FPGA), etc. It may also be a dedicated large-scale integration (LSI) or other digital circuit.

[0023] The processor 241 receives user input for the picking device 100 via the UI 21. Before starting the robot 1 to move, the processor 241 displays information for positioning the picking device 100 on the monitor 212. The positioning information is, for example, a guide indicating the position of the object to be imaged from the position where the picking device 100 should be placed, and is displayed on the monitor 212 superimposed on the image data (hereinafter simply referred to as "image") output by the imaging device 13. The user moves the trolley 2 so that the position, size, and orientation of the object in the image displayed on the monitor 212 match the guide. The orientation of the imaging device 13 during positioning is assumed to be constant because the robot 1 assumes a predetermined posture. The user may also adjust the orientation of the object to match the guide. In addition, multiple guides may be stored in the storage device 242, for example, corresponding to the type of workpiece.

[0024] When the robot 1 is instructed to perform a serving operation, the processor 241 acquires information indicating the angles of the motors of each joint of the robot arm 11 and the end effector 12, as well as data output by the imaging device 13, and stores it in the storage device 242. The processor 241 then determines the operation of the robot 1 based on the data output by the imaging device 13 and the rotation angles of each joint of the robot arm 11. Specifically, the processor 241 determines the position for picking one of the workpieces placed in the first container and the place position in the second container where the picked workpiece will be placed, based on the image output by the imaging device 13, and calculates the required rotation angles for each joint from the target position of the end effector 12 based on inverse kinematics. The second container, which is the place position, is transported by a transport device. Therefore, the processor 241 determines the place position taking into account the time required for the movement of the second container and the movement of the end effector 12. The processor 241 may also perform feedback processing based on the operation of the robot 1 captured by the imaging device 13 and adjust the place position. Furthermore, the processor 241 changes the angles of the joints of the robot arm 11 based on the determined picking position and place position, and moves the end effector 12 to follow the second container while placing the workpiece in the second container.

[0025] The storage device 242 includes main memory such as RAM (Random Access Memory) and ROM (Read Only Memory), and auxiliary storage devices (secondary storage devices) such as HDD (Hard Disk Drive), SSD (Solid State Drive), and flash memory. The main memory temporarily stores programs read by the processor 241 and reserves the processor's workspace. The auxiliary storage device stores programs executed by the processor 241 and information used for the operation of the robot 1. Note that the processor 241 and storage device 242 may be microcontrollers with built-in memory.

[0026] <Alignment process> Figure 4 is a process flow diagram showing an example of the alignment process performed by the picking device 100. Yes. As mentioned above, during the alignment process, a guide is displayed to help the user align the trolley 2.

[0027] First, the processor 241 of the control device 24 accepts the selection of a serving operation to be performed by the picking device 100 via the UI 21 (Figure 4: S1). In this step, input such as the type of the first container is accepted. That is, the user selects the type of the first container from a predetermined list of candidates according to the size of the container to be used. In addition to, or instead of, the type of the first container, the system may also accept inputs such as the direction in which the belt conveyor 3 transports the second container, the type of the second container, and the type of workpiece. For example, the direction in which the second container is transported indicates whether it is from right to left or left to right as viewed from the robot 1. The type of the second container indicates the shape of the lunch box and the position in the partitioned area of ​​the lunch box where the workpiece is placed. The type of workpiece indicates the type of food that the picking device 100 handles.

[0028] In this embodiment, the size of the first container, as well as the position and orientation in which the first container is placed, are defined in association with the type of the first container. Figure 5 is an example of information regarding the arrangement of the first container, which is pre-stored in the storage device 242. The table in Figure 5 includes the attributes "type," "size," "position," and "orientation." The "type" field stores identification information indicating the type of the first container. The "size" field stores information indicating the length, width, and height of the first container. The "position" field stores information indicating the position of the first container in a predetermined coordinate system. The "orientation" field stores information indicating the rotation angle representing the orientation of the first container in a predetermined coordinate system. Note that multiple first containers of the same size or different sizes may be placed.

[0029] Furthermore, the processor 241 calculates the position for displaying a guide on the monitor 212 according to the type of the first container (Figure 3: S2). In this step, information indicating the shape of the first container as seen from the imaging device 13 of the robot 1 is calculated using the information shown in Figure 5. For example, the size, position, and orientation of the first container, determined based on the information shown in Figure 5, are converted into a view coordinate system as seen from the imaging device 13 of the robot 1.

[0030] Furthermore, the processor 241 displaces the robot 1 to a predetermined posture (Figure 3: S3). In this step, the robot 1 is made to assume a predetermined posture so that the imaging device 13 faces a predetermined direction. If, for example, the imaging device 13A is used for alignment, the servo motors of the waist and neck of the robot 1 are displaced. If, for example, the imaging device 13B is used, the servo motor of the waist of the robot 1 is displaced. If the robot 1 is already in a predetermined posture, the process in S3 is omitted. The processes in S2 and S3 may be executed in reverse order or in parallel. Also, for example, the elevation and depression angles of the servo motor of the neck of the robot 1 may be dynamically determined in S2 based on information such as the position of the first container read in S1, so that the guide is positioned in the center in the view coordinate system converted in S2. In this case, in S2, a guide shape is calculated that corresponds to the angle that the imaging device 13 can face and the distance that the end effector 12 reaches to the workpiece placed on the first container.

[0031] The processor 241 then superimposes the guide image, whose position and other properties were calculated in S2, onto the image output by the imaging device 13 and displays it on the monitor 212 (Figure 3: S4). Figure 6 shows an example of the guide displayed on the monitor 212. In the example in Figure 6, the guide, shown by a dashed line, is superimposed on the image output by the imaging device 13. The guide is a figure representing the outer shape of the first container, and in the example in Figure 6, it is a rectangular parallelepiped. Furthermore, until the "OK" button shown in Figure 6 is pressed, the processor 241 will display the image continuously output by the imaging device 13 on the monitor 212 at a predetermined frequency.

[0032] <Effects> According to the alignment process described above, the user can position the picking device 100 in the appropriate location by moving the trolley 2 and aligning it with the guide and the first container displayed on the monitor 212. The user may also change the angle of the container 4 (first container) relative to the support base 41 shown in Figure 3, according to the guide. In other words, even if the type of work handled by the picking device 100 changes, or if the layout of the production line changes, the user can easily align the relative position of the first container and the picking device 100. In this embodiment, as shown in Figure 3, the belt conveyor 3 (transport device) is arranged to cross between the container 4 and the picking device 100. Therefore, by aligning the relative position of the first container and the picking device 100, the picking device 100 can also place work in the lunch box container 31 (second container) moving on the belt conveyor 3.

[0033] Furthermore, the guide for the first container corresponding to the workpiece may be prepared as an image, for example, with a transparent background, as shown by the dashed line in Figure 6. However, by dynamically calculating the position of the guide, etc., as shown in S2 of Figure 3, the amount of data stored in the storage device 242 can be reduced, and a new type of second container can be added simply by inputting the dimensions of the second container, etc.

[0034] Furthermore, for example, a table that holds the correspondence between the type of workpiece and the type of the first container may be stored in the storage device 242 in advance. In this case, in S1 of Figure 4, the type of the first container corresponding to the type of workpiece selected by the user is identified, and the "size," "position," and "orientation" of the identified first container are read from the table shown in Figure 5. Note that multiple first containers of the same or different sizes may be associated with the type of workpiece, in which case the display positions for multiple guides are calculated in S2. Also, one picking device 100 may pick and place multiple types of workpieces, in which case as well, the display positions for multiple guides corresponding to multiple first containers are calculated in S2.

[0035] <Other> The above-described configuration is merely an example, and the present invention is not limited to the illustrated configuration. The above-described matters can be appropriately combined and implemented without departing from the problems to be addressed or the technical idea of ​​the present invention. Furthermore, while the picking device 100 can be suitably used in production lines such as food processing plants, the workpieces handled by the picking device 100 are not limited to food products.

[0036] Furthermore, the monitor 212 may be installed on the robot 1 instead of the trolley 2, or it may be installed on a computer such as a tablet connected via wireless communication such as a Wi-Fi network.

[0037] Furthermore, the configuration does not have to be the combination of robot 1 and trolley 2 as described above. For example, the housing of robot 1 may be equipped with UI 21, casters 22, fixing device 25, etc., and may have a built-in control device 24. Also, the means of movement provided by the picking device 100 may be a track instead of a rolling device such as casters 22, or it may be equipped with both.

[0038] Furthermore, the imaging device 13 may be provided independently of the robot 1. For example, the imaging device 13, which is supported to look downwards, and the robot 1, which is a vertical articulated robot or a horizontal articulated robot, may be fixed to the trolley 2. Even in such an example, the position of the picking device 100 can be determined according to the guide displayed on the UI 21. It is possible to perform a joint operation.

[0039] Furthermore, the present invention includes a computer program that performs the above-described processing. Moreover, a computer-readable recording medium on which the program is recorded also falls within the scope of the present invention. The above-described processing becomes possible by having a computer read and execute the program on the recording medium on which the program is recorded.

[0040] Here, a computer-readable recording medium refers to a recording medium that stores information such as data and programs through electrical, magnetic, optical, mechanical, or chemical means and can be read by a computer. Examples of such recording media that are removable from a computer include flexible disks, magneto-optical disks, optical disks, magnetic tapes, and memory cards. Examples of recording media that are fixed to a computer include hard disk drives and ROMs. [Explanation of Symbols]

[0041] 100: Picking device 1: Robot 11(11A, 11B): Robot arm 12 (12A, 12B): End effector 13 (13A, 13B): Imaging device 2: Dolly 21: User Interface (UI) 211: Button 212: Monitor (touch panel) 213: Speaker 22: Caster 23: Toride 24: Control device 241: Processor 242: Storage device 25:Fixing device 3: Belt conveyor (conveying device) 31: Lunch box container (second container) 4: Container (first container) 41: Support stand

Claims

1. Imaging device and A robot that picks the workpiece from a container holding the workpiece and transports it to a predetermined destination, A user interface for inputting and outputting information with the user, A control device that receives operations from the user via the user interface, identifies the container and the destination based on the image captured by the imaging device, and controls the operation of the robot, A picking device equipped with, The control device superimposes a guide indicating the position, size, and orientation of the container, as captured from the position where the picking device should be placed, onto the image captured by the imaging device, and displays it on the user interface in a manner visible to the user. This allows the user to align the robot by moving it so that the container and the guide in the image overlap while viewing the display. Picking device.

2. The guide is a graphic representation of the container's external shape, and is dynamically generated based on the container's position, size, and orientation. The picking device according to claim 1.

3. The control device, in accordance with the type of container input via the user interface, causes the user interface to display the guide stored in the storage device in association with that type. The picking device according to claim 1 or 2.

4. Imaging device and A robot that picks the workpiece from a container holding the workpiece and transports it to a predetermined destination, A user interface for inputting and outputting information with the user, The system accepts operations from the user via the user interface, identifies the container and the destination based on the image captured by the imaging device, and controls the robot's operation. A control device that controls and A picking device equipped with, The control device superimposes a guide, which indicates the position of the container as captured from the position where the picking device should be placed, onto the image captured by the imaging device. This guide is stored in a storage device and corresponds to the type of container input via the user interface. By displaying this guide on the user interface in a way that is visible to the user, the user can move the robot so that the container and the guide in the image overlap while viewing the display, thereby aligning the robot. Picking device.

5. A means for moving the robot, A stopper for restricting the movement of the robot, A picking device according to any one of claims 1 to 4, further comprising the above.

6. The work is food, and the container is a food box. A picking device according to any one of claims 1 to 5.

Images