Robot system, robot hand, position detection method, and program

Abstract

To provide a technology which can reduce costs when a spatial position of an object stacked on a pallet is detected.SOLUTION: A robot system 1 handles an object (a container 7 etc.) stacked on a pallet 6 and includes: a robot 2; a distance sensor which detects a height of an upper surface of the object; a two-dimensional camera which detects a horizontal position of the object; and a controller 5. The controller 5 specifies a position of the object in a space based on the upper surface height and the horizontal position detected by the distance sensor and the two-dimensional camera.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a robot system that uses a robot to handle objects loaded on a pallet, and to techniques related thereto. [Background technology]

[0002] As a method for detecting the spatial position (spatial position) of an object loaded on a pallet, there is a method that uses a 3D camera.

[0003] For example, Patent Document 1 describes a technique for acquiring the spatial position (three-dimensional position) of an object placed on a pallet using a three-dimensional camera. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-25743 Summary of the Invention [Problem to be solved by the invention]

[0005] However, 3D cameras are expensive, so technology that can reduce costs is desirable.

[0006] Therefore, an object of the present invention is to provide a technique that can reduce costs when detecting the spatial position of an object loaded on a pallet. [Means for solving the problem]

[0007] In order to solve the above problem, the robot system of the present invention is a robot system that handles objects loaded on a pallet, and includes a robot that handles the objects, a distance sensor that detects the top surface height of the objects, a two-dimensional camera that detects the horizontal position of the objects, and a control means, wherein the control means identifies the position of the objects in space based on the detected top surface height and horizontal position.

[0008] The distance sensor may be fixed to the robot, and the two-dimensional camera may be fixed to the robot.

[0009] The robot may include a robot arm and a robot hand attached to the end of the robot arm, the distance sensor may be fixed to the robot hand, and the two-dimensional camera may be fixed to the robot hand.

[0010] The control means may drive the robot to change the horizontal position of the distance sensor and detect the height of the upper surface of the object at a plurality of different horizontal positions.

[0011] The control means may determine the loading direction of the plurality of objects on the top shelf based on the top surface heights detected at a plurality of different horizontal positions.

[0012] A plurality of objects may be stacked with pinholes on the pallet, and the control means may determine the orientation of each object stacked with pinholes on the top shelf based on the top surface height detected at a plurality of different horizontal positions.

[0013] The object has an approximately rectangular shape when viewed from above, and has a colored portion colored in a specific color on one of the four sides of the approximately rectangular shape, and the control means may determine which of four horizontal orientations the object has by identifying the one side of the object based on an image captured by the two-dimensional camera.

[0014] The control means may drive the robot to move the two-dimensional camera to a position a predetermined distance vertically away from the top surface of the object that is at the top of the pallet, and then use the two-dimensional camera to capture an image of the object.

[0015] The robot hand may be capable of rotating around a vertical axis in response to driving of one rotary joint of the robot arm.

[0016] The control means may cause the two-dimensional camera to photograph the object while changing the optical axis direction of the two-dimensional camera from horizontal to vertical and moving the vertical position of the two-dimensional camera upward in response to driving of one rotary joint of the robot arm.

[0017] In order to solve the above problems, the robot hand of the present invention is characterized by having a distance sensor that detects the height of the top surface of an object loaded on a pallet, and a two-dimensional camera that detects the horizontal position of the object.

[0018] In order to solve the above problem, the position detection method of the present invention is a position detection method for detecting the position in space of an object loaded on a pallet, and is characterized by comprising the steps of: a) detecting the top surface height of the object using a distance sensor; b) detecting the horizontal position of the object using a two-dimensional camera; and c) identifying the position of the object in space based on the detected top surface height and horizontal position.

[0019] In order to solve the above problem, the present invention provides a program for causing a computer to execute the above position detection method. [Effects of the Invention]

[0020] According to the present invention, it is possible to reduce costs. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 2 is a top view of the robot system. [Figure 2] FIG. 1 is an elevation view (front view) of the robot system. [Figure 3] FIG. 2 is a perspective view of the robot hand as seen from diagonally below. [Figure 4] FIG. 2 is a perspective view of the robot hand as seen from diagonally above. [Figure 5] FIG. 1 is a front view of a robot hand (unheld state). [Figure 6] FIG. 1 is a front view of a robot hand (holding state). [Figure 7] FIG. 1 is a side view of a robot hand (unheld state). [Figure 8] FIG. 1 is a side view of a robot hand (holding state). [Figure 9] FIG. 1 is a top view of the robot hand (unheld state). [Figure 10] FIG. 1 is a top view of the robot hand (holding state). [Figure 11] 10 is a flowchart showing a depalletizing process. [Figure 12] FIG. 10 is a diagram illustrating the orientation of pinhole stacking. [Figure 13] FIG. 10 is a diagram showing a distance measurement result obtained by a distance sensor. [Figure 14] FIG. 10 is a diagram illustrating a change in the attitude of a two-dimensional camera. [Figure 15] FIG. 10 is a diagram showing the field of view of a two-dimensional camera suitable for detecting corners of a container. [Figure 16] FIG. 10 is a diagram showing the field of view of a two-dimensional camera suitable for detecting a colored portion. [Figure 17] FIG. 1 is a diagram showing landmarks on a mobile robot. [Figure 18] FIG. 10 is a diagram showing a state in which a plurality of containers are transferred onto a mobile robot. [Figure 19] FIG. 10 is a diagram showing a state in which the robot hand picks up a container on a pallet. [Figure 20] FIG. 10 is a top view showing a state in which a container on a pallet is picked up by a robot hand. [Figure 21] FIG. 1 illustrates a robotic hand positioned at a ready position and having a ready state. [Figure 22] This is a top view showing two of the top four containers still remaining on the pallet. DETAILED DESCRIPTION OF THE INVENTION

[0022] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0023] <1. System Overview> FIG. 1 is a top view showing the robot system 1, and FIG. 2 is an elevation view (front view) showing the robot system 1. As shown in FIG.

[0024] As shown in FIG. 1, the robot system 1 includes a robot 2, a distance sensor (laser sensor (laser distance measuring sensor)) 3 (see FIG. 3), a two-dimensional camera (two-dimensional image sensor, etc.) 4 (see FIG. 3), and a controller (control unit) 5.

[0025] The robot 2 is configured with a robot arm 2a and a robot hand 2b. The robot hand 2b is attached to the end of the robot 2 (more specifically, the end of the robot arm 2a). The distance sensor 3 and the two-dimensional camera 4 are fixedly disposed at predetermined positions on the robot hand 2b (see FIG. 3).

[0026] The robot system 1 is a system that uses a robot 2 to handle (pick up, etc.) objects (containers 7 in this case) loaded on a pallet 6. The robot system 1 is a system that can remove (depalletize) objects (containers 7, etc.) loaded on the pallet 6 from the pallet 6 using the robot 2, and is also called a depalletizing system. The robot system 1 can perform depalletizing, etc. on two pallets 6 arranged in two locations (on the left and right sides in FIG. 1).

[0027] Here, the target object is exemplified by a container 7. The container 7 is a foldable (assembly type) container, and when in use (assembled), has a roughly rectangular box shape with an open top (see Figures 4 and 18, etc.). Each container 7 is loaded with, for example, a plurality of small boxes (baggage) each containing a product (goods).

[0028] The container 7 has a generally rectangular shape in top view and includes two side portions 71 corresponding to the two long sides and two side portions 72 corresponding to the two short sides (see FIG. 12, etc.). Each side portion 71 has a side panel 75, and each side portion 72 has a side panel 76 (see FIG. 4, etc.). The four side portions 71, 71, 72, 72 each have protrusions 73, 73, 74, 74 that protrude outward from the upper outer periphery. More specifically, the protrusion 73 (see FIGS. 4 and 7, etc.) is provided on the upper outer periphery of the side portion 71 corresponding to the long sides, and the protrusion 74 (see FIGS. 4 and 7, etc.) is provided on the upper outer periphery of the side portion 72 corresponding to the short sides. The protrusion 73 protrudes outward beyond the outer surface of the side panel 75, and the protrusion 74 protrudes outward beyond the outer surface of the side panel 76. In other words, recesses (75, 76) are provided below the protruding portions 73, 74. The robot hand 2b holds the container 7 by supporting the protruding portions 73, 74 from below.

[0029] A plurality of containers 7 are stacked in multiple tiers (four tiers in this example) on a pallet 6 (see FIG. 2). Four containers 7 are arranged (e.g., horizontally arranged) in each tier in pinhole stacking (windmill stacking) (see FIG. 1). In this pinhole stacking, a hollow portion (hereinafter also referred to as a pinhole) 6h is created in the center of the pallet 6. In addition, in order to improve loading efficiency, each container 7 on each tier is arranged in contact with an adjacent container 7. In detail, one side surface 72 of each container 7 is in contact with a side surface 71 of the adjacent container 7 (see FIG. 12).

[0030] The robot 2 can remove the container 7 from the pallet 6. The container 7 removed from the pallet 6 by the robot 2 is transferred to a mobile robot 8 (see FIGS. 1 and 17) and transported to another location by the mobile robot 8.

[0031] The system including the mobile robot 8 and the robot system 1 is also called a transport system.

[0032] The robot arm 2a is configured as, for example, a vertical articulated robot. The robot arm 2a has six drive axes (six rotary joints), and can freely change the position and posture (six degrees of freedom in total) of the end effector (and thus the robot hand 2b) within its range of motion.

[0033] For example, a robot arm 2a has six rotational joints, from a first rotational joint to a sixth rotational joint, arranged in this order from a base portion 2c of the robot arm 2a toward a hand portion (see FIG. 2). The first rotational joint is a rotational joint that can rotate about a vertical axis relative to the base portion 2c, and the second and third rotational joints are two rotational joints that can rotate about horizontal axes. The fourth rotational joint is a rotational joint that can rotate about an axis parallel to the extension direction of the link 2d on the hand side of the third rotational joint. The fifth rotational joint is a rotational joint that can rotate about an axis perpendicular to the rotational axis of the fourth rotational joint. The sixth rotational joint is a rotational joint that can rotate about an axis perpendicular to both the rotational axis of the fourth rotational joint and the rotational axis of the fifth rotational joint. Note that FIGS. 1 and 2 show the robot arm 2a in a simplified schematic view. Also, in FIG. 2, the rotation direction of each rotational joint of the robot arm 2a is indicated by arrows.

[0034] The robot arm 2a has a sufficiently wide range of motion for performing depalletizing work. The robot arm 2a can move the robot hand 2b attached to its tip to each position of multiple containers 7 stacked in multiple layers on a pallet 6. The robot arm 2a also has a range of motion that allows it to move the container 7 held by the robot hand 2b on the pallet 6 onto the mobile robot 8 (FIG. 1) and place it on the mobile robot 8.

[0035] The robot hand 2b has a configuration that is particularly suitable for depalletizing (removing the containers 7 from the pallet 6) a plurality of containers 7 that have been pinhole stacked (also called pinwheel stacking (see FIGS. 1 and 19, etc.)) on a pallet 6. The configuration of the robot hand 2b will be described in detail later. FIG. 19 is a schematic perspective view showing how the robot hand 2b picks up a container 7 on a pallet 6. The robot arm 2a is not shown in FIG. 19, etc.

[0036] The distance sensor 3 (see FIG. 3) and the two-dimensional camera 4 detect the position (three-dimensional position) and posture (orientation) of the container 7. Specifically, the distance sensor 3 detects the height (vertical position) of the top surface of the container 7 (specifically, the topmost container 7), and the two-dimensional camera 4 detects the horizontal position (two-dimensional position) of the container 7 (see FIG. 14). The distance sensor 3 and the two-dimensional camera 4 are fixedly disposed on the robot hand 2b. The position and posture (position and posture) of the distance sensor 3 and the position and posture of the two-dimensional camera 4 are changed in response to changes in the position and posture of the robot hand 2b caused by the driving of the robot 2. As will be described later, the distance sensor 3 is also used in the process of determining the orientation of pinhole stacking. The two-dimensional camera 4 is also used in the process of detecting the colored portion (coloring portion) 78 (see FIG. 16) of the container 7 to determine the posture of the container 7 (which of the four orientations D1 to D4 in the horizontal plane it has).

[0037] These processes are controlled by the controller 5 (Fig. 1). The controller 5 is a control device that controls the depalletizing process, etc. The controller 5 detects the position and posture (direction) of the container 7, and controls the operation of driving the robot 2 (robot arm 2a and robot hand 2b) to remove the container 7 from the pallet 6, etc., based on the detected top surface height and horizontal position.

[0038] The controller (also referred to as a control unit) 5 is configured as a computer system (also referred to simply as a computer) including one or more hardware processors (for example, a central processing unit (CPU) and a graphics processing unit (GPU)). The controller 5 performs various processes by executing, in the CPU or the like, a predetermined software program (hereinafter also referred to simply as a program) stored in a storage unit (a non-volatile storage unit such as a ROM and / or a hard disk). The program (more specifically, a group of program modules) (also referred to as a "program product") may be recorded on a portable storage medium such as a USB memory, read from the storage medium, and installed on the computer. Alternatively, the program may be downloaded via a communication network or the like and installed on the computer.

[0039] The controller 5 is also configured to include a storage unit, a communication unit, an operation unit, etc. The storage unit is configured with a storage device such as a hard disk drive (HDD) and / or a solid state drive (SSD). The communication unit is capable of performing network communication (wireless communication, etc.) via a network. Network communication uses various protocols such as TCP / IP (Transmission Control Protocol / Internet Protocol). By using this network communication, the robot 2 can exchange various types of data with desired destinations (for example, the mobile robot 8 and a server that controls the overall system). The operation unit includes an operation input unit that accepts various setting operations related to the robot 2, and a display unit that displays and outputs various types of information.

[0040] <2. Configuration of robot hand 2b> Figures 3 and 4 are diagrams showing the schematic configuration of the robot hand 2b. Figure 3 is a perspective view of the robot hand 2b in a horizontal position (position holding a container 7) as seen obliquely from below, and Figure 4 is a perspective view of the robot hand 2b in the horizontal position as seen obliquely from above. Figure 4 also shows the robot hand 2b holding a container 7 (shown by a dashed line).

[0041] 5 and 6 are front views of the robot hand 2b. Fig. 6 shows a state (holding state) in which the holding unit 40 is in contact with the inner side surface (inner wall surface) of the side portion 72 (specifically, the side panel 76) of the container 7. Fig. 5 shows a state (non-holding state) in which the holding unit 40 is separated from the inner side surface. In Figs. 5 to 10, the container 7 is indicated by a two-dot chain line.

[0042] 7 and 8 are side views (right side views) of the robot hand 2b as viewed from the right side. Fig. 8 shows a state (holding state) in which the holding part 20 is in contact with the side part 71 (more specifically, the protruding part 73) of the container 7, and Fig. 7 shows a state (non-holding state) in which the holding part 20 is separated from the side part 71 (the protruding part 73).

[0043] 9 and 10 are views (top views) of the robot hand 2b seen from above. In FIGS. 9 and 10, the front of the robot hand 2b faces left. FIG. 10 shows a holding state similar to FIGS. 6 and 8, and FIG. 9 shows an unholding state similar to FIGS. 5 and 7. Note that FIG. 9 shows a state in the middle of the transition from the preparation state Qa (described later) in FIG. 21 to the holding state Qb in FIG. 10. Furthermore, the position Pb of the robot hand 2b in FIG. 9 is the same as the position Pb of the robot hand 2b in FIG. 10, and is the position after movement from the position Pa (described later) of the robot hand 2b in FIG. 21.

[0044] As shown in FIG. 3 and other figures, the robot hand 2b includes a main body 50 and holders 10, 20, 30, and 40 for holding the container 7.

[0045] The main body 50 includes a first main body 51 extending in a first direction (e.g., the X direction in FIG. 3 ) and a second main body 52 extending in a second direction (e.g., the Y direction in FIG. 3 ). The second direction is perpendicular to the first direction. In a top view (when the robot hand 2b in a horizontal orientation is viewed from above), the first main body 51 has a substantially rectangular shape (elongated rectangle) extending in the first direction, and the second main body 52 has a substantially rectangular shape (elongated rectangle) extending in the second direction (see FIG. 10 , etc.). The second main body 52 is fixed to the first main body 51 near the center of the first main body 51 in the first direction. Among various orientations of the robot hand 2b, the horizontal orientation of the robot hand 2b is an orientation in which both the extension direction (first direction) of the first main body 51 and the extension direction (second direction) of the second main body 52 are horizontal with respect to the floor surface.

[0046] The main body 50 (and the robot hand 2b) in a horizontal position has a substantially T-shape when viewed from above. For convenience of illustration, the robot hand 2b may be illustrated in a simplified T-shape. The length of the robot hand 2b in the second direction is greater than the length of the robot hand 2b in the first direction. Therefore, the first direction is also referred to as the short-side direction of the robot hand 2b, and the second direction is also referred to as the long-side direction of the robot hand 2b.

[0047] 4 and other figures, the length of the robot hand 2b in the short side direction (the length of the first main body portion 51) is approximately the same as the length of the container 7 in the short side direction, and the length of the robot hand 2b in the long side direction is approximately the same as the length of the container 7 in the long side direction. The robot hand 2b holds the container 7 with the short side direction of the robot hand 2b aligned with the short side direction of the container 7 and the long side direction of the robot hand 2b aligned with the long side direction of the container 7. Specifically, the robot hand 2b holds the container 7 using the holders 10, 20, 30, and 40.

[0048] The holding unit 10 is provided at one end of the main body 50 (specifically, the first main body 51) in the first direction (for example, the X direction in FIG. 3), and holds one end of the container 7 (object) in the first direction (the short side direction of the container 7). The holding unit 20 is provided at the other end of the main body 50 (specifically, the first main body 51) in the first direction, and holds the other end of the container 7 in the first direction.

[0049] The holding unit 10 includes a vertical plate (also referred to as a vertical portion) 13 and a support portion 12 (see FIG. 8 , etc.). The vertical plate 13 is provided at one end (left side in FIG. 8 ) of the first main body portion 51 in the first direction (X direction in FIG. 3 ). The vertical plate 13 is a member extending (hanging) vertically downward (in a direction perpendicular to both the first direction and the second direction and downward) from one end of the first main body portion 51, which has a horizontal posture. The vertical plate 13 has a (substantially rectangular) thin plate shape and is fixed to one end of the first main body portion 51 in the first direction with the normal of its main surface parallel to the first direction (the main surface parallel to the second direction). A support portion 12 for supporting the container 7 is fixed to the lower end of the vertical plate 13. The support portion 12 has a substantially rectangular parallelepiped shape (a narrow prism shape extending in the second direction). The support portion 12 is provided as a protrusion that protrudes from the inner main surface of the vertical plate 13 toward the holding portion 20 (the other end side in the first direction (the right side in FIG. 8)). The protrusion length (length in the first direction) of the support portion 12 is approximately the same as the protrusion length of the protrusion portion 73 of the side surface portion 71 of the container 7, and the length of the support portion 12 in the second direction is approximately the same as the length of the vertical plate 13 in the second direction.

[0050] The holding unit 20 includes a vertical plate (also referred to as a vertical portion) 23 and a support portion 22. The vertical plate 23 is disposed on the other end side (right side in FIG. 8 ) of the first main body portion 51 in the first direction (X direction in FIG. 3 ). The vertical plate 23 is disposed perpendicular to the first main body portion 51 (in a direction perpendicular to both the first direction and the second direction). The vertical plate 23 has a (substantially rectangular) thin plate shape and is disposed on the other end side of the first main body portion 51 with the normal of its main surface parallel to the first direction. A support portion 22 for supporting the container 7 is fixed to the lower end of the vertical plate 23. The support portion 22 has a substantially rectangular parallelepiped shape (a narrow rectangular column shape extending in the second direction). The support portion 22 is disposed as a protrusion that protrudes from the inner main surface of the vertical plate 23 toward the holding unit 10 (the one end side in the first direction (left side in FIG. 8 )). The protruding length (length in the first direction) of the support portion 22 is approximately the same as the protruding length of the protruding portion 73 of the side portion 71 of the container 7, and the length of the support portion 22 in the second direction is approximately the same as the length of the vertical plate 23 in the second direction.

[0051] The vertical plate 23 is fixed to the moving body 24 (see FIG. 7). The vertical plate 23 and the moving body 24 can be moved in a first direction by a drive mechanism having an air cylinder 26 and a linear guide (not shown). More specifically, a base portion (main body portion) 26a of the air cylinder 26 is fixed to the first main body portion 51 via a fixing member 28, and the tip side of the extension / contraction portion 26b of the air cylinder 26 is fixed to the vertical plate 23 (or the moving body 24). The holding unit 20 (more specifically, the support portion 22, the vertical plate 23, and the moving body 24) moves in the first direction in response to the extension / contraction movement of the air cylinder 26 (movement of the extension / contraction portion 26b relative to the fixed portion 26a). Furthermore, the linear movement of the holding unit 20 in the first direction is guided by a linear guide provided on the moving body 24.

[0052] The holding unit 10 and the holding unit 20 are relatively movable in the first direction, allowing the width of the robot hand 2b in the first direction (for example, the distance between the vertical plate 13 of the holding unit 10 and the vertical plate 23 of the holding unit 20) to be changed (see FIGS. 7 and 8). Specifically, the holding unit 10 is fixed to the first main body unit 51, and the holding unit 20 is movable in the first direction relative to the first main body unit 51 in response to the actuation of the air cylinder 26. By changing the width of the robot hand 2b in the first direction, it is possible to switch between a state in which the holding units 10 and 20 hold two side surfaces 71 (specifically, their protrusions 73) of the container 7 from the outside (holding state) and a state in which the holding is released (non-holding state). The width of the robot hand 2b in the short-side direction (the distance between the vertical plate 13 and the vertical plate 23) is approximately the same as the short-side width of the container 7 in the holding state, and is larger than the short-side width of the container 7 in the non-holding state.

[0053] The holding unit 20 is movable between a first position P21 (see FIG. 8) and a second position P22 (see FIG. 7) relative to the holding unit 20. The first position P21 is a position where the container 7 is held on the outside of one side surface member (side surface portion 71) of the container 7 (a position where the support portion 22 supports the protruding portion 73 of the container 7). The second position P22 is a position farther away from the holding unit 10 than the first position P21 in the first direction (a position farther away from the container 7). The second position P22 is also a position where the holding unit 20 (particularly the vertical plate 23) and the holding unit 10 (particularly the vertical plate 13) avoid interference with the side surface portion 71 of the container 7 when the robot hand 2b (main body portion 50) moves up and down relative to the container 7.

[0054] Furthermore, the holding portion 10 and the holding portion 20 (particularly the support portion 12, the vertical plate 13, the support portion 22, and the vertical plate 23) each have a width W1 that is shorter than the length Wh of one side of the pinhole 6h in the second direction.

[0055] One of the holding units 10 and 20 holds the container 7 while inserted into the pinhole 6h (see Figures 19 and 20, etc.) in the pinhole stacking. Specifically, the one holding unit holds the object by contacting one end side of the container 7 in the short direction (the protrusion 73 (see Figure 8) provided on the outer surface of the container 7 on the pinhole 6h side). In other words, the one holding unit holds the object by contacting the protrusion 73 on the inner periphery of the pinhole stack within the pinhole 6h. The other holding unit holds the object by contacting the side opposite the pinhole 6h, specifically the other end side of the container 7 in the short direction (the protrusion 73 provided on the outer surface of the container 7 on the opposite side (from the pinhole 6h)). In other words, the other holding unit holds the object by contacting the protrusion 73 on the outer periphery of the pinhole stack.

[0056] The holding portion 30 is provided at one end of the main body portion 50 (more specifically, the second main body portion 52) in the second direction (for example, the Y direction in FIG. 3), and holds one end (the right side in FIG. 6) of the container 7 in the second direction (longitudinal direction). The holding portion 30 is provided on the opposite side of the holding portions 10 and 20 in the second direction. In other words, the holding portions 10 and 20 are provided at the other end (the left side in FIG. 6) of the main body portion 50 in the second direction.

[0057] The holding unit 30 includes a vertical plate (also referred to as a vertical portion) 33 and a support portion 32. The vertical plate 33 is provided on one end side (right side in FIG. 6 ) of the second main body portion 52 in the second direction (for example, the Y direction in FIG. 3 ). The vertical plate 33 is a member extending (hanging down) vertically downward (in a direction perpendicular to both the first direction and the second direction and downward) from one end of the second main body portion 52, which has a horizontal posture. The vertical plate 33 has a (substantially rectangular) thin plate shape and is fixed to one end of the second main body portion 52 in the second direction with the normal of its main surface parallel to the second direction (the main surface parallel to the first direction). A support portion 32 for supporting the container 7 is fixed to the lower end of the vertical plate 33. The support portion 32 has a substantially rectangular parallelepiped shape (a narrow prism shape extending in the first direction). The support portion 32 is provided as a protrusion that protrudes from the inner main surface of the vertical plate 33 toward the holding portion 40 (the other end side in the second direction). The protrusion length (length in the second direction) of the support portion 32 is approximately the same as the protrusion length of the protrusion portion 74 of the side surface portion 72 of the container 7, and the length of the support portion 32 in the first direction is approximately the same as the length of the vertical plate 33 in the first direction.

[0058] The holding portion 40 is provided near the holding portion 30 on one end side in the second direction (the same side as the holding portion 30) (the right side in FIG. 6).

[0059] The holding unit 40 includes a holding plate 43. The holding plate 43 has a thin plate shape and is disposed opposite the vertical plate 33 with the normal of its main surface parallel to the second direction. The holding plate 43 is disposed with a gap between it and the vertical plate 33. The holding plate 43 is a movable member that is movable relative to the holding unit 30 (and the second main body unit 52) ​​in the second direction. The holding plate 43 is a member that restricts movement of the side surface unit 72 in the second direction (relative movement between the protrusion 73 and the support unit 32) in order to maintain support of the side surface unit 72 by the holding unit 30 (support unit 32), and is also referred to as a restricting member (or stopper).

[0060] The holding plate 43 is fixed to the moving body 44. The holding plate 43 and the moving body 44 can be moved in the second direction by a drive mechanism having an air cylinder 46 and a linear guide (not shown). In detail, a base portion (main body portion) 46a of the air cylinder 46 is fixed to the second main body portion 52 via a fixing member 48, and the tip side of the expansion / contraction portion 46b of the air cylinder 46 is fixed to the holding plate 43 (or the moving body 44). The holding portion 40 (in detail, the holding plate 43, the moving body 44, etc.) moves in the second direction in response to the expansion / contraction movement of the air cylinder 46 (movement of the expansion / contraction portion 46b relative to the fixed portion 46a).

[0061] The holding portion 30 and the holding portion 40 are relatively movable in the second direction, and the distance between the vertical plate 33 of the holding portion 30 and the holding plate 43 of the holding portion 40 can be changed. In particular, the holding portion 30 is fixed to the second main body portion 52, and the holding portion 40 (holding plate 43) is movable in the second direction relative to the second main body portion 52 in response to the drive of the air cylinder 46. By changing the distance between the holding portion 30 and the holding portion 40 (the distance between the vertical plate 33 and the holding plate 43), it is possible to switch between a state in which movement of the side portion 72 in the second direction (the relative movement between the protrusion 73 and the support portion 32) is restricted and a state in which the restriction is released.

[0062] In particular, the holding plate 43 is movable between a first position P41 (with respect to the holding portion 40) (see FIG. 6) and a second position P42 (see FIG. 5).

[0063] The first position P41 (see FIG. 6 ) of the holding plate 43 is a position (see also FIG. 4 ) where the holding plate 43 faces the vertical plate 33 across one side member (side portion 72) of the container 7. The first position P41 also restricts movement of the one side member in the second direction to prevent the holding portion 30 from releasing its support for the container. A gap exists in the second direction between the holding plate 43 at the first position P41 and the support portion 32 of the holding portion 30. The first position P41 is set so that the gap is equal to or less than the length (thickness) of the side portion 72 (more specifically, the side panel 76, which is the non-protruding portion of the side portion 72) in the second direction plus a margin. The margin is a value smaller (close to zero) than the protrusion length of the protrusion 74 in the second direction. This makes it possible to prevent the protrusion 74 of the side portion 72 from falling off the support portion 32 (resulting in the release of support for the container 7).

[0064] The second position P42 (see FIG. 5) is a position on the side away from the holding part 30 with respect to the first position P41 in the second direction. The second position P42 is also a position where interference between the holding plate 43 and the side part 72 is avoided when the robot hand 2b (main body part 50) moves up and down relative to the container 7.

[0065] 4, 6, 8, and 10, the robot hand 2b holds the container 7 by supporting protrusions 73 (73a, 73b) provided on each of the two side surfaces 71 and a protrusion 74 (74c) provided on one side surface 72. The two protrusions 73 on the two side surfaces 71 can also be expressed as protrusions provided on one end and the other end of the container 7 in the short direction, and the two protrusions 74 on the two side surfaces 72 can also be expressed as protrusions provided on one end and the other end of the container 7 in the long direction.

[0066] Specifically, the holding unit 10 supports a protruding portion 73 (e.g., 73a) at one end in the lateral direction of the container 7 (see FIG. 8). More specifically, the support portion 12 protruding from the vertical plate 13 supports the protruding portion 73a of the container 7 from below. Furthermore, the holding unit 20 (when moved to position P21) supports the protruding portion 73 (e.g., 73b) at the other end in the lateral direction of the container 7 from below. More specifically, the support portion 22 protruding from the vertical plate 23 supports the protruding portion 73b of the container 7 from below. Furthermore, the holding unit 30 supports the protruding portion 74 (74c) at one end in the longitudinal direction of the container 7 from below (see also FIGS. 6 and 4). More specifically, the support portion 32 protruding from the vertical plate 33 supports the protruding portion 74c of the container 7 from below.

[0067] In this way, the robot hand 2b holds (supports) the container 7 at three locations (the protrusions 73a, 73b, and 74c) including the protrusion 73a and 73b. Therefore, the container 7 can be held more stably than when it is held at only two locations (for example, the protrusions 73a and 73b).

[0068] In particular, the robot hand 2b supports and holds the protruding portions 73a, 73b, and 74c of the container 7 from below. Compared to clamping the container 7 from both sides by applying a relatively large force (squeezing force) from the outside to the inside to both side portions 71 of the container 7, it is possible to stably hold the container 7, which has a relatively large weight, while avoiding deformation of the container 7.

[0069] Furthermore, with the holding unit 40 moved to position P41, the robot hand 2b positions one side surface 72 (particularly the side surface panel 76 and the protruding portion 74c) of the container 7 in the narrow gap between the vertical plate 33 and the holding plate 43 (see FIG. 6). More specifically, the side surface panel 76 is disposed in the narrow gap between the support unit 32 and the holding plate 43. This restricts movement of the one side surface 72 (72c (FIG. 6)) in the second direction, preventing the container 7 from shifting from the robot hand 2b in the second direction.

[0070] The distance sensor 3 is configured as a laser sensor (laser distance measuring sensor) that emits laser light, receives the light reflected from an object, and measures the distance to the object. The distance sensor 3 is fixed to the outer surface of the vertical plate 33. For example, when the robot hand 2b is in a horizontal state (when the robot hand 2b has a horizontal posture), the distance sensor 3 emits laser light vertically downward (see the dashed line in Figure 3) and measures the distance to an object vertically below.

[0071] The two-dimensional camera 4 is configured with an optical system (lens, etc.) and an imaging element (RGB image sensor, etc.). The two-dimensional camera 4 is capable of generating an image (more specifically, two-dimensional image data) within the field of view. The two-dimensional camera 4 is fixed to the lower part of the second main body part 52 (the lower part of the robot hand 2b in a horizontal position) near the center of the second main body part 52 in the second direction. The position and orientation of the two-dimensional camera 4 changes in response to changes in the position and orientation of the robot hand 2b. When the robot hand 2b is in a horizontal position, the two-dimensional camera 4 faces horizontally (the optical axis of the two-dimensional camera 4 is parallel to the horizontal direction).

[0072] The robot hand 2b further includes a lighting unit 56 and a docking unit 57.

[0073] The lighting unit 56 is provided above the two-dimensional camera 4 (when the robot hand 2b is in a horizontal position), and illuminates the field of view of the two-dimensional camera 4 and the like.

[0074] The docking unit 57 is a connection unit that connects (couples) the robot hand 2b to the robot arm 2a. Although the robot arm 2a is not shown in Figure 3 and subsequent figures, the robot hand 2b is actually connected (coupled) to the tip of the robot arm 2a using the docking unit 57.

[0075] <3. Depalletizing process> <Step S11> 11 is a flowchart showing the depalletizing process (depalletizing method). The process in FIG. 11 is executed by the controller 5 and the like.

[0076] In step S11, the controller 5 uses the distance sensor 3 to detect the height of the uppermost container 7 on the pallet 6 and the orientation of the pinhole stacking of the uppermost container 7 on the pallet 6.

[0077] Figure 12 is a diagram (top view) explaining the orientation of pinhole stacking. The upper row (Figure 12(a)) and lower row (Figure 12(b)) of Figure 12 have in common the formation of a pinhole 6h (hollow portion) in the center, but the planar arrangement of the multiple containers 7 on the pallet 6 (vertical / horizontal placement of each container 7) is different.

[0078] In the upper row of Figure 12 (Figure 12(a)), containers 7 having a vertically long state are placed at the upper left and lower right, and containers 7 having a horizontally long state are placed at the lower left and upper right. On the other hand, in the lower row of Figure 12 (Figure 12(b)), containers 7 having a horizontally long state are placed at the upper left and lower right, and containers 7 having a vertically long state are placed at the lower left and upper right. Note that, to distinguish between the two based on the longitudinal direction from the upper left starting point, the pinhole stacking direction in the upper row of Figure 12 is also referred to as the "counterclockwise" direction, and the pinhole stacking direction in the lower row of Figure 12 is also referred to as the "clockwise" direction.

[0079] Here, we assume a situation in which the pinhole stacking direction on each pallet 6 at either of the two pallets 6 can be either "counterclockwise" or "clockwise." In the robot system 1, although the pinhole stacking direction on each pallet 6 is unknown (unknown) at the start of depalletizing, the pinhole stacking direction is detected (identified) in step S11.

[0080] The controller 5 drives the robot 2 to move the robot hand 2b horizontally along a predetermined trajectory while maintaining the height of the distance sensor 3 (fixed to the robot hand 2b) at a predetermined value H0 and maintaining the orientation of the robot hand 2b horizontal. The predetermined value H0 corresponds to a value higher than the top surface of the container 7 of the highest level (here, the fourth level) of the maximum number of levels that can be loaded on the pallet 6. The predetermined trajectory is, for example, a square-shaped trajectory (a trajectory along a square that is slightly smaller than the pallet 6 and larger than the pinhole 6h) that passes over four containers 7 while crossing the boundary lines between adjacent containers 7, as shown by the thick arrow in FIG. 12 (when viewed from above). The predetermined trajectory is a trajectory that progresses continuously along the left side, bottom side, right side, and top side of the square in this order, and progresses across the boundary lines between two adjacent containers 7 on each side of the square (each line segment trajectory). The predetermined trajectory is a trajectory common to multiple pallets 6.

[0081] As the robot moves horizontally, distance sensor 3 (see FIG. 3, etc.) measures the distance to an object located directly below it.

[0082] FIG. 13 is a diagram showing the distance measurement results obtained by the distance sensor 3. In FIG. 13, the distance measurement results are shown at each position on a straight trajectory (segment trajectory) corresponding to one side of a square. In detail, the distance measurement results for a container 7 loaded with cargo (multiple small boxes) are shown by a solid line, and the distance measurement results for an empty container 7 are shown by a dashed line. The graph in the upper part of FIG. 13 corresponds to the pinhole stacking orientation in the upper part of FIG. 12, and the graph in the lower part of FIG. 13 corresponds to the pinhole stacking orientation in the lower part of FIG. 12.

[0083] In each graph in Fig. 13, the horizontal axis indicates the position on one line segment trajectory, and the vertical axis indicates the height H. The height H is the amount obtained by converting the distance z from the distance sensor 3 to the object (the value obtained by the distance sensor 3) into the height from the reference plane (here, the top surface of the pallet 6) (H = H0 - z).

[0084] As shown in FIG. 13, at a horizontal position Pk corresponding to the vicinity of the boundary between the side portions (side members) 71, 72 of the container 7, the height H has a (largely protruding) peak value H1.

[0085] The controller 5 determines the peak value H1 as the height (top surface height) of the top surface (top surfaces of the side portions 71, 72) of the container 7. When cargo (small boxes, etc.) is loaded inside the container 7, a certain height is detected at positions other than the horizontal position (peak position) Pk (see the solid line in FIG. 13). However, by utilizing the fact that cargo is loaded only in the space below the top surface of the container 7 to a certain extent, it is possible to identify the position Pk having the greatest height (peak value H1) as the horizontal position (position on the linear trajectory) of the side portions 71, 72.

[0086] Furthermore, the controller 5 detects the orientation of the pinhole stack. Specifically, when the horizontal position Pk appears in the latter half of the scanning range (one line segment trajectory) (FIG. 13(a)), the orientation of the pinhole stack is determined to be counterclockwise (FIG. 12(a)). Conversely, when the horizontal position Pk appears in the first half of the scanning range (one line segment trajectory) (FIG. 12(b)), the orientation of the pinhole stack is determined to be clockwise (FIG. 12(b)).

[0087] In this way, in step S11, the pinhole stacking orientation of the top four containers 7 among the multiple containers 7 loaded on the pallet 6 and the height of the top containers 7 are detected.

[0088] Furthermore, based on the orientation of the pinhole stack determined as described above, the controller 5 also determines whether each of the four containers 7 forming the pinhole stack has a vertical or horizontal orientation. For example, if it is determined that the orientation of the pinhole stack is counterclockwise (FIG. 12(a)), it is determined that the loading orientation of the two containers 7, the upper left and lower right, is vertical (vertical placement), and that the loading orientation of the two containers 7, the lower left and upper right, is horizontal (horizontal placement).

[0089] <Step S12> Next, in step S12, the controller 5 uses the two-dimensional camera 4 to detect the horizontal position of each of the uppermost containers 7.

[0090] First, the controller 5 drives the robot 2 to change the posture of the robot hand 2b as shown in FIG. 14. In detail, the controller 5 drives a certain rotary joint of the robot arm 2a (for example, the second (or third) rotary joint from the hand tip) to change the direction of the optical axis of the two-dimensional camera 4 by 90 degrees from horizontal to vertical. In this state (a state in which the two-dimensional camera 4 faces vertically downward), the controller 5 causes the two-dimensional camera 4 to photograph the container 7. Note that FIG. 14 is a diagram (side view) showing the posture change of the two-dimensional camera 4. FIG. 14 also shows the positions of the upper left and upper right containers 7 when photographing a specific corner (described below) of the upper left container 7 in FIG. 12(a).

[0091] Furthermore, the controller 5 drives the robot 2 so that the field of view 211 of the two-dimensional camera 4 falls within the following range (see FIG. 15), thereby changing the position (horizontal position, etc.) of the robot hand 2b. FIG. 15 is a diagram showing the field of view (photography range) 211 of the two-dimensional camera 4 suitable for detecting corners of the container 7.

[0092] Specifically, the robot 2 is driven so that the field of view 211 of the two-dimensional camera 4 is a range including a specific corner of the ith container 7 (starting with i = 1) among the four topmost containers 7. More specifically, the robot 2 is driven so that (the optical axis of) the two-dimensional camera 4 is located near the specific corner of the ith container 7. This specific corner is the corner near the pinhole 6h among the four corners of the container 7. The position (approximate position) of this specific corner is estimated based on the orientation of the pinhole stack (and / or the peak position Pk) determined in step S11. For example, the top left container 7 (see FIG. 15(a)) of the counterclockwise pinhole stack is determined to be arranged vertically, and the position (approximate position) of the corner near the pinhole 6h (the lower right corner of the container 7) is estimated. FIG. 15 shows the field of view 211 for the top left container 7. The field of view range 211 in the upper part of FIG. 15 corresponds to the orientation of the pinhole stack in the upper part of FIG. 12, and the field of view range 211 in the lower part of FIG. 15 corresponds to the orientation of the pinhole stack in the lower part of FIG.

[0093] Furthermore, the two-dimensional camera 4 is moved vertically upward to a position a predetermined distance away from the top surface of the uppermost container 7 (the container 7 that is currently at the top of the pallet 6). The focus of the two-dimensional camera 4 is manually adjusted in advance in accordance with this predetermined distance. By disposing the two-dimensional camera 4 at a position that is this predetermined distance away from the top surface of the container 7, it is possible to obtain a good focus state (without the need for manual focus readjustment (or an autofocus mechanism)).

[0094] Then, with the two-dimensional camera 4 in the position and orientation as driven by the robot 2, the two-dimensional camera 4 takes an image and acquires a captured image of the field of view 211. The controller 5 performs image processing and the like on the captured image to calculate (measure) the exact position of the specific corner of the i-th container 7. In detail, the position of the specific corner is accurately calculated by extracting the outline (edge) of the specific corner (shown by the thick line in FIG. 15).

[0095] Similarly, the positions of the specific corners of the second to fourth containers 7 (i=2, 3, 4) are also determined.

[0096] In the above, photographing is performed in a state where a certain rotary joint of the robot arm 2a (for example, the second rotary joint from the hand) is driven to change the optical axis direction of the 2D camera 4 by 90 degrees from horizontal to vertical. In particular, with such rotational drive, the position of the 2D camera 4 is moved upward, making it possible to photograph from a relatively high position. Therefore, even when photographing from a position close to the upper limit of the movable range of the tip of the robot arm 2a (such as when photographing the fourth container 7 from the bottom), the 2D camera 4 can be spaced a certain distance from the top surface of the container 7. In other words, it is possible to ensure a good field of view for the camera.

[0097] Furthermore, even when the number of containers 7 stacked on the pallet 6 decreases as depalletizing progresses, the robot 2 is driven and the two-dimensional camera 4 is positioned at a predetermined distance vertically upward from the top surface of the uppermost container 7 (topmost container 7) on the pallet 6 (at that point in time). This makes it possible to always obtain a good focus state.

[0098] The controller 5 identifies the position (spatial position) of each container 7 in space (three-dimensional space) based on the information obtained by the processing of steps S11 and S12. The position of the container 7 is expressed, for example, by the center position of the container 7 in a horizontal plane (the horizontal position of the center of the container 7) and the top surface position (height) of the container 7. However, without being limited to this, the position of the container 7 may also be expressed using the horizontal position of a specific corner, etc. In the depalletizing operation of step S15 (described below), the controller 5 drives the robot 2 to pick up the target container 7 based on the position in space.

[0099] <Step S13> In the next step S13, the controller 5 identifies which of four horizontal orientations D1 to D4 (see FIG. 16) each container 7 has, based on the image captured by the two-dimensional camera 4. The four orientations D1 to D4 are orientations that are shifted by 90 degrees, for example, leftward D1, forward D2, rightward D3, and backward D4. The four orientations can also be expressed as leftward D1, downward D2, rightward D3, and upward D4 in a top view.

[0100] As shown in Fig. 16, the container 7 has a generally rectangular shape when viewed from above, and has a colored portion (coloring portion) 78 colored in a specific color (for example, red) on only one of the four sides (two long sides and two short sides) of the rectangular shape. Here, the colored portion 78 is provided on only one of the two short sides of the four sides. More specifically, the colored portion 78 is provided on the upper surface and outer side surface of the side portion 72 (corresponding to the short sides) (see also Fig. 18).

[0101] The controller 5 drives the robot 2 to change the position and orientation of the robot hand 2b so that the two-dimensional camera 4 can capture a field of view 213 (see FIG. 16) that includes only one of the two short sides (side surface portions 72) of the container 7 (for example, only the upper side surface portion 72 of the upper left container 7 in FIG. 16). The position of each short side can be obtained based on the orientation (direction of each container 7) acquired in step S11. FIG. 16 is a diagram showing the field of view (capture range) 213 of the two-dimensional camera 4 that is suitable for detecting the colored portion 78.

[0102] Then, the controller 5 causes the two-dimensional camera 4 to take an image and acquires a captured image of the field of view 213.

[0103] Furthermore, the controller 5 performs image processing on the captured image to determine whether or not the captured image includes a colored portion 78.

[0104] If the captured image includes a colored portion 78, the controller 5 determines that the colored portion 78 is present on one of the short sides included in the field of view (capture range) 213 (for example, the upper short side of the upper left container 7 in the upper row of FIG. 16). The controller 5 then determines the orientation of the container 7. For example, if it is determined that the colored portion 78 is included in the field of view 213 including the upper short side of a vertically placed container 7, it is determined that the orientation of the container 7 is "upward" (D4). If it is determined that the colored portion 78 is included in the field of view 213 including the left short side of a horizontally placed container 7, it is determined that the orientation of the container 7 is "leftward" (D1).

[0105] Conversely, if the captured image does not include the colored portion 78, the controller 5 determines that the colored portion 78 is present on the other short side that is not included in the field of view 213 (for example, the lower short side of the upper left container 7 in the upper row of Figure 16). Then, the controller 5 determines the orientation of the container 7. For example, if it is determined that the colored portion 78 is not included in the field of view 213 that includes the upper short side of a vertically placed container 7, it is determined that the orientation of the container 7 is "downward" (D2). Furthermore, if it is determined that the colored portion 78 is not included in the field of view 213 that includes the left short side of a horizontally placed container 7, it is determined that the orientation of the container 7 is "rightward" (D3).

[0106] This process is performed for each of the four uppermost containers 7, and it is determined which of the four orientations D1 to D4 each container 7 is facing. The orientation of the container 7 determined in step S13 is used in the transfer operation in step S15.

[0107] <Step S14> In the next step S14, the controller 5 checks the position and posture of the destination mobile robot 8. Specifically, the controller 5 drives the robot 2 to move the robot hand 2b (particularly the two-dimensional camera 4) above the mobile robot 8.

[0108] As shown in Fig. 17, the mobile robot 8 is provided with landmarks 81 for position confirmation. The two-dimensional camera 4 moves above the mobile robot 8 and captures an image of the landmarks 81 provided on the top surface of the mobile robot 8. The controller 5 calculates the relative position and orientation of the robot 2 and the mobile robot 8 based on the landmarks 81 in the captured image. In this way, the controller 5 recognizes the position and orientation of the mobile robot 8 (its position and orientation relative to the robot 2). The position and orientation of the mobile robot 8 are used in step S15.

[0109] <Step S15> In step S15, the controller 5 executes a process to transfer the top four containers 7 from the pallet 6 to the mobile robot 8 based on the information obtained in steps S11 to S14. Note that FIG. 18 is a perspective view showing the state in which the four containers 7 (7a to 7d) have been transferred to the mobile robot 8.

[0110] Specifically, the controller 5 first transfers the first container 7 (for example, the upper left container 7 (7a) in FIG. 12(a)) of the four containers 7 (7a to 7d) on the top row to the mobile robot 8, and then transfers the second container 7 to the mobile robot 8. The second container 7 is, for example, the container 7 diagonally opposite the first container 7 (the lower right container 7 (7b)). The first container 7 is placed on one of two rows of roller conveyors 83, 84 (see FIG. 17) on the mobile robot 8 (for example, the right row roller conveyor 84), and the second container 7 is loaded on the first container 7 in the same row (the same row). The two rows of roller conveyors 83, 84 on the mobile robot 8 on the left and right are used for further transferring the container 7 transferred to the mobile robot 8 to another device, etc.

[0111] When transferring the first and second containers 7, the robot 2 first picks up the container 7 while inserting one of the holding units 10 and 20 of the robot hand 2b (for example, the holding unit 10) into the pinhole 6h (see FIG. 19). The holding units 10 and 20 hold one and the other of the two side surfaces 71 in the short direction of the container 7, respectively, and the holding unit 30 holds one side surface 72 in the long direction of the container 7 (the outer periphery of the pinhole stack) (see also FIG. 20). When picking up the first and second containers 7, the holding unit 30 may hold the side surface 72 on the colored portion 78 side of the two side surfaces 72, or may hold the side surface 72 opposite the colored portion 78 (the side surface 72 without the colored portion 78).

[0112] FIG. 20 is a top view showing how a container 7 on a pallet 6 is picked up by a robot hand 2b. The upper part of FIG. 20 corresponds to the pinhole stacking orientation in the upper part of FIG. 12, and the lower part of FIG. 20 corresponds to the pinhole stacking orientation in the lower part of FIG. 12. In the upper part of FIG. 20, the position, posture, etc. of the robot hand 2b holding the upper left container 7 stacked with pinholes counterclockwise is shown by a solid line, and the positions, postures, etc. of the robot hands 2b holding each of the remaining three containers 7 are shown by two-dot chain lines. In addition, the lower part of FIG. 20 similarly shows the positions, postures, etc. of the robot hands 2b holding each of the containers 7 stacked with pinholes in the opposite direction (clockwise).

[0113] 20, in the counterclockwise pinhole stacking (upper row), the holding unit 10 is disposed on the pinhole 6h side, and the holding unit 20 is disposed on the opposite side of the pinhole 6h. Specifically, the holding unit 10 is inserted into the pinhole 6h and contacts the container 7 on the pinhole 6h side to hold the container 7, while the holding unit 20 contacts the container 7 on the opposite side of the pinhole 6h (the outer periphery of the pallet) to hold the container 7.

[0114] On the other hand, for pinhole stacking in the reverse direction (clockwise) (lower row), the holding unit 20 is arranged on the pinhole 6h side, and the holding unit 10 is arranged on the opposite side of the pinhole 6h. Specifically, the holding unit 20 is inserted into the pinhole 6h and contacts the container 7 on the pinhole 6h side to hold the container 7, while the holding unit 10 contacts the container 7 on the opposite side of the pinhole 6h (the outer periphery of the pallet) to hold the container 7.

[0115] In this way, the robot hand 2b can appropriately handle pinhole stacking in both directions.

[0116] The robot 2 then transfers the container 7 picked up (held) by the robot hand 2b onto the mobile robot 8. Specifically, the robot 2 places the container 7 on the mobile robot 8 with the orientation of the container 7 aligned. For example, the first and second containers 7 are transferred with the containers 7 aligned "downward" in a top view (with the colored portion 78 facing forward (see FIG. 18 )). Specifically, the orientation of the containers 7 can be aligned by rotating the robot hand 2b about the vertical axis by, for example, driving a certain rotary joint of the robot arm 2a (for example, the rotary joint closest to the hand tip). The angle (posture) of the robot hand 2b about the vertical axis at the time of placement on the mobile robot 8 differs by 180 degrees between when the holding unit 30 of the robot hand 2b holds the side surface portion 72 on the colored portion 78 side and when the holding unit 30 holds the side surface portion 72 opposite the colored portion 78.

[0117] Thereafter, the controller 5 transfers the two containers 7 (third and fourth containers 7c, 7d) (see FIG. 22) remaining on the topmost level (at that time) of the pallet 6 to the mobile robot 8. The third and fourth containers 7 are also placed on the mobile robot 8 with their orientations aligned "downward" (facing forward) (see FIG. 18). The third container 7c is placed on the other row (for example, the left roller conveyor 83) of the two left and right rows of roller conveyors provided on the mobile robot 8, and the fourth container 7d is loaded on the third container 7c in the other row (the same row).

[0118] When the remaining third and fourth containers 7 are transferred, there are no other containers 7 on the pallet 6 that are in contact with (or very close to) the third and fourth containers 7 (7c, 7d) (see FIG. 22). Therefore, interference with the other containers 7 can be avoided without the need to place the holder 10 or the holder 20 at the position of the pinhole 6h.

[0119] On the other hand, on the mobile robot 8, there is a possibility that the first and second containers 7 that have already been transferred onto the mobile robot 8 may interfere with the robot hand 2b that is transferring the remaining third (and fourth) containers 7. It is preferable to appropriately avoid such interference.

[0120] Therefore, for example, when the long sides of the containers 7 are arranged in the depth direction of the mobile robot 8, the left-hand row of containers 7 and the right-hand row of containers 7 are placed at horizontal positions offset from each other in the depth direction of the mobile robot 8 (the long sides of the containers 7) (see FIG. 18). For example, the two-tiered containers 7 in the left-hand row are placed a certain distance L1 (= width of holding unit 20 (or holding unit 10) + margin width) further back than the two-tiered containers 7 in the right-hand row.

[0121] Furthermore, when the robot hand 2b picks up the container 7 (7c, 7d) from the pallet 6, it takes into consideration in advance the orientation of the container 7 when it is loaded onto the mobile robot 8. Specifically, the robot hand 2b picks up the container 7 (7c, 7d) in a state where the relationship between the orientation of the container 7 when it is loaded onto the mobile robot 8 and the posture of the robot hand 2b is already established (see FIG. 22). FIG. 22 shows a state where, of the four containers 7 on the top level, the first and second containers 7a, 7b have already been transferred to the mobile robot 8, and the third and fourth containers 7c, 7d remain on the pallet 6.

[0122] For example, as shown in FIG. 22, the robot hand 2b picks up the third container 7c on the pallet 6 with the holding unit 30 holding the side surface 72 on the coloring unit 78 side (and the holding units 10 and 20 each holding the side surface 71). Then, the robot 2 that picked up the third container 7c moves above the mobile robot 8 (more specifically, the left-hand roller conveyor 83 (see FIG. 18)). With the holding unit 20 of the robot hand 2b positioned in the space behind the first and second containers 7 (7a, 7b), the robot 2 lowers (moves vertically downward) the third container 7c from above and places it on the left-hand roller conveyor 83. As a result, the container 7c is placed on the left-hand roller conveyor 83 (with the coloring unit 78 located in the front) without interfering with the containers 7a and 7b that have already been transferred. Next, in the same manner, the fourth container 7d is transferred from the pallet 6 to the mobile robot 8 (see FIG. 18). However, the fourth container 7d is placed on top of the third container 7c that has already been transferred onto the left-hand roller conveyor 83.

[0123] In this way, the third and fourth containers 7 (7c, 7d) are transferred to the mobile robot 8 while avoiding interference (on the mobile robot 8) between the robot hand 2b and the loaded containers 7 (7a, 7b).

[0124] In this manner, the four containers 7 on the pallet 6 are transferred to the mobile robot 8.

[0125] <Details of step S15> The pick-up process of the first and second containers 7 will now be described in more detail.

[0126] In step S15, the robot hand 2b first moves from the ready state Qa to the ready position Pa. Next, the robot hand 2b descends vertically (downward). Then, the robot hand 2b transitions from the ready state Qa to the holding state Qb (see FIG. 4, etc.) while moving to the final position Pb.

[0127] Here, the final position Pb is the position where the robot hand 2b holds the container 7, and the holding state Qb is the state where the robot hand 2b holds the container 7 (see FIGS. 6, 8, and 10). On the other hand, the preparation position Pa and the preparation state Qa (see FIG. 21) are the positions and states of the preparation stage before the robot hand 2b holds the container 7. The preparation position Pa (particularly its horizontal position) and the preparation state Qa are also the positions and states for avoiding interference between the robot hand 2b and the container 7 when the robot hand 2b descends.

[0128] The vertical position (Z-direction position) of the preparation position Pa is a position that is a predetermined distance above the Z-direction position of the final position Pb (i.e., a position that is a certain degree above the uppermost container 7). Moreover, the horizontal position (position in the X-direction and Y-direction) of the preparation position Pa (also referred to as the horizontal position) is a position that is offset (displaced) from the horizontal position of the final position Pb.

[0129] FIG. 21 shows a situation in which the robot hand 2b is in the preparation position Pa (see particularly the horizontal position) when picking up a container 7 (such as the container 7 at the bottom right) that has been pinhole stacked counterclockwise (see FIG. 12(a)). FIG. 21 also shows the pinhole 6h (see). FIG. 21 also shows the preparation state Qa of the robot hand 2b.

[0130] As shown in FIG. 21, the horizontal position of the preparation position Pa is offset by a value α toward the pinhole 6h in the short-side direction (of the robot hand 2b and the container 7 to be picked up) and by a value β toward the holding unit 30 in the long-side direction relative to the horizontal position of the final position Pb (see FIGS. 9 and 10). The horizontal position of the preparation position Pa is also the same when picking up a container 7 stacked through a pinhole in a clockwise direction (see FIG. 12(b)). However, the horizontal position of the preparation position Pa in the first direction may be offset by a value α toward the opposite side of the pinhole 6h in the short-side direction relative to the horizontal position of the final position Pb.

[0131] Furthermore, the ready state Qa is a state in which the vertical plate 23 of the holder 20 is located at position P22 (FIG. 7) and the holding plate 43 of the holder 40 is located at position P42 (FIG. 5). On the other hand, the holding state Qb is a state in which the vertical plate 23 of the holder 20 is located at position P21 (FIG. 8) and the holding plate 43 of the holder 40 is located at position P41 (FIG. 6). In the ready state Qa, the distance between the vertical plate 13 and the vertical plate 23 is larger, and the distance between the vertical plate 33 and the holding plate 43 is larger than in the holding state Qb. In both the ready state Qa and the holding state Qb, the robot hand 2b has a horizontal posture (see FIGS. 2 and 4, etc.).

[0132] More specifically, in step S15, the robot hand 2b first moves from above the target container 7 in the preparation state Qa to the preparation position Pa while approaching the container 7 in response to the driving of the robot 2 (see the upper part of Figure 21).

[0133] Next, in response to the driving of the robot 2, the robot hand 2b moves vertically (downward) while maintaining the horizontal position of the preparation position Pa, and descends to the Z-direction position of the final position Pb. During the descent, the robot hand 2b is in the horizontal position of the preparation position Pa and in the preparation state Qa. Therefore, none of the holding unit 10, the holding unit 20, the holding unit 30, and the holding unit 40 interfere with the container 7. In particular, one of the holding unit 10 and the holding unit 20 (holding unit 10 in FIG. 21) is inserted into the pinhole 6h, so that it does not interfere with other adjacent containers 7.

[0134] Then, the robot hand 2b moves to the horizontal position of the final position Pb (so as to cancel the offset amount α in the short side direction and the offset amount β in the long side direction) in response to the driving of the robot 2 (see FIGS. 5, 7, and 9). As a result, the robot hand 2b moves to the final position Pb in a state of being in the preparation state Qa (see especially FIG. 9).

[0135] Furthermore, the vertical plate 23 and the holding plate 43 move (to positions P21 and P41, respectively) in response to the driving of the holding units 20 and 40 (see FIGS. 6, 8, and 10). FIGS. 6, 8, and 10 also show the final position Pb and the holding state Qb. In this way, movement from the preparation position Pa to the final position Pb and transition from the preparation state Qa to the holding state Qb are performed.

[0136] Here, after the robot hand 2b descends, the movement to the horizontal position of the final position Pb and the transition to the holding state Qb are performed sequentially, but this is not limiting. For example, (after the robot hand 2b descends) the movement to the horizontal position of the final position Pb and the transition to the holding state Qb may be performed simultaneously in parallel.

[0137] Next, the robot hand 2b rises in response to the driving of the robot 2. At this time, the support portion 12 supports the protruding portion 73 on one side from below, the support portion 22 supports the protruding portion 73 on the other side from below, and the support portion 32 supports the protruding portion 74 from below, so that the container 7 is securely held by the robot hand 2b.

[0138] In this manner, the pickup process of the first and second containers 7 is performed. Thereafter, the container 7 is transferred onto the mobile robot 8 (in response to the driving of the robot 2) while being held by the robot hand 2b.

[0139] The pick-up process for the third and fourth containers 7 is performed in the same manner. As described above, since there are no other containers 7 adjacent to the third and fourth containers 7, it is not necessary to place the holding unit 10 or the holding unit 20 at the position of the pinhole 6h. However, the same process may be performed as the pick-up process for the containers 7.

[0140] Furthermore, when each container 7 (7a to 7d) is lowered onto the mobile robot 8, the following process may be performed. Specifically, when the robot hand 2b is moved to a spatial position on the mobile robot 8 that is a predetermined horizontal position (for example, a position directly above the roller conveyor 84) and a certain height in response to the drive of the robot 2, the robot hand 2b begins to descend. Then, after the container 7 is placed (seated) on the mobile robot 8, the robot hand 2b further descends a small distance and releases its support (holding) of the container 7. Thereafter, the robot hand 2b transitions to a non-holding state in response to the drive of the holder 20 (air cylinder 26) and the holder 40 (air cylinder 46). Then, the robot hand 2b moves to a horizontal position (retracted position) where it can rise while avoiding interference with the container 7. Specifically, it moves by an offset amount α in the short side direction and an offset amount β in the long side direction. Then, the robot hand 2b rises in response to the drive of the robot 2 and returns to the pallet 6 to transfer the next container 7.

[0141] <Step S16> In step S16 (FIG. 11), the controller 5 determines whether or not the transfer of all containers 7 on the pallet 6 has been completed. If it is determined that the transfer of all containers 7 has been completed, it is determined that the transfer work from the pallet 6 has been completed, and the processing in FIG. 11 ends. On the other hand, if it is determined that the work is not yet completed, the process returns to step S11, and the same processing is performed for each row of the remaining containers 7.

[0142] Furthermore, after the depalletizing process for one pallet 6 is completed, the depalletizing process for another pallet 6 is executed in the same manner. In this robot system 1, it is possible to place pallets 6 in a total of two locations, on the left and right sides of the robot 2. For example, when the depalletizing process for the left pallet 6 is completed, the depalletizing process for the right pallet 6 is started next. Thereafter, the depalletizing process for the left pallet 6 and the depalletizing process for the right pallet 6 are executed alternately in the same manner.

[0143] <4. Other sensors on robot hand 2b> The robot hand 2b further includes transmission type beam sensors 15 and 35, reflection type beam sensors 16 and 36, and a transmission type beam sensor 27 (see FIGS. 5 to 10). For the sake of simplicity, these sensors are not shown in FIGS. 3 and 4.

[0144] The through-beam sensors 15 and 35 are approach detection sensors (sensors that detect approach to the container 7 or the like).

[0145] The transmission type beam sensor 15 (see FIG. 7, etc.) includes a light-projecting unit 15a and a light-receiving unit 15b. The light-receiving unit 15b is provided directly below the support unit 12, and the light-projecting unit 15a is provided above the light-receiving unit 15b and in the vicinity of the first main body unit 51. The light-receiving unit 15b is fixed to the vertical plate 13 (or the support unit 12) via a fixture (not shown).

[0146] When the robot hand 2b is away from the container 7 (particularly the side surface 71), the light beam emitted from the light-projecting unit 15a passes through a through-hole that penetrates the support unit 12 in the vertical direction and is received by the light-receiving unit 15b. On the other hand, when the holding unit 10 (vertical plate 13, etc.) of the robot hand 2b approaches the container 7 (particularly the side surface 71) to a distance less than a predetermined distance, the light beam is blocked by the container 7 (side surface 71). Therefore, when the light beam changes from a light-receiving state to a blocked state, it is detected that the holding unit 10 (vertical plate 13, etc.) of the robot hand 2b has approached the container 7 to a predetermined extent. By completing the approaching operation of the robot hand 2b in the first direction when the robot hand 2b has advanced a further small distance (for example, a few millimeters), it is possible to very accurately position the support unit 12 directly below the protrusion 73.

[0147] The transmission type beam sensor 35 (see FIG. 5, etc.) includes a light-projecting unit 35a and a light-receiving unit 35b. The light-receiving unit 35b is provided directly below the support unit 32, and the light-projecting unit 35a is provided above the light-receiving unit 35b and in the vicinity of the second main body unit 52. The light-receiving unit 35b is fixed to the vertical plate 33 (or the support unit 32) via a fixture (not shown).

[0148] Using the same principle as the through-beam sensor 15, the through-beam sensor 35 detects that the holding part 30 (vertical plate 33, etc.) of the robot hand 2b has approached the container 7 to a predetermined extent in response to a change from a light-receiving state to a light-blocking state. By completing the approaching operation of the robot hand 2b in the second direction when the robot hand 2b has advanced a small distance further, it is possible to position the support part 32 very accurately in a position directly below the protrusion 74.

[0149] For example, in step S15, these through-beam sensors 15, 35 are used when approaching each container 7. As described above, a planar position can be determined with a certain degree of accuracy by measurement processing based on images captured by the two-dimensional camera 4, and the robot hand 2b can be positioned with a certain degree of accuracy relative to the container 7. However, by also using these through-beam sensors 15, 35, it is possible to perform even more accurate positioning (to bring the robot hand 2b closer to the container 7 even more accurately) to an extent that exceeds the position measurement accuracy achieved by image processing.

[0150] The reflective beam sensors 16, 36 are seating detection sensors (sensors that detect the seating of the container 7 when the container 7 held by the robot hand 2b is placed on a placement location (for example, a predetermined position on the mobile robot 8)).

[0151] The reflective beam sensor 16 is positioned so as to be able to emit and receive light at a position a certain distance (a small distance) higher than the top surface of the container 7 when the robot hand 2b is in the holding state Qb (see FIG. 7, etc.). The reflective beam sensor 16 is positioned on the outside of the vertical plate 13 of the holding unit 10, and the light beam emitted from the reflective beam sensor 16 passes through a through-hole provided in the vertical plate 13 in a first direction and heads toward the inside of the vertical plate 13 (toward the center of the robot hand 2b). While the robot hand 2b is transferring the container 7 (particularly while descending from a position directly above the placement location), the side surface 71 of the container 7 is not located opposite the reflective beam sensor 16 until just before the container 7 is seated, and the reflective beam sensor 16 does not receive the reflected light of the light beam. On the other hand, after the container 7 is actually seated, when the robot hand 2b further descends vertically to a certain extent, the side surface 71 of the container 7 is located opposite the reflective beam sensor 16. Therefore, the reflective beam sensor 16 receives the reflected light of the light ray (light reflected from the side surface portion 71). Therefore, at the time when the light ray changes from a non-light receiving state to a light receiving state, it is detected that the container 7 has been securely seated (more specifically, seated near the side surface portion 71).

[0152] At this point, or after descending a further small distance (for example, a few mm) and reliably releasing the container 7 from its supporting state, the robot hand 2b stops descending. Then, the state of the robot hand 2b transitions to a non-holding state in response to the driving of the holding units 20 and 40. After that, the robot hand 2b moves to a retracted position (for example, a horizontal position spaced a distance α in the lateral direction and a distance β in the longitudinal direction from the container 7), and then the robot hand 2b rises. This allows the robot hand 2b to leave the container 7 after reliably seating the container 7 (and preventing it from descending too far).

[0153] Similarly, the reflective beam sensor 36 can reliably detect the seating of the container 7 (specifically, seating near the side surface 72). The reflective beam sensor 36 has a configuration similar to that of the reflective beam sensor 16. However, the reflective beam sensor 36 is disposed on the outside of the vertical plate 33 of the holding unit 30, and the light beam emitted from the reflective beam sensor 36 passes through a through-hole provided in the vertical plate 33 in the second direction and heads toward the inside of the vertical plate 33 (toward the center of the robot hand 2b). In particular, when the container 7 is actually seated and then further descends vertically to a certain extent, the reflective beam sensor 36 receives reflected light of the light beam (light reflected from the side surface 72). When the light beam changes from a non-light-receiving state to a light-receiving state, it is detected that the container 7 has been reliably seated.

[0154] The through-beam sensor 27 is an interference prevention sensor for fail-safe purposes. The through-beam sensor 27 (see FIGS. 7 and 9, etc.) includes a light-projecting unit 27a and a light-receiving unit 27b. One of the light-projecting unit 27a and the light-receiving unit 27b (for example, the light-projecting unit 27a) is provided outside the vertical plate 13, and the other is provided outside the vertical plate 23. The light-projecting unit 27a and the light-receiving unit 27b are positioned so that they can project and receive light at a position higher than the upper surface of the container 7 by a certain amount or more when the robot hand 2b is in the holding state Qb (see FIG. 7, etc.). During normal operation of the robot hand 2b, a light beam emitted from the light-projecting unit 27a along the first direction (at a position higher than the detection position of the reflective beam sensor 16 by a certain amount) passes through through-holes provided in the vertical plate 13 and the vertical plate 23, respectively, and is received by the light-receiving unit 27b. On the other hand, if the light beam is blocked, it is determined that the robot hand 2b has come too close to the container 7 (an abnormality has occurred), and the movement of the robot hand 2b is stopped (emergency stop). This makes it possible to prevent (avoid) interference between the robot hand 2b and the container 7 even when an abnormality has occurred.

[0155] <5. Effects of the embodiment> In the above embodiment, the controller 5 detects the height of the top surface of the container 7 using the distance sensor (laser sensor) 3, and detects the horizontal position of the container 7 using the 2D camera 4. Then, the controller 5 identifies (detects) the position of the container 7 in space based on the detected height of the top surface and horizontal position. Therefore, it is possible to identify the position of the container 7 in space (in three-dimensional space) without using a three-dimensional camera. Therefore, it is possible to reduce costs (compared to using a three-dimensional camera). In other words, it is possible to build a system more inexpensively than with technology that uses a three-dimensional camera.

[0156] Furthermore, the position measurement devices (the two-dimensional camera 4 and the distance sensor 3) are provided on the robot 2 (specifically, the robot hand 2b). Therefore, it is possible to perform measurement processing on both of the two pallets 6 near the robot 2 using a single two-dimensional camera 4 (and a single distance sensor 3). In other words, it is not necessary to install a measurement device at each of the two locations. Therefore, it is possible to reduce costs compared to when a measurement device is provided for each of the two pallets 6. In particular, it is possible to significantly reduce costs compared to when a three-dimensional camera is provided as a measurement device for each of the two pallets 6.

[0157] Furthermore, the horizontal position of the distance sensor 3 is changed in response to the driving of the robot 2, and the height of the top surface of the container 7 is detected at a plurality of different horizontal positions. Then, based on the height of the top surface detected at a plurality of different horizontal positions, the loading direction of a plurality of objects on the topmost level is determined. More specifically, the loading direction (vertical direction / horizontal direction) of each pinhole-stacked container 7 is detected. Therefore, it is possible to perform appropriate processing (such as adjusting the posture (direction, etc.) of the robot hand 2b in response to the direction of the container 7) that is detected as the loading direction (loading direction of the container 7).

[0158] Furthermore, based on the image captured by the two-dimensional camera 4, one of the four sides of the container 7, which has a substantially rectangular shape when viewed from above, on which the colored portion 78 is arranged is identified, and it is determined which of the four horizontal orientations D1 to D4 the container 7 has. Therefore, it is possible to easily determine the orientation of the container 7 (the four orientations D1 to D4). Furthermore, based on the obtained orientations D1 to D4, it is possible to appropriately perform subsequent work (such as transferring the container to the mobile robot 8).

[0159] Furthermore, the two-dimensional camera 4 can capture images of the containers 7 from a position a predetermined distance vertically away from the top surface of the topmost container 7 in response to the drive of the robot 2. Therefore, it is possible to obtain images in an appropriately focused state without the need for focus readjustment (or an autofocus mechanism). In particular, even if the number of containers 7 stacked on the pallet 6 decreases as depalletizing progresses, the same effect can be obtained by the same process.

[0160] Furthermore, the robot hand 2b is rotated around a vertical axis in response to the driving of one of the rotary joints of the robot arm 2a (for example, the rotary joint at the distal end of the hand). Therefore, the robot hand 2b can be easily aligned with the orientation of the container 7.

[0161] Furthermore, in response to the driving of one of the rotary joints of the robot arm 2a (for example, the second (or third, etc.) rotary joint from the hand tip), the optical axis direction of the two-dimensional camera 4 is changed by 90 degrees from horizontal to vertical, and the two-dimensional camera 4 is moved to a higher position. In this state, the two-dimensional camera 4 captures an image of the container 7. This allows the two-dimensional camera 4 to be positioned at a certain distance (above) from the top surface of the container 7, even when the robot hand 2b is located in an area close to the upper limit of the movable range of the robot arm 2a (near the fourth container 7). Therefore, it is possible to ensure a good field of view of the two-dimensional camera 4.

[0162] <6. Modifications, etc.> Although the embodiment of the present invention has been described above, the present invention is not limited to the above-described contents.

[0163] For example, in the above embodiment, the cover section 82 of the mobile robot 8 is provided on three side surfaces corresponding to the three sides of the mobile robot 8, which has a substantially rectangular shape in top view, and the cover section 82 is not provided on the side surface facing the robot 2 (see FIG. 1, etc.). However, this is not limited to this. For example, conversely, the cover section 82 may not be provided only on the side surface opposite the robot 2 side, and the cover section 82 may be provided on the other three side surfaces.

[0164] In the above-described embodiment, the container 7 has a rectangular shape when viewed from above, and the colored portion 78 is provided on only one of the four sides of the rectangular shape. However, this is not limiting. The colored portion 78 may also be provided on one or more of the other three sides. In this case, it is preferable that the colored portion 78 on each side has a different color. The controller 5 may distinguish and recognize the color of each colored portion 78 to determine which of the four orientations D1 to D4 the container 7 is oriented in.

[0165] In the above embodiment, the two-dimensional camera 4 and the distance sensor 3 are fixed to the robot hand 2b, but this is not limiting.

[0166] For example, the two-dimensional camera 4 and the distance sensor 3 may be fixed to a link (particularly a link near the tip) of the robot arm 2a.

[0167] Alternatively, the two-dimensional camera 4 and the distance sensor 3 may be fixed to the ceiling directly above the pallet 6 or the like.

[0168] Furthermore, in the above-described embodiment, a container 7 without a lid is exemplified as the target object, but this is not limiting. For example, the target object may be a container with a lid, or may be something other than a container (such as a cardboard box). If the target object is a cardboard box or the like, in step S11, the position of the boundary line (groove) between adjacent objects is detected on each line segment trajectory (based on height variations, etc.), and the pinhole stacking orientation is determined based on whether the boundary line position is in the first half or second half of each line segment trajectory. Furthermore, the height of the top surface of the cardboard box may be detected as the height of the top surface of the target object.

[0169] In addition, in the above-described embodiment, the holders 10, 20, and 30 of the robot hand 2b are arranged clockwise in a top view, but this is not limiting. For example, the holders 10, 20, and 30 may be arranged counterclockwise in a top view (the holders 10 and 20 may be arranged in a mirror image).

[0170] Furthermore, in the above-described embodiment, the container 7 is held by supporting the lower surfaces of the protruding portions 73, 74 of the side portions 71, 72 from below, but this is not limiting. For example, if the total weight of the container 7 and the cargo is lighter than a certain level and / or the container 7 is sturdy, the container 7 may be held by applying a force (squeezing force) from the outside to the inside of both of the two side portions 71 of the container 7 (squeezing the container 7 from both sides).

[0171] In addition, in the above-described embodiment, the robot hand 2b is particularly suited to pinhole stacking, but is not limited to this. For example, the robot hand attached to the end of the robot arm 2a may be of another type (such as a suction type).

[0172] In addition, in the above embodiment, the objects (containers 7) are stacked pinhole-wise, but this is not limiting. The objects may be stacked on the pallet 6 in other ways (for example, block stacking, brick stacking, etc.). [Explanation of symbols]

[0173] 1. Robot System 2. Robot 2a Robotic Arm 2b Robot Hand 3. Distance Sensor 4. 2D Camera 6 palettes 6h Pinhole 7 Containers 8. Mobile Robots 10,20,30,40 Holding part 12,22,32 Support part 13,23,33 Vertical plate 43 Holding plate 50 Main body 51 First main body part 52 Second main body part 71,72 Side part 73,74,73a,73b,74c protrusion 75,76 Side panels

Claims

1. A robot system for handling objects loaded on a pallet, comprising: a robot that handles the object; a distance sensor for detecting the height of the upper surface of the object; a two-dimensional camera for detecting the horizontal position of the object; a control means; Equipped with The robot system is characterized in that the control means identifies the position of the object in space based on the detected upper surface height and horizontal position.

2. the distance sensor is fixed to the robot; The robot system according to claim 1 , wherein the two-dimensional camera is fixed to the robot.

3. the robot includes a robot arm and a robot hand attached to a tip of the robot arm, the distance sensor is fixed to the robot hand; The robot system according to claim 1 , wherein the two-dimensional camera is fixed to the robot hand.

4. 3. The robot system according to claim 2, wherein the control means drives the robot to change the horizontal position of the distance sensor and detects the height of the upper surface of the object at a plurality of different horizontal positions.

5. 5. The robot system according to claim 4, wherein the control means determines the loading direction of the plurality of objects on the top shelf based on the top surface heights detected at a plurality of different horizontal positions.

6. A plurality of objects are pinhole stacked on the pallet, 5. The robot system according to claim 4, wherein the control means determines the orientation of each object stacked with pinholes on the top shelf based on the height of the upper surface detected at a plurality of different horizontal positions.

7. the object has a substantially rectangular shape in top view, and has a colored portion colored with a specific color on one of four sides of the substantially rectangular shape, 2. The robot system according to claim 1, wherein the control means determines which of four horizontal orientations the object has by identifying the one side of the object based on the image captured by the two-dimensional camera.

8. 3. The robot system according to claim 2, wherein the control means drives the robot to move the two-dimensional camera to a position a predetermined distance vertically away from the top surface of the object located at the top of the pallet, and then uses the two-dimensional camera to capture an image of the object.

9. 4. The robot system according to claim 3, wherein the robot hand is capable of rotating about a vertical axis in response to driving of one rotary joint of the robot arm.

10. 4. The robot system according to claim 3, wherein the control means changes the optical axis direction of the two-dimensional camera from horizontal to vertical and moves the vertical position of the two-dimensional camera upward in response to driving of one rotary joint of the robot arm, and causes the two-dimensional camera to photograph the object.

11. A robotic hand, a distance sensor for detecting the height of the top surface of an object loaded on a pallet; a two-dimensional camera for detecting the horizontal position of the object; A robot hand comprising:

12. A position detection method for detecting the position in space of an object loaded on a pallet, comprising: a) detecting the height of the top surface of the object using a distance sensor; b) detecting the horizontal position of the object using a two-dimensional camera; c) determining the position of the object in space based on the detected top surface height and horizontal position; A position detection method comprising:

13. A program for causing a computer to execute the position detection method according to claim 12.

Images