Control device, robot system, and robot control method

The control device and robot system address the challenge of placing multiple workpieces by using a search, evaluation, and determination process to select optimal placement positions within the storage space, ensuring efficient and accurate placement of workpieces.

WO2024247136A1PCT designated stage expired Publication Date: 2024-12-05KAWASAKI JUKOGYO KK
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2023/020185
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-30
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Existing robot control systems face challenges in ensuring the appropriate placement of multiple workpieces without interference, leading to potential misplacement of subsequent workpieces.

Method used

A control device and robot system that includes a search device for candidate placement positions, an evaluator to calculate evaluation values based on free space, and a determiner to select the optimal placement position, ensuring efficient accommodation of workpieces in a storage space.

Benefits of technology

This solution enables the appropriate placement of multiple workpieces by determining the best placement position based on empty space evaluation, improving the efficiency of workpiece arrangement and reducing misplacement.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2023020185_05122024_PF_FP_ABST
    Figure JP2023020185_05122024_PF_FP_ABST
Patent Text Reader

Abstract

A main control device 3 comprises: a searcher 47 that searches for a plurality of position candidates that are candidates for a place target position of a target workpiece W in a storage space; an evaluator 48 that calculates an evaluation value for each of the plurality of position candidates; and a determiner 49 that, on the basis of the evaluation values, determines a place target position from among the plurality of position candidates. The evaluator 48 calculates the evaluation values on the basis of an empty space in the storage space after the target workpiece W is placed in each position candidate.
Need to check novelty before this filing date? Find Prior Art

Description

Control device, robot system, and robot control method

[0001] The technology disclosed herein relates to a control device, a robot system, and a method for controlling a robot.

[0002] Conventionally, there have been known control devices that control a robot that places a workpiece. For example, a control device described in Patent Document 1 calculates a placement position for the workpiece. The control device scans the empty space in a storage container to search for a placement position where the workpiece will not interfere with other existing workpieces.

[0003] Japanese Patent Application Laid-Open No. 2019-181620

[0004] Even if the current workpiece can be placed in a position where it does not interfere with other objects, there is a risk that the subsequent workpieces will not be placed properly, which may result in many workpieces not being placed properly.

[0005] The technology disclosed herein has been made in consideration of the above points, and its purpose is to realize the appropriate placement of multiple works.

[0006] The control device of the present disclosure includes a searcher that searches for multiple position candidates that are candidates for the target placement position of the target work in the storage space, an evaluator that calculates an evaluation value for each of the multiple position candidates, and a determiner that determines the target placement position from among the multiple position candidates based on the evaluation value, and the evaluator calculates the evaluation value based on the available space in the storage space after the target work has been placed at each of the position candidates.

[0007] The robot system of the present disclosure includes a robot having a robot arm and a hand attached to the robot arm, and the control device.

[0008] The robot control method disclosed herein is a method for controlling a robot having a robot arm and a hand attached to the robot arm, and includes searching for a plurality of position candidates that are candidates for a target placement position of a target workpiece in an accommodation space, calculating an evaluation value for each of the plurality of position candidates, and determining the target placement position from among the plurality of position candidates based on the evaluation value, wherein calculating the evaluation value calculates the evaluation value based on the available space in the accommodation space after the target workpiece has been placed at each of the position candidates.

[0009] The control device can realize appropriate placement of multiple workpieces.

[0010] The robot system can realize appropriate placement of multiple workpieces.

[0011] According to the robot control method, it is possible to realize appropriate placement of a plurality of workpieces.

[0012] FIG. 1 is a schematic diagram showing the configuration of a robot system. FIG. 2 is a diagram showing the general hardware configuration of a robot control device and a main control device. FIG. 3 is a block diagram showing the configuration of a control system of a processor. FIG. 4 is a diagram showing the interior of a first container as viewed from above. FIG. 5 is an explanatory diagram showing a hand when picking up a target workpiece. FIG. 6 is an exemplary perspective view showing the interior of a second container. FIG. 7 is a schematic plan view of the interior of the second container for explaining a search for position candidates. FIG. 8 is a schematic plan view of the interior of the second container for explaining narrowing down of position candidates. FIG. 9 is an exemplary perspective view of the interior of the second container showing an empty space after a target workpiece has been placed. FIG. 10 is a diagram showing multiple types of virtual workpieces. FIG. 11 is a schematic plan view of the interior of the second container for explaining calculation of an evaluation value. FIG. 12 is a flowchart of pick-and-place processing. FIG. 13 is a flowchart of a pick-up position planning subroutine. FIG. 14 is a schematic diagram showing the relative position and orientation of the hand when viewed in the direction of the reference axis. Fig. 15 is a schematic diagram showing a region where the angle formed between the side serving as the reference side and the offset direction of the suction device is equal to or greater than a threshold value α, and a region where the angle formed is less than the threshold value α. Fig. 16 is a flowchart of a subroutine for place position planning.

[0013] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An exemplary embodiment will now be described in detail with reference to the accompanying drawings. 1 is a schematic diagram showing the configuration of a robot system 100. As shown in FIG.

[0014] The robot system 100 includes a robot 1 and a main control device 3 that controls the robot 1. The main control device 3 controls the robot 1 to cause the robot 1 to execute a process. In this example, the process performed by the robot 1 is a pick-and-place process. For example, the robot 1 picks up a target workpiece from a first container 91 in which multiple workpieces are piled up randomly, and transfers the target workpiece to a second container 92. The robot 1 arranges the target workpieces in a regular pattern within the second container 92. The robot system 100 arranges the multiple workpieces in the first container 91 within the second container 92 by repeating the pick-and-place process. The target workpiece has, for example, a substantially rectangular parallelepiped shape.

[0015] The robot system 100 may be equipped with a camera 51 that acquires images of the workpieces. The camera 51 may include a first camera 51A that acquires images of the inside of the first container 91 and a second camera 51B that acquires images of the inside of the second container 92. The first camera 51A is fixedly disposed above the first container 91. The first camera 51A captures images of the inside of the first container 91 from above. The first camera 51A acquires images including a plurality of workpieces in the first container 91. The second camera 51B is fixedly disposed above the second container 92. The second camera 51B captures images of the inside of the second container 92 from above. The second camera 51B acquires images including a plurality of workpieces in the second container 92. Hereinafter, when there is no need to distinguish between the first camera 51A and the second camera 51B, they will be simply referred to as cameras 51.

[0016] Here, the image is a two-dimensional image or a three-dimensional image. The three-dimensional image may be point cloud data, an RGB-D image, an RGB image, a depth image, a voxel image, or the like. That is, the camera 51 may be a three-dimensional camera, i.e., an RGB-D camera that outputs an RGB-D image, a stereo camera that acquires an RGB image, or a three-dimensional vision sensor that acquires point cloud data.

[0017] -Robot- The robot 1 has a robot arm 12 and a hand 14 attached to the robot arm 12. In this example, the robot 1 is an industrial robot. The hand 14 is a so-called end effector. A base coordinate system with three orthogonal axes is defined in the space in which the robot 1 is placed.

[0018] The robot arm 12 is configured to move three-dimensionally. Specifically, the robot arm 12 is configured to perform movements including translation with at least three degrees of freedom. In this example, the robot arm 12 is a vertical multi-joint robot arm. The robot arm 12 is supported by a base 10. The robot arm 12 has a plurality of links, a plurality of joints connecting the plurality of links, and servo motors that rotate the plurality of joints.

[0019] In detail, the robot arm 12 has a first link 12a connected to the base 10, a second link 12b connected to the first link 12a, a third link 12c connected to the second link 12b, a fourth link 12d connected to the third link 12c, and a fifth link 12e connected to the fourth link 12d.

[0020] Specifically, the base 10 and the first link 12a are connected to each other via a first joint 13a that is rotatable about an axis extending in the vertical direction. The first link 12a and the second link 12b are connected to each other via a second joint 13b that is rotatable about an axis extending in the horizontal direction. The second link 12b and the third link 12c are connected to each other via a third joint 13c that is rotatable about an axis extending in the horizontal direction. The third link 12c and the fourth link 12d are connected to each other via a fourth joint 13d that is rotatable about the axis of the fourth link 12d (i.e., the direction in which the fourth link 12d extends). The fourth link 12d and the fifth link 12e are connected to each other via a fifth joint 13e that is rotatable about an axis perpendicular to the axis of the fourth link 12d.

[0021] The robot arm 12 has servo motors 16 (see FIG. 2) that rotate and drive each joint. Each servo motor 16 has an encoder 16a (see FIG. 2).

[0022] The robot arm 12 configured in this manner is configured to perform translational movements in the directions of three orthogonal axes and rotational movements around each of the three orthogonal axes.

[0023] The hand 14 includes a suction device 15 that is disposed at a position offset in the radial direction around a predetermined reference axis T of the robot arm 12. The suction device 15 is disposed at a position eccentric to the reference axis T. For example, the hand 14 has a base 14a that is attached to the robot arm 12. The base 14a extends in the radial direction around the reference axis T. The suction device 15 is attached to the end of the base 14a that is farther from the reference axis T via a plate, a block, or the like.

[0024] Furthermore, the suction device 15 is configured to be able to change its position relative to the robot arm 12. For example, the hand 14 is attached to the robot arm 12 so as to be rotatable about a reference axis T. Specifically, the hand 14 is connected to the tip of the robot arm 12, i.e., the fifth link 12e. The fifth link 12e and the hand 14 are connected to each other via a sixth joint 13f so as to be rotatable about the reference axis T. The suction device 15 changes its position relative to the robot arm 12 by rotating the hand 14 about the reference axis T.

[0025] The hand 14 sucks the workpiece W using a suction device 15. The suction device 15 has a suction pad. An air hose connected to a negative pressure generator is connected to the suction device 15. A solenoid valve serving as an actuator 15a (see FIG. 2) is provided on the air hose. By controlling the solenoid valve, the suction device 15 can switch between suction and release. The suction device 15 sucks the surface of the target workpiece W. A pressure sensor 15b (see FIG. 2) is provided on the air hose. The pressure sensor 15b detects the magnitude of the negative pressure in the air hose.

[0026] 2 is a diagram showing the general hardware configuration of the robot controller 2 and the main controller 3. The main controller 3 transmits and receives signals, information, etc. to and from the robot controller 2. The main controller 3 controls the robot 1 via the robot controller 2. The main controller 3 outputs commands to the robot controller 2. The robot controller 2 controls the servo motor 16 of the robot arm 12 in response to commands from the main controller 3.

[0027] Images from the first camera 51A and the second camera 51B are input to the main control device 3. The main control device 3 detects the target workpiece W based on the images. In addition, the main control device 3 generates a target path for the robot 1 based on the detected target workpiece W, and outputs a command corresponding to the generated target path to the robot control device 2. An actuator 15a and a pressure sensor 15b are connected to the main control device 3. The main control device 3 controls suction by the hand 14 via the actuator 15a and the pressure sensor 15b. The main control device 3 is an example of a control device.

[0028] The robot control device 2 has a processor 21 , a storage device 22 , and a memory 23 .

[0029] The processor 21 controls the entire robot control device 2. The processor 21 performs various types of arithmetic processing. For example, the processor 21 is formed of a processor such as a CPU (Central Processing Unit). The processor 21 may also be formed of an MCU (Micro Controller Unit), an MPU (Micro Processor Unit), an FPGA (Field Programmable Gate Array), a PLC (Programmable Logic Controller), a system LSI, or the like.

[0030] The storage unit 22 stores various data and programs executed by the processor 21. The storage unit 22 is formed of a non-volatile memory, a hard disk drive (HDD), a solid state drive (SSD), etc. The memory 23 temporarily stores data, etc. For example, the memory 23 is formed of a volatile memory.

[0031] The processor 21 drives the servo motor 16 based on a command from the main control device 3. At this time, the robot control device 2 feedback controls the current supplied to the servo motor 16 based on the detection result of the encoder 16a.

[0032] The main control device 3 has a processor 31 , a storage device 32 , and a memory 33 .

[0033] The processor 31 controls the entire main control device 3. The processor 31 performs various types of arithmetic processing. For example, the processor 31 is formed of a processor such as a CPU (Central Processing Unit). The processor 31 may also be formed of an MCU (Micro Controller Unit), an MPU (Micro Processor Unit), an FPGA (Field Programmable Gate Array), a PLC (Programmable Logic Controller), a system LSI, or the like.

[0034] The storage unit 32 stores various data and programs executed by the processor 31. The storage unit 32 is formed of a nonvolatile memory, a hard disk drive (HDD), a solid state drive (SSD), or the like.

[0035] For example, the memory 32 stores an object detection program for detecting an object, a position and orientation determination program for determining the position and orientation of the hand 14, and a path generation program for generating a target path for the robot 1. The various programs cause the main control device 3 to realize various functions.

[0036] The memory 33 temporarily stores data etc. For example, the memory 33 is formed of a volatile memory. The memory 33 stores images from the camera 51.

[0037] The processor 31 causes the robot 1 to perform a picking operation in which the hand 14 moves to the target workpiece W and causes the suction device 15 to pick up the target workpiece W, and a placing operation in which the hand 14 that has sucked up the target workpiece W moves to a target place position and causes the suction device 15 to release the suction of the target workpiece W. The processor 31 causes the robot 1 to perform a placing operation after the picking operation.

[0038] Prior to the picking operation, the processor 31 determines the relative position and posture of the hand 14, which is the position and posture of the hand 14 with respect to the target workpiece W at the time of picking. Prior to the placing operation, the processor 31 determines the target placement position of the target workpiece W at the time of placing.

[0039] FIG. 3 is a block diagram showing the configuration of the control system of the processor 31. The processor 31 realizes various functions by reading and storing a program from the storage device 32 into the memory 33. Specifically, the processor 31 functions as a searcher 47 that searches for multiple position candidates that are candidates for the target placement position of the target workpiece in the storage space, an evaluator 48 that calculates an evaluation value for each of the multiple position candidates, and a determiner 49 that determines the target placement position from among the multiple position candidates based on the evaluation values. The processor 31 also functions as a height detector 46 that detects height information of the storage space. Furthermore, the processor 31 can also function as a workpiece detector 41 that detects the target workpiece W to be picked, a position and orientation determiner 42 that determines the relative position and orientation of the hand 14, a path generator 43 that generates a target path for the robot arm 12 and the hand 14, an operation controller 44 that causes the robot arm 12 and the hand 14 to perform the picking operation and the placing operation, and an imaging controller 45 that causes the camera 51 to acquire images.

[0040] The photography controller 45 controls the first camera 51A and the second camera 51B to capture images from each of the first camera 51A and the second camera 51B. The first camera 51A captures an image of the inside of the first container 91. The second camera 51B captures an image of the inside of the second container 92. The photography controller 45 stores the images from the cameras 51A in the memory 33.

[0041] The workpiece detector 41 detects the target workpiece W from the image by image recognition. The image may include multiple workpieces. The workpiece detector 41 performs plane detection using, for example, RANSAC (Random Sample Consensus) to detect the surface of the target workpiece W (hereinafter referred to as the "target surface S"). In other words, the workpiece detector 41 detects the target workpiece W by detecting a plane of a predetermined shape included in the image. FIG. 4 is a top view of the inside of the first container 91. The workpiece detector 41 can detect multiple target surfaces S, i.e., multiple target workpieces W. When detecting the target surface S, the workpiece detector 41 determines at least the center position of the target surface S, the normal direction of the target surface S, the vertical and horizontal orientation of the target surface S, and the size (e.g., vertical and horizontal dimensions) of the target surface S. The size of the target surface S may be stored in the memory 32 in advance.

[0042] The position and orientation determiner 42 determines the relative position and orientation of the hand 14 during picking, using the target surface S detected as the target workpiece W as the suction surface. FIG. 5 is an explanatory diagram showing the hand 14 when suctioning the target workpiece W. Specifically, the position and orientation of the hand 14 is determined so that the suction device 15 is located at a suction position on the target surface S. The suction position is, for example, the center of the target surface S. Because the suction device 15 is eccentric with respect to the reference axis T, even if the position of the suction device 15 is determined, the reference axis T can be positioned at any position within 360 degrees around the suction device 15. Therefore, the relative position and orientation of the hand 14 is determined by the position of the suction device 15 with respect to the target workpiece W, the position of the reference axis T, and the orientation of the reference axis T. In other words, once the relative position and orientation of the hand 14 is determined, the position and orientation of the reference axis T with respect to the target workpiece W and the offset direction of the suction device 15 with respect to the reference axis T are also determined. The "offset direction of the suction device 15 with respect to the reference axis T" is the direction in which a straight line connecting the reference axis T and the suction device 15 extends from the reference axis T when viewed in the direction of the reference axis T, and in this example, is the direction in which the base 14a extends from the reference axis T. In other words, the offset direction of the suction device 15 with respect to the reference axis T is the eccentric direction of the suction device 15 with respect to the reference axis T. Hereinafter, the offset direction of the suction device 15 with respect to the reference axis T will be simply referred to as the "offset direction of the suction device 15."

[0043] In this example, the position and posture determiner 42 determines the relative position and posture of the hand 14 so that, when viewed in the direction of the reference axis T, the reference side of the target workpiece W (hereinafter referred to as the "reference side") does not overlap with the hand 14, and the offset direction of the suction device 15 relative to the reference axis T is non-parallel to the reference side.

[0044] The "side of the target workpiece W" is a portion of the target workpiece W facing to the side. Specifically, when the target workpiece W is viewed in the direction of the reference axis T, it is a portion of the target workpiece W facing in a direction intersecting the normal direction. In other words, the side of the target workpiece W is a side surface of the target workpiece W that intersects with the suction surface that is suctioned by the suction device 15. In this example, since the target workpiece W has a roughly rectangular parallelepiped shape, the portion of the target workpiece W facing in a direction roughly perpendicular to the normal direction of the target surface S, that is, the four side surfaces other than the target surface S and the surface opposite it, are the side of the target workpiece W.

[0045] The path generator 43 generates a target path for the robot arm 12 and the hand 14. The target path is an ordered set of target positions and target orientations of the robot arm 12 and the hand 14. That is, the target path is a time-dependent change in the target position and target orientation of the robot arm 12 and the hand 14. The target path may also be an ordered set of target rotation angles of each joint of the robot arm 12 (including the sixth joint 13f) that realizes the time-dependent change in the target position and target orientation of the robot arm 12 and the hand 14. The target path uniquely defines the position and orientation of the robot arm 12 and the hand 14. The target path also defines the target rotation position of the suction device 15 around the reference axis T. For example, the path generator 43 generates a target path for the robot arm 12 and the hand 14 to move the hand 14 to the determined relative position and orientation of the hand 14. The path generator 43 generates a target path for the robot arm 12 and the hand 14 to move the target workpiece W to the determined target placement position.

[0046] For example, the path generator 43 executes path planning using a probabilistic roadmap method (PRM) or a rapidly-exploring random tree (RRT). In these methods, the path generator 43 also executes an interference check, i.e., checks whether the robot arm 12 and the hand 14 interfere with other objects. The interference check may be performed using point cloud data or CAD data, for example.

[0047] The motion controller 44 outputs a command according to the target path to the robot control device 2. Specifically, the motion controller 44 outputs a command according to the target rotation angle of each joint of the robot arm 12 to the robot control device 2. As described above, the robot control device 2 drives the servo motor 16 based on the command, causing the robot arm 12 and the hand 14 to move along the target path.

[0048] The operation controller 44 switches between suction by the suction device 15 and release of the suction by controlling the actuator 15a of the hand 14. The operation controller 44 controls the actuator 15a to generate negative pressure in the suction device 15, thereby causing the suction device 15 to suction the workpiece. The operation controller 44 controls the actuator 15a to release the negative pressure in the suction device 15, thereby causing the suction device 15 to release the suction of the workpiece W. The operation controller 44 determines that suction by the suction device 15 has been completed, based on the detection result of the pressure sensor 15b. The operation controller 44 determines that suction has been completed when the negative pressure detected by the pressure sensor 15b is equal to or greater than a predetermined threshold.

[0049] The height detector 46 detects height information of the storage space in which the target workpiece W will be stored. In this example, the storage space is the inside of the second container 92. The height information is information about the height of each position in the storage space, specifically, height information about each position within the second container 92. In areas within the second container 92 where the bottom surface is exposed, the height information is the height of each position on the bottom surface. In areas within the second container 92 where workpieces are present, the height information is the height of each position on the top surface of the workpieces. The height detector 46 detects the height information of the storage space based on an image of the inside of the second container 92. The height detector 46 obtains the height information of the storage space, for example, by performing preprocessing such as discretization or smoothing on point cloud data of a three-dimensional image. The height detector 46 outputs the height information of the storage space as a height map.

[0050] The searcher 47 searches for a plurality of position candidates that are candidates for the target placement position of the target work W in the accommodation space. The work detector 41 and the position and orientation determiner 42 identify the target work W, and thereby the dimensions of the target work W are also acquired. The searcher 47 searches for positions in the accommodation space where the target work W can be placed, as position candidates for the target placement position.

[0051] For example, the searcher 47 detects a space in the accommodation space where the target work W can be placed (hereinafter referred to as a "placeable space"). The searcher 47 detects a plane P in the accommodation space where the target work W can be placed (hereinafter referred to as a "placeable surface") based on the height map, and the space above the placeable surface P is designated as the placeable space. FIG. 6 is an exemplary perspective view showing the inside of the second container 92. In FIG. 6, the dotted surface is the placeable surface P. The placeable surface P is a plane exposed upward and is an area with a uniform height in the height map. The placeable surface P is an exposed portion of the bottom surface of the second container 92. If the top surface of the existing work W0 is also exposed, it can also be a placeable surface P. Note that if multiple existing workpieces W0 are lined up, the top surfaces of these existing workpieces W0 are exposed upward, and the heights of the top surfaces of these existing workpieces W0 are the same, the top surfaces of these existing workpieces W0 become a single placeable surface P. In the example of FIG. 6, the top surfaces of the two existing workpieces W0 are treated as separate placeable surfaces P.

[0052] The searcher 47 determines whether the target work W can be placed at each position while changing the position of the target work W on the placement surface P. FIG. 7 is a schematic plan view of the inside of the second container 92 to explain the search for position candidates. A position determined to be possible for placement becomes a position candidate. Whether or not the target work W can be placed is determined based on whether or not the target work W can be placed without interfering with other objects. At this time, the searcher 47 sets the size of the target work W for search (see dashed line) to the actual size of the target work W (see dashed line) plus a predetermined margin. Specifically, the searcher 47 enlarges the bottom surface of the target work W by the amount of the predetermined margin.

[0053] In this way, the searcher 47 searches for position candidates using not only the bottom surface of the second container 92 but also the top surface of the existing workpiece W0 as the possible placement surface P.

[0054] Furthermore, the searcher 47 narrows down the multiple position candidates from the perspective of workpiece storage efficiency. For example, the searcher 47 selects a position candidate corresponding to a corner space in the storage space from the multiple position candidates. FIG. 8 is a schematic plan view of the inside of the second container 92 to explain the narrowing down of the position candidates. A corner may be formed inside the second container 92 by the side wall of the second container 92 and the side surface of the existing workpiece W0. When storing workpieces in the second container 92, it is preferable to pack the workpieces into a space including the corner from the perspective of workpiece storage efficiency. Therefore, as shown in FIG. 8, the searcher 47 selects a position candidate corresponding to a corner space from the multiple position candidates. Note that if the multiple position candidates do not include a position candidate corresponding to a corner space, the searcher 47 does not narrow down the position candidates.

[0055] The evaluator 48 calculates an evaluation value based on the vacant space in the storage space after the target work W has been placed at each position candidate. FIG. 9 is an exemplary perspective view of the second container 92, showing the vacant space after the target work W has been placed. In FIG. 9, the dotted surface is the placeable surface P. The vacant space is the same space as the placeable space. In other words, the vacant space is a space in which a work can be placed after the target work W has been placed. As in the search, a vacant space is formed above the placeable surface P. The placeable surface P is a plane exposed upward. When the target work W placed at the position candidate is lined up with the existing work W0, the top surfaces of both workpieces are exposed upward, and the heights of the top surfaces of both workpieces are the same, the top surfaces of both workpieces are considered to be a single placeable surface P.

[0056] The evaluation value is a parameter that indicates the efficiency with which virtual work can be accommodated in free space. Accommodation efficiency is the efficiency with which virtual work can be accommodated in free space, and indicates how efficiently virtual work can be accommodated in free space. For example, the accommodation efficiency increases the greater the number of virtual works that can be accommodated and the greater the variety of locations at which virtual work can be accommodated. For example, the evaluation value is a parameter related to the number of candidate locations at which virtual work can be placed in free space. Alternatively, the evaluation value may be a parameter related to the number of virtual works that can be accommodated in free space.

[0057] Specifically, the evaluator 48 calculates the number of candidate positions where the virtual workpiece Wi can be placed on the placement surface P after the target workpiece W has been placed at the target candidate position. Fig. 10 is a diagram showing multiple types of virtual workpieces. Fig. 11 is a schematic plan view of the inside of the second container 92 for explaining the calculation of the evaluation value.

[0058] The virtual workpiece Wi may include multiple types of virtual workpieces Wi of different sizes. When differentiating the types of virtual workpieces Wi, they are distinguished by the number following the symbol "Wi." Specifically, the multiple types of virtual workpieces Wi include a first virtual workpiece Wi1, a second virtual workpiece Wi2, a third virtual workpiece Wi3, a fourth virtual workpiece Wi4, and a fifth virtual workpiece Wi5. The multiple types of virtual workpieces Wi have different two-dimensional sizes. In this example, the multiple types of virtual workpieces Wi have approximately square two-dimensional shapes with different areas. The first virtual workpiece Wi1 has the smallest square two-dimensional shape. The second virtual workpiece Wi2 has a two-dimensional shape that is larger than the first virtual workpiece Wi1. The third virtual workpiece Wi3 has a two-dimensional shape that is larger than the second virtual workpiece Wi2. The fourth virtual workpiece Wi4 has a two-dimensional shape that is larger than the third virtual workpiece Wi3. The fifth virtual workpiece Wi5 has a two-dimensional shape that is larger than the fourth virtual workpiece Wi4.

[0059] A different score is assigned to each of the multiple types of virtual workpieces Wi. In this example, a score corresponding to the size of each of the multiple types of virtual workpieces Wi is assigned. If a large virtual workpiece Wi can be accommodated, it is natural that a smaller virtual workpiece Wi can also be accommodated. Therefore, the larger the area of ​​the two-dimensional shape of the multiple types of virtual workpieces Wi, the higher the score assigned. More specifically, the first virtual workpiece Wi1 is assigned a score of 3 points. The second virtual workpiece Wi2 is assigned a score of 6 points. The third virtual workpiece Wi3 is assigned a score of 9 points. The fourth virtual workpiece Wi4 is assigned a score of 12 points. The fifth virtual workpiece Wi5 is assigned a score of 15 points.

[0060] The evaluator 48 determines how many candidate positions there are for placing the virtual work Wi in the empty space after the target work W has been placed. The evaluator 48 moves the position of the virtual work Wi two-dimensionally by a predetermined amount within the empty space and determines whether the virtual work Wi can be placed without interfering with other objects. Specifically, as shown in FIG. 11 , the evaluator 48 moves the virtual work Wi on the placeable surface P after the target work W has been placed in one of the candidate positions, and determines at each position whether the virtual work Wi can be placed without interfering with other objects. If the virtual work Wi can be placed, the evaluator 48 counts that position as a candidate position. The evaluator 48 moves the virtual work Wi over the entire range of the placeable surface P and counts the number of candidates. The evaluator 48 also moves the virtual work Wi to the placeable surfaces P for the target work W and the existing work W0 and counts the number of candidates. When the target workpiece W is lined up with the existing workpiece W0, the top surfaces of the target workpiece W and the existing workpiece W0 are exposed upward, and the heights of the top surfaces of both works are the same, the evaluator 48 determines whether the virtual workpiece Wi can be placed on the top surfaces of both works as a uniform placeable surface P.

[0061] The greater the number of candidates, the greater the possibility of accommodating more virtual works Wi. In other words, the number of candidates is related to the capacity of the virtual works Wi. The evaluator 48 multiplies the number of candidates by the score assigned to the virtual work Wi to obtain the total score of the virtual work Wi.

[0062] The evaluator 48 calculates the total score of the virtual work Wi for all types of virtual work Wi. The evaluator 48 calculates the sum of the total scores of all types of virtual work Wi as the evaluation value.

[0063] After calculating the evaluation value for one position candidate, the evaluator 48 calculates the evaluation value when the target work W is placed at another position candidate. Finally, the evaluator 48 calculates the evaluation values ​​for all position candidates.

[0064] The evaluation value thus obtained increases as the available space can accommodate more large virtual works Wi. Being able to accommodate more large virtual works Wi means that more works can be placed with greater freedom in the future. In other words, the evaluation value indicates the efficiency with which virtual works Wi can be accommodated in the available space after the target work W has been placed.

[0065] The determiner 49 determines the position candidate with the highest evaluation value from among the multiple position candidates as the target placement position. The path generator 43 generates a target path for the robot arm 12 and hand 14 to transfer the target workpiece W to the determined target placement position.

[0066] Next, a description will be given of the operation of the robot system 100 configured as above. The pick-and-place process in the robot system 100 will be described with reference to the flowchart in Fig. 12. Fig. 12 is a flowchart of the pick-and-place process.

[0067] First, in step S101, the photography controller 45 acquires an image of the inside of the first container 91. More specifically, the photography controller 45 causes the first camera 51A to acquire an image of the inside of the first container 91.

[0068] Next, in step S102, the workpiece detector 41 detects the target workpiece W, specifically the target surface S, from the image inside the first container 91. For example, the workpiece detector 41 detects a plurality of target surfaces S.

[0069] Subsequently, in step S103, the position and orientation determiner 42 executes a pickup position plan. In the pickup position plan, the target workpiece W to be picked up by the hand 14 is determined, and the relative position and orientation of the hand 14 with respect to the target workpiece W is also determined. At this time, a target path for the robot arm 12 and the hand 14 to realize the relative position and orientation of the hand 14 is also generated.

[0070] Once the relative position and posture of the hand 14 has been determined, in step S104, the motion controller 44 causes the robot arm 12 and the hand 14 to perform a picking operation. That is, the motion controller 44 controls the robot arm 12 and the hand 14 so that the target workpiece W determined in the pickup position plan is picked up at the determined relative position and posture of the hand 14. Specifically, the motion controller 44 moves the robot arm 12 and the hand 14 along the target path, and moves the suction device 15 to the target workpiece W. The motion controller 44 causes the suction device 15 to pick up the target workpiece W.

[0071] During the picking operation or after the picking operation is completed, in step S105, the photography controller 45 captures an image of the inside of the second container 92. More specifically, the photography controller 45 causes the second camera 51B to capture an image of the inside of the second container 92.

[0072] Next, in step S106, a place position plan is executed. In the place position plan, a target place position of the target workpiece W is determined. At this time, a target path of the robot arm 12 and the hand 14 for realizing the target place position of the target workpiece W is also generated.

[0073] Once the target placement position has been determined, in step S107, the motion controller 44 causes the robot arm 12 and hand 14 to perform a placement operation. That is, the motion controller 44 controls the robot arm 12 and hand 14 so that the target workpiece W is placed at the target placement position determined in the placement position plan. Specifically, the motion controller 44 moves the robot arm 12 and hand 14 along the target path to move the target workpiece W to the target placement position. When the target workpiece W reaches the target placement position, the motion controller 44 releases the suction by the suction device 15. In this way, the placement of the target workpiece W at the target placement position is completed. This completes the pick-and-place process.

[0074] When the main control device 3 completes the pick-and-place process for one target workpiece W, it repeats the process from step S101 to pick-and-place another target workpiece W. By repeating this control, the main control device 3 sequentially transfers the target workpieces W in the first container 91 into the second container 92.

[0075] =Pickup Position Planning= Next, the pickup position planning will be described in detail. Fig. 13 is a flowchart of the pickup position planning subroutine.

[0076] In step S201, the position and orientation determiner 42 generates candidates for the relative position and orientation of the hand 14. At this time, the position and orientation determiner 42 may generate multiple candidates for the relative position and orientation of the hand 14. More specifically, the position and orientation determiner 42 may generate multiple candidates for the relative position and orientation of the hand 14 with respect to one target surface S. Furthermore, if there are multiple target surfaces S, the position and orientation determiner 42 may generate candidates for the relative position and orientation of the hand 14 with respect to each of the target surfaces S.

[0077] FIG. 14 is a schematic diagram showing the relative position and orientation of the hand 14 when viewed in the direction of the reference axis T. In FIG. 14, the dashed-dotted line indicates the offset direction OD of the suction device 15. During picking, the suction device 15 is positioned at a pickup position on the target surface S, and the reference axis T is approximately perpendicular to the target surface S. The position of the reference axis T can be any position within 360 degrees around the suction device 15. In other words, the position and orientation determiner 42 can generate multiple candidates for the relative position and orientation of the hand 14, in which the reference axis T is located at different positions in the circumferential direction around the pickup position on the target surface S. In this example, the pickup position is approximately the center of the target surface S. Furthermore, the position and orientation determiner 42 can generate multiple candidates for the relative position and orientation of the hand 14 for different target surfaces S.

[0078] Next, in step S202, the position and orientation determiner 42 assigns priorities to the 14 relative position and orientation candidates of the hand. For example, the position and orientation determiner 42 assigns priorities from two perspectives.

[0079] In the first viewpoint, a high priority is assigned to a candidate of the target workpiece W whose reference side does not overlap with the hand 14 when viewed in the direction of the reference axis T and whose offset direction of the suction device 15 with respect to the reference axis T is not parallel to the reference side. When this condition is met, the hand 14 picks up the target workpiece W in a state where the reference axis T and the reference side are relatively far apart when viewed in the direction of the reference axis T.

[0080] In this example, the position and orientation determiner 42 sets two reference sides for the target workpiece W. For example, the position and orientation determiner 42 sets any two of the sides R1, R2, R3, and R4 of the target workpiece W, specifically, the two sides that are connected to form a corner of the target workpiece W, as the reference sides. The position and orientation determiner 42 then assigns a high priority to candidates for which the hand 14 does not overlap either of the two reference sides and the offset direction of the suction device 15 with respect to the reference axis T is non-parallel for both of the two sides. Furthermore, the position and orientation determiner 42 may change the reference sides to two other sides and assign a high priority to candidates for which the two changed sides satisfy the above-described conditions.

[0081] Specifically, the position and orientation determiner 42 regards the edges of the target surface S as the sides of the target workpiece W. The position and orientation determiner 42 assigns a relatively high priority to candidates that satisfy the above-described conditions for each pair of edges that form the four corners of the target surface S, i.e., for a total of four pairs of edges. The position and orientation determiner 42 then assigns a relatively low priority to the remaining candidates. For example, if the angle between the edge serving as the reference side and the offset direction of the suction device 15 is equal to or greater than a threshold value α, the position and orientation determiner 42 regards the reference side and the offset direction as non-parallel. FIG. 15 is a schematic diagram showing a region A where the angle between the edge serving as the reference side and the offset direction of the suction device 15 is equal to or greater than the threshold value α and a region B where the angle is less than the threshold value α. As a result, as shown in FIG. 15 , a relatively high priority is assigned to candidates whose offset direction of the suction device 15 falls within region A. On the other hand, a relatively low priority is assigned to candidates whose offset direction of the suction device 15 falls within region B.

[0082] Alternatively, two reference sides may be determined in advance for the target workpiece W. In this case, the position and orientation determiner 42 assigns a high priority to candidates that satisfy the above-described conditions for the pair of reference sides, and assigns a low priority to other candidates.

[0083] In the second aspect, the position and orientation determiner 42 assigns a higher priority to a candidate with a smaller amount of movement from the current state, i.e., the current position, of the robot arm 12. For example, a higher priority may be assigned to a candidate with a smaller total amount of movement of all the joints of the robot arm 12.

[0084] Next, in step S203, the position and orientation determiner 42 determines whether or not there are candidates for the relative position and orientation of the hand 14. If there is no appropriate target surface S, it is possible that no candidates for the relative position and orientation of the hand 14 are generated in step S201 described above. In such a case, the pick-and-place process ends in step S206.

[0085] If there are candidates for the relative position and orientation of the hand 14, in step S204, the position and orientation determiner 42 extracts the candidate with the highest priority. Then, the path generator 43 generates a target path for the robot arm 12 from the current position and orientation of the hand 14 to the extracted relative position and orientation of the hand 14. At this time, the path generator 43 checks for interference with the target path and generates a target path that is free of interference. If interference is detected, the path generator 43 does not generate a target path.

[0086] In step S205, the position and orientation determiner 42 determines whether or not a target path has been generated. If a target path has not been generated, the position and orientation determiner 42 excludes the extracted candidate for the relative position and orientation of the hand 14 from the candidates, and the process returns to step S203. The position and orientation determiner 42 again extracts the candidate with the highest priority, and the path generator 43 again generates a target path and performs an interference check.

[0087] In this way, the position and orientation determiner 42 extracts candidates in descending order of priority from among the candidates for the relative position and orientation of the hand 14. When an interference-free target path is generated for the extracted candidates, the position and orientation determiner 42 determines the relative position and orientation of the hand 14 and ends the pickup position planning.

[0088] =Place Position Planning= Next, the place position planning will be described in detail. Fig. 16 is a flowchart of the place position planning subroutine.

[0089] In step S301, the height detector 46 creates a height map of the interior of the receiving space, i.e., the second container 92.

[0090] Next, in step S302, the searcher 47 searches for a plurality of position candidates for the target placement position. Specifically, the searcher 47 creates a plurality of position candidates for the target placement position of the target work W within the second container 92. Step S302 corresponds to searching for a plurality of position candidates that are candidates for the target placement position of the target work W in the accommodation space.

[0091] Next, in step S303, the searcher 47 determines whether or not there are any position candidates. If there is no space in the second container 92, it is possible that no position candidates are found in step S302. In such a case, the pick-and-place process ends in step S309.

[0092] If there are location candidates, in step S304, the searcher 47 narrows down the location candidates. For example, the searcher 47 selects, from among the multiple location candidates, a location candidate corresponding to a corner space in the storage space. Note that if there are no location candidates to be narrowed down, the searcher 47 does not narrow down the location candidates.

[0093] Thereafter, in step S305, the evaluator 48 calculates an evaluation value for the location candidate. If multiple location candidates are selected in step S304, the evaluator 48 calculates an evaluation value for each of the multiple location candidates. Step S305 corresponds to calculating an evaluation value for each of the multiple location candidates.

[0094] In step S306, the path generator 43 generates a target path for the position candidate with the highest evaluation value. That is, the path generator 43 generates a target path for the robot arm 12 and the hand 14 from the current position and posture of the robot arm 12 and the hand 14 to the position and posture corresponding to the position candidate with the highest evaluation value. At this time, the path generator 43 performs an interference check of the target path to confirm whether or not there is interference. If there is interference, the path generator 43 excludes the position candidate where the interference occurred and generates a target path for the position candidate with the next highest evaluation value. In this way, the path generator 43 generates target paths from the position candidates in descending order of evaluation value.

[0095] In step S307, the determiner 49 determines the place target position to be the position candidate for which the interference-free target route has been generated. This completes the place position planning. Step S307 corresponds to determining the place target position from among the plurality of position candidates based on the evaluation value.

[0096] On the other hand, if an interference-free target path is not generated, i.e., if a place target position is not determined, the searcher 47 changes the margin added to the actual size of the target work W when searching for position candidates in step S308, and returns to searching for position candidates in step S302.

[0097] By changing the size of the target workpiece W used when searching for location candidates, the derived location candidates may change from the previous search. Specifically, the searcher 47 expands the margin. As a result, when searching for location candidates again, the searcher 47 searches for location candidates using a target workpiece W that is larger than the previous one.

[0098] When the target workpiece W is placed at a place target position determined based on such position candidates, the target workpiece W has a larger margin between it and surrounding objects. Therefore, by using such position candidates in step S306, the likelihood of generating a target path without interference increases. Once the place target position and target path are generated, the place position planning is completed.

[0099] If an appropriate target route is still not generated, the margin is further increased and the process is repeated from step S302.

[0100] In such a pick-and-place process, the target placement position is determined based on the future workpiece storage efficiency after the target workpiece W has been placed in the storage space. If the target placement position is determined taking only the current target workpiece W into consideration, there is a risk that the storage efficiency for the target workpiece W from the next time onwards will decrease. The main control device 3 calculates an evaluation value based on the available space after the target workpiece W has been placed, and determines the position candidate with the highest evaluation as the target placement position. This makes it possible to achieve appropriate placement of multiple workpieces, including not only the current target workpiece W but also the next and subsequent target workpieces W.

[0101] The evaluation of the free space can be performed from various viewpoints. For example, the evaluation may be based on how many workpieces can be accommodated in the free space. Alternatively, the evaluation may be based on how freely the workpieces can be arranged in the free space. In this example, the evaluation value relates to the accommodation efficiency of the virtual workpiece Wi in the free space. This makes it possible to determine the target placement position so that the accommodation efficiency of the virtual workpiece Wi in the free space after the target workpiece W is placed is high. In other words, the accommodation efficiency of multiple workpieces from this time onwards can be improved.

[0102] Furthermore, the virtual work Wi includes a plurality of types of virtual work Wi with different sizes, and evaluation values ​​are calculated for the plurality of types of virtual work Wi. Therefore, even if work of various sizes is placed in the future, the accommodation efficiency of the plurality of work Wi can be improved.

[0103] The evaluator 48 also treats the top surface of the current target workpiece W as an empty space. In other words, when the target workpiece W and the existing workpiece W0 are placed side by side, the top surfaces of the target workpiece W and the existing workpiece W0 are exposed upward, and the heights of the top surfaces of the target workpiece W and the existing workpiece W0 are the same, the evaluation value is calculated for these top surfaces as a single placeable surface P. In other words, placing the target workpiece W next to the existing workpiece W0 of the same height tends to expand the empty space and increase the evaluation value. In this way, the evaluator 48 can consider not only the two-dimensional storage efficiency of the virtual workpiece Wi in the empty space, but also the three-dimensional storage efficiency.

[0104] The evaluator 48 also calculates, as an evaluation value, a parameter related to the number of candidate positions where the virtual work Wi can be placed in the available space. The evaluator 48 can also calculate, as an evaluation value, a parameter related to the capacity of the virtual work Wi that can be placed in the available space. However, the evaluator 48 can reduce the amount of calculation by using a parameter related to the number of candidate positions where the virtual work Wi can be placed in the available space.

[0105] In this example, the evaluator 48 calculates evaluation values ​​by giving priority to position candidates corresponding to corners of the storage space from among multiple position candidates. Placing workpieces closer to the corners tends to increase storage efficiency. Therefore, by giving priority to calculating evaluation values ​​for position candidates corresponding to corners and determining the placement target position, the amount of calculation can be reduced.

[0106] In addition, the relative position and posture of the hand 14 that picks up the target workpiece W is determined so that the reference side of the target workpiece W and the hand 14 do not overlap when viewed in the direction of the reference axis T, and the offset direction of the suction device 15 is non-parallel to the reference side. The reference sides are two sides of the target workpiece W that are connected to form an angle. As a result, when viewed in the direction of the reference axis T, the distance between the reference axis T and each of the two reference sides is greater than the distance between the suction device 15 and each of the two reference sides. Therefore, when viewed in the direction of the reference axis T, the tip of the robot arm 12 and the two reference sides are less likely to overlap. As a result, it becomes easier to place the target workpiece W by aligning the corner of the target workpiece W with the corner of the storage space.

[0107] Furthermore, the position candidates are searched for based on the size of the target workpiece W, which is the actual size of the target workpiece W expanded by a predetermined margin. Therefore, the finally determined target placement position has a margin within which the target workpiece W can be displaced. In the placement operation, the robot arm 12 and hand 14 move the target workpiece W to the target placement position, and then move the target workpiece W laterally to bring the target workpiece W into contact with another object that forms a corner of the storage space. This allows the corner of the target workpiece W to be properly aligned with the corner of the storage space.

[0108] Other Embodiments As described above, the above-described embodiments have been described as examples of the technology disclosed in the present application. However, the technology of the present disclosure is not limited to these embodiments and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made as appropriate. Furthermore, the components described in the above-described embodiments can be combined to create new embodiments. Furthermore, the components described in the accompanying drawings and detailed description may include not only components essential for solving the problem, but also components that are not essential for solving the problem in order to exemplify the technology. Therefore, the fact that these non-essential components are described in the accompanying drawings or detailed description should not be interpreted as immediately determining that these non-essential components are essential.

[0109] For example, the robot system 100 incorporating the main control device 3 is not limited to one that transfers the target workpiece W from the first container 91 to the second container 92. For example, the robot 1 may perform devanning. The destination of the target workpiece W is not limited to the second container 92. For example, the destination of the target workpiece W may be a conveyor, a shelf, a pallet, or the like.

[0110] The robot arm 12 is not limited to a vertical articulated robot arm, but may be a horizontal articulated robot arm, a parallel link robot arm, a rectangular coordinate robot arm, a polar coordinate robot arm, or the like.

[0111] The suction device 15 does not have to rotate around the reference axis T relative to the robot arm 12. Even if the suction device 15 cannot rotate around the reference axis T, it only needs to be disposed at a position offset in the radial direction from the reference axis T of the robot arm 12, that is, at a position eccentric to the reference axis T. The suction device 15 may attract the workpiece by magnetic force or the like instead of negative pressure.

[0112] The camera 51 does not have to include the first camera 51A and the second camera 51B. One camera 51 may capture an image of the inside of the first container 91 and an image of the inside of the second container 92. The camera 51 may not be fixedly disposed, but may be attached to the robot arm 12 and moved by the robot arm 12.

[0113] The robot system 100 may not be equipped with the camera 51. In that case, an image may be input to the main control device 3 from outside. The method of acquiring the image does not matter. The image input from outside may be stored in the memory 33 or the storage device 32.

[0114] The target workpieces to be picked may not be piled up randomly but may be aligned. The target workpieces are not limited to rectangular parallelepiped workpieces. For example, the target workpieces may have a substantially triangular prism shape. The target workpieces may also be bags filled with powder or granular material such as fertilizer, lime, or gravel.

[0115] The processing by the workpiece detector 41, the position and orientation determiner 42, the motion controller 44, and the imaging controller 45 is merely an example. For example, the workpiece detector 41 can detect the target surface S from the image by various methods. The position and orientation determiner 42 can determine the relative position and orientation of the hand 14 by various methods.

[0116] The reference side of the target workpiece is not limited to a flat surface. If the target workpiece is a bag-shaped workpiece, the reference side may be a portion facing laterally relative to the suction surface that comes into contact with the suction unit 15.

[0117] Alternatively, the position and orientation determiner 42 may approximate the target workpiece to a rectangular parallelepiped, use one face of the approximate rectangular parallelepiped as the chucking face, and use the face on the side of the chucking face as the reference side. For example, the position and orientation determiner 42 may approximate the target workpiece using an OBB (Oriented Bounding Box).

[0118] The position and orientation determiner 42 may execute the pickup position planning after the place position planning. In this case, the position and orientation determiner 42 can determine the relative position and orientation of the hand 14 during picking based on the previously determined target place position. For example, when the target place position is a position corresponding to a corner of the storage space, two sides of the target workpiece W that come into contact with the two side surfaces that form the corner are determined in advance. The position and orientation determiner 42 determines the relative position and orientation of the hand 14 using the two sides as reference sides.

[0119] When generating candidates for the relative position and posture of the hand 14, the position and posture determiner 42 does not need to generate as candidates the relative position and posture of the hand 14 in which the reference side of the target workpiece W and the hand 14 overlap when viewed in the direction of the reference axis T, or the relative position and posture of the hand 14 in which the offset direction of the suction unit 15 is parallel to the reference side when viewed in the direction of the reference axis T. In other words, the position and posture determiner 42 may generate as candidates only the relative position and posture of the hand 14 in which the reference side of the target workpiece W and the hand 14 do not overlap when viewed in the direction of the reference axis T and the offset direction of the suction unit 15 with respect to the reference axis T is not parallel to the reference side.

[0120] Furthermore, the determination of the relative position and posture of the hand 14 by the position and posture determiner 42 is merely one preferred example, and the reference side of the target workpiece W may overlap with the hand 14 when viewed in the direction of the reference axis T, or the offset direction of the suction device 15 may be parallel to the reference side when viewed in the direction of the reference axis T.

[0121] The processing of the searcher 47, the evaluator 48, or the determiner 49 is merely an example. For example, when the searcher 47 narrows down multiple position candidates, the searcher 47 may narrow down the multiple position candidates in stages. As described above, if the place target position cannot be determined by narrowing down the position candidates to those corresponding to corners, the remaining position candidates may be narrowed down to those that contact side surfaces within the storage space. Alternatively, the searcher 47 may narrow down the multiple position candidates to both position candidates that correspond to corners and position candidates that contact side surfaces at the same time. The searcher 47 does not have to narrow down multiple position candidates.

[0122] When searching for position candidates, the searcher 47 adds a predetermined margin to the actual size of the target workpiece W, but is not limited to this. The searcher 47 may also search for position candidates using the actual size of the target workpiece W.

[0123] The evaluator 48 may calculate a parameter related to the number of virtual works Wi that can be placed in an available space as the evaluation value. In other words, the evaluator 48 may adjust the arrangement of the virtual works Wi in the available space to determine the maximum number of virtual works Wi that can be accommodated in the available space for each of multiple types of virtual works Wi. The evaluator 48 may calculate a total score for each virtual work Wi by multiplying the score corresponding to the maximum number of each virtual work Wi, and use the sum of these as the evaluation value.

[0124] The virtual work Wi does not have to include multiple types of virtual work Wi. In other words, the evaluator 48 may calculate the evaluation value using a single type of virtual work Wi.

[0125] When the virtual work Wi includes multiple types of virtual work Wi, the number of types is not limited to 5. The number of types may be 2, 3, 4, or 6 or more.

[0126] The two-dimensional shape of the virtual workpiece Wi is not limited to a square. For example, if the shape of a future workpiece is known, the two-dimensional shape of the virtual workpiece Wi may be the shape of the known workpiece.

[0127] The above-mentioned scores assigned to each of the multiple types of virtual work Wi are merely an example. The evaluation value can be set arbitrarily. The evaluation value may be set so that the higher the evaluation, the smaller the value. When different scores are assigned to each of the multiple types of virtual work Wi, for example, a higher score may be assigned to a virtual work Wi that is expected to be used more frequently in reality.

[0128] The control device does not have to be a single device like the main control device 3. The control device may include a plurality of separate devices. For example, the workpiece detector 41, the position and orientation determiner 42, and the motion controller 44 may each be realized by separate devices.

[0129] The flowchart is merely an example. Steps in the flowchart may be changed, replaced, added, omitted, etc. as appropriate. The order of steps in the flowchart may also be changed, and serial processing may be performed in parallel. For example, photographing the interior of the second container 92 in step S105 and planning the placement position in step S106 in FIG. 12 may be performed at any time before the placing operation. Specifically, photographing the interior of the second container 92 and planning the placement position may be performed prior to or in parallel with the pickup position planning, etc.

[0130] The functions performed by the components described herein may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), a Central Processing Unit (CPU), conventional circuits, and / or combinations thereof, programmed to perform the described functions. A processor includes transistors and other circuits and is considered a circuit or processing circuit. A processor may also be a programmable processor that executes a program stored in a memory.

[0131] In this specification, a circuit, unit, or means is hardware that is programmed to realize or performs the described functions, which may be any hardware disclosed herein or any hardware known to be programmed to realize or perform the described functions.

[0132] If the hardware is a processor considered to be a type of circuitry, the circuit, means, or unit is a combination of hardware and software used to configure the hardware and / or processor.

[0133] The technology of the present disclosure can be summarized as follows.

[0134] [1] The main control device 3 (control device) includes a searcher 47 that searches for multiple position candidates that are candidates for the target placement position of the target work W in the storage space, an evaluator 48 that calculates an evaluation value for each of the multiple position candidates, and a determiner 49 that determines the target placement position from among the multiple position candidates based on the evaluation value, and the evaluator 48 calculates the evaluation value based on the available space in the storage space after the target work W has been placed at each of the position candidates.

[0135] According to this configuration, the target placement position is determined based on the space available after the target work W has been placed. This allows the target placement position to be determined taking into consideration the placement of subsequent workpieces after the target work W has been placed. As a result, it is possible to realize appropriate placement of not only the current target work W, but also multiple workpieces including subsequent workpieces.

[0136] [2] In the main control device 3 described in [1], the evaluation value is a parameter indicating the efficiency with which virtual works Wi can be accommodated in the available space.

[0137] According to this configuration, the target placement position is determined based on the efficiency with which the virtual work Wi can be accommodated in the space available after the target work W has been placed. This allows the target placement position to be determined taking into consideration how efficiently the virtual work Wi can be placed in the space available after the target work W has been placed. As a result, it is possible to improve the efficiency with which multiple works to be placed in the future can be accommodated.

[0138] [3] In the main control device 3 described in [1] or [2], the evaluation value is a parameter related to the number of virtual work Wi that can be placed in the available space or the number of candidate positions where the virtual work Wi can be placed in the available space.

[0139] According to this configuration, the placement target position is determined based on the capacity of the virtual work Wi to be accommodated in the space available after the target work W has been placed. In other words, the capacity of the virtual work Wi directly indicates the capacity of the virtual work Wi. The number of candidate positions at which the virtual work Wi can be placed is related to the capacity of the virtual work Wi. This allows the placement target position to be determined taking into account how many virtual works Wi can be placed in the space available after the target work W has been placed. As a result, the efficiency of accommodating multiple works to be placed in the future can be improved.

[0140] [4] In the main control device 3 described in any one of [1] to [3], the virtual work Wi includes multiple types of virtual work of different sizes, different scores are assigned to the multiple types of virtual work, and the evaluator 48 calculates the evaluation value according to the number of storage spaces or the number of candidates and the score.

[0141] According to this configuration, the efficiency of accommodating virtual works Wi in an available space is evaluated taking into account differences in the size of the virtual works. In other words, when evaluating the efficiency of accommodating virtual works Wi in an available space, differences in the size of the virtual works can be taken into account. As a result, the efficiency of accommodating multiple works can be improved by taking into account variations in the size of works to be placed in the future.

[0142] [5] In the main control device 3 described in any one of [1] to [4], the evaluator 48 calculates the evaluation value by regarding the space above the exposed part of the bottom surface of the storage space and the space above the exposed part of the top surface of the existing work W0 in the storage space as the free space.

[0143] According to this configuration, the evaluator 48 evaluates not only the bottom surface of the accommodation space but also the space above the top surface of the existing work W0 as a space for placing the virtual work Wi. As a result, it is possible to take into account not only the two-dimensional accommodation efficiency of the virtual work Wi in the empty space but also the three-dimensional accommodation efficiency.

[0144] [6] In the main control device 3 described in any one of [1] to [5], when the target work W and the existing work W0 are lined up, the top surface of the target work W and the top surface of the existing work W0 are exposed, and the heights of the top surfaces of the target work W and the existing work W0 are aligned, the evaluator 48 calculates the evaluation value by treating the top surface of the target work W and the top surface of the existing work W0 as a single surface.

[0145] According to this configuration, the top surfaces of the target work W and the existing work W0, which are aligned at the same height, are treated as a single surface, which contributes to improving the storage efficiency of the virtual work Wi. Placing the target work W so that its height is aligned with the existing work W0 tends to result in a higher evaluation value. In other words, the placement target position is determined taking into consideration how much of the virtual work Wi can be placed on the top surface of the target work W after it has been placed. As a result, not only the planar storage efficiency of the virtual work Wi in the available space but also the three-dimensional storage efficiency is considered in more detail.

[0146] [7] In the main control device 3 described in any one of [1] to [6], the searcher 47 searches for the position candidate corresponding to a corner included in the storage space.

[0147] This configuration reduces the amount of calculation required for the evaluation value. Specifically, when storing the target work W in a storage space, it is preferable to place the target work W tightly in a space that includes a corner, from the perspective of work storage efficiency. Ultimately, the storage efficiency of the virtual work Wi in an empty space is often high when the target work W is placed in a space that includes a corner. By narrowing down the position candidates for the placement target position to those that correspond to corners included in the storage space in advance, it is possible to omit the calculation of the evaluation value for less likely position candidates. As a result, the amount of calculation required for the evaluation value can be reduced.

[0148] [8] The robot system 100 comprises a robot 1 having a robot arm 12 and a hand 14 attached to the robot arm 12, and a main control device 3 described in any one of [1] to [7].

[0149] According to this configuration, the target placement position is determined based on the space available after the target work W has been placed. This allows the target placement position to be determined taking into consideration the placement of subsequent workpieces after the target work W has been placed. As a result, it is possible to realize appropriate placement of not only the current target work W, but also multiple workpieces including subsequent workpieces.

[0150] [9] A control method for a robot 1 having a robot arm 12 and a hand 14 attached to the robot arm 12, the method including: searching for a plurality of position candidates that are candidates for a target placement position of a target work W in a storage space; calculating an evaluation value for each of the plurality of position candidates; and determining the target placement position from among the plurality of position candidates based on the evaluation value; the calculation of the evaluation value includes calculating the evaluation value based on the vacant space in the storage space after the target work W has been placed at each of the position candidates.

[0151] According to this configuration, the target placement position is determined based on the space available after the target work W has been placed. This allows the target placement position to be determined taking into consideration the placement of subsequent workpieces after the target work W has been placed. As a result, it is possible to realize appropriate placement of not only the current target work W, but also multiple workpieces including subsequent workpieces.

[0152] 100 Robot system 1 Robot 12 Robot arm 14 Hand 3 Main control device (control device) 47 Searcher 48 Evaluator 49 Decider W Target work

Claims

1. A control device comprising: a searcher that searches for multiple position candidates that are candidates for a target placement position of a target work in an accommodation space; an evaluator that calculates an evaluation value for each of the multiple position candidates; and a determiner that determines the target placement position from among the multiple position candidates based on the evaluation value, wherein the evaluator calculates the evaluation value based on the available space in the accommodation space after the target work has been placed at each of the position candidates.

2. A control device according to claim 1, wherein the evaluation value is a parameter indicating the efficiency with which virtual work can be accommodated in the free space.

3. A control device according to claim 1, wherein the evaluation value is a parameter related to the number of virtual works that can be placed in the free space or the number of candidate positions at which virtual works can be placed in the free space.

4. A control device according to claim 3, wherein the virtual work includes a plurality of types of virtual work of different sizes, each of the plurality of types of virtual work is assigned a different score, and the evaluator calculates the evaluation value in accordance with the number of accommodations or the number of candidates and the score.

5. A control device as described in claim 1, wherein the evaluator calculates the evaluation value by considering the space above the exposed portion of the bottom surface of the storage space and the space above the exposed portion of the top surface of an existing workpiece in the storage space as the free space.

6. A control device according to claim 5, wherein the evaluator calculates the evaluation value by treating the top surface of the target workpiece and the top surface of the existing workpiece as a single surface when the target workpiece and the existing workpiece are lined up, the top surface of the target workpiece and the top surface of the existing workpiece are exposed, and the heights of the top surface of the target workpiece and the top surface of the existing workpiece are aligned.

7. A control device according to claim 1, wherein the searcher searches for the position candidate corresponding to a corner included in the storage space.

8. A robot system comprising: a robot having a robot arm and a hand attached to the robot arm; and a control device according to any one of claims 1 to 7.

9. A method for controlling a robot having a robot arm and a hand attached to the robot arm, comprising: searching for a plurality of position candidates which are candidates for a target position for placing a target workpiece in an accommodation space; calculating an evaluation value for each of the plurality of position candidates; and determining the target position for placing from among the plurality of position candidates based on the evaluation value, wherein calculating the evaluation value is based on the available space in the accommodation space after the target workpiece has been placed in each of the position candidates.

Citation Information

Patent Citations

  • Intelligent stacking method based on digital twinning technology

    CN113183156A

  • Robotic system for palletizing packages by using real time placement simulation

    JP2020196611A