Control support system, control support method, and control support program

Abstract

To determine a work-piece holding position that contributes an efficient operation of a robot.SOLUTION: A control support system comprises: a simulation part that, with respect to each of two or more candidate positions which are two or more candidates of a holding position on a work-piece held by an end effector of a robot, virtually executes pick processing such that the end effector holds the work-piece at the candidate position, and post processing for the work-piece held at the candidate position by simulation based a work model indicating a shape of the work-piece and a robot model indicating the robot having the end effector; and a determination part that determines one of at least one candidate position where the pick processing and the post processing can be completed in the simulation as the holding position.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] One aspect of the present disclosure relates to a control support system, a control support method, and a control support program.

Background Art

[0002] Citation Document 1 describes a robot system for picking up one object from a group of objects using a robot. This system includes a camera that provides an image of the object, a deep learning neutral network that generates a segmented image of the object, means for identifying a location for picking up the object using the segmented image, and means for rotating the object using the orientation of the object in the segmented image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] A mechanism that can determine a holding position of a workpiece that contributes to efficient operation of a robot is desired.

Means for Solving the Problems

[0005] A control support system relating to one aspect of this disclosure includes a simulation unit that virtually executes, for each of two or more candidate positions which are two or more candidates for a holding position on a workpiece held by the robot's end effector, a picking process in which the end effector holds the workpiece at the candidate position and post-processing for the workpiece held at the candidate position, by simulation based on a workpiece model showing the shape of the workpiece and a robot model showing the robot having the end effector, and a determination unit that determines one of at least one candidate position in which the picking process and post-processing can be completed in the simulation as the holding position.

[0006] A control support method relating to one aspect of the present disclosure is a control support method performed by a control support system comprising at least one processor, and includes the steps of: for each of two or more candidate positions which are two or more candidates for a holding position on a workpiece held by the end effector of a robot, virtually performing a picking process in which the end effector holds the workpiece at the candidate position and post-processing for the workpiece held at the candidate position by a simulation based on a workpiece model showing the shape of the workpiece and a robot model showing the robot having the end effector; and determining one of the at least one candidate position in which the picking process and post-processing can be completed in the simulation as the holding position.

[0007] A control support program relating to one aspect of this disclosure causes a computer to perform the following steps for each of two or more candidate positions which are two or more candidates for a holding position on a workpiece held by the robot's end effector: a picking process in which the end effector holds the workpiece at the candidate position and post-processing for the workpiece held at the candidate position, by simulation based on a workpiece model showing the shape of the workpiece and a robot model showing the robot having the end effector; and a step of determining one of the at least one candidate positions in which the picking process and post-processing can be completed in the simulation as the holding position. [Effects of the Invention]

[0008] According to one aspect of this disclosure, it is possible to determine the workpiece holding position that contributes to the efficient operation of the robot. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example of the application of a control support system. [Figure 2] This figure shows an example of the functional configuration of a control support system. [Figure 3] This figure shows an example of a computer hardware configuration used for a control support system. [Figure 4] The image below shows an example of the process performed by the control support system. [Figure 5] This flowchart provides a detailed example of setting candidate locations. [Figure 6] This figure shows an example of a hypothetical candidate position. [Figure 7] This figure shows an example of how to calculate the degree of contact. [Figure 8] This figure shows an example of a series of processes for determining the holding position. [Modes for carrying out the invention]

[0010] The following describes various examples in this disclosure in detail with reference to the attached drawings. In the description of the drawings, identical or equivalent elements are denoted by the same reference numeral, and redundant descriptions are omitted.

[0011] [System Overview] The control support system described herein is a computer system for determining the holding position on a workpiece held by the robot's end effector. The control support system sets two or more candidate positions that are candidates for the holding position. For each of the two or more candidate positions, the control support system virtually executes a process involving the robot holding the workpiece at that candidate position through simulation. Based on the results of the simulation, the control support system determines one of the two or more candidate positions as the holding position. In this disclosure, the robot holding the workpiece with the end effector is also simply referred to as "the robot holding the workpiece." Holding may be achieved by various methods such as gripping and suction.

[0012] Traditionally, the holding position of a workpiece is determined manually. Determining the holding position requires considering various factors, such as the stability of the workpiece while it is being held and the processing to be performed afterward. Regarding post-processing, constraints must be met, such as the robot being able to operate for the processing and not interfering with obstacles. In addition, the holding position must be determined for each type of workpiece. Even for a single workpiece, the holding position can change depending on various factors, including post-processing and the actual surrounding environment in which the workpiece is processed. Because so many factors must be considered, determining the holding position is a difficult task.

[0013] The control support system determines the holding position considering not only the holding of the workpiece but also the processing after holding. By adopting a holding position that takes post-holding processing into account, it becomes unnecessary to change the holding position between the time the workpiece is held and the completion of subsequent processing. In other words, the control support system can determine a workpiece holding position that contributes to the efficient operation of the robot.

[0014] In one example, the control support system controls a robot placed in a real workspace to hold a workpiece at a determined holding position. For example, the control support system generates an action program based on the holding position and controls the real robot based on that action program. In this way, the control support system may generate a series of robot actions involving workpiece holding. By using the control support system, a series of robot actions including a picking process and subsequent post-processing can be easily generated.

[0015] [System Configuration] Figure 1 shows an example of the application of a control support system. The control support system 1 shown in this example determines the holding position of a real workpiece held by a real robot in a real workspace 9. In the example in Figure 1, a first robot 21 and a second robot 22 are positioned in the workspace 9, and these robots cooperate to fix a first workpiece 81 on a second workpiece 82. The second workpiece 82 is a different workpiece from the first workpiece 81. Hereafter, it is assumed that the first robot 21 holds the first workpiece 81. Therefore, the control support system 1 determines the holding position on the first workpiece 81 held by the first robot. The second robot 22 is an example of a separate device different from the first robot 21. A separate device such as the second robot 22 contributes to the processing of the workpiece together with the robot that holds the workpiece (e.g., the first robot 21). The control support system 1 is connected via a communication network to at least a robot controller 3 that controls the first robot 21. The communication network may be a wired network or a wireless network. The communication network may consist of at least one of the internet and an intranet. Alternatively, the communication network could simply be implemented using a single communication cable.

[0016] In the example of FIG. 1, the second workpiece 82 is an H-beam, and the first workpiece 81 is a substantially T-shaped steel material (T-shaped steel) welded to the H-beam. The first robot 21 fits the T-shaped steel material onto the H-beam by pick-and-place processing for the T-shaped steel material and continues to hold the T-shaped steel material until welding is completed. The second robot 22 welds the T-shaped steel material fitted onto the H-beam to fix the T-shaped steel material to the H-beam. At least one of the first robot 21 and the second robot 22 may move on a rail provided to extend along the H-beam. Alternatively, at least one of the first robot 21 and the second robot 22 may be an autonomous mobile robot (AMR) or may be supported by an automated guided vehicle (AGV). The first robot 21 and the second robot 22 may operate in cooperation with a positioner that holds the workpiece.

[0017] The first robot 21 is a device that receives power and performs a predetermined operation according to the purpose to execute useful work. In one example, the first robot 21 includes a plurality of joints, an arm, and a first end effector 21a attached to the tip of the arm. The first robot 21 holds the workpiece using the first end effector 21a. Examples of the first end effector 21a include a gripper, a suction hand, a magnetic hand, and the like. A joint axis is set for each of the plurality of joints. Some components of the first robot 21, such as the arm and the swivel part, rotate about the joint axis. As a result, the first robot 21 can change the position and orientation of the first end effector 21a within a predetermined range. In one example, the first robot 21 is a multi-axis serial link type vertical articulated robot. The first robot 21 may be a 6-axis vertical articulated robot or a 7-axis vertical articulated robot with one redundant axis added to the 6 axes. As described above, the first robot 21 may be a self-propelled mobile robot, for example, an autonomous mobile robot (AMR) or a robot supported by an automated guided vehicle (AGV). Alternatively, the first robot 21 may be a stationary robot fixed at a predetermined location.

[0018] In one example, the second robot 22 has the same configuration as the first robot 21. The second end effector 22a of the second robot 22 has a function for fixing the first workpiece 81 to the second workpiece 82. Examples of the second end effector 22a include a welding gun, a screw tightening device, and the like.

[0019] The robot controller 3 is a device that controls at least the first robot 21 according to a pre-generated operation program. In one example, the robot controller 3 calculates a joint angle target value (the angle target value of each joint of the first robot 21) for matching the position and orientation of the end effector with the target value indicated in the operation program, and controls the first robot 21 according to the angle target value.

[0020] FIG. 2 is a diagram showing an example of the functional configuration of the control support system 1. In this example, the control support system 1 includes, as functional components, a model acquisition unit 11, a candidate setting unit 12, a simulation unit 13, a positioning unit 14, and a robot control unit 15. The model acquisition unit 11 is a functional module that acquires model data used for simulation. The candidate setting unit 12 is a functional module that sets two or more candidate positions that are candidates for the holding position of the first workpiece 81. The simulation unit 13 is a functional module that virtually executes, by simulation, the processes executed by the first robot 21 and the second robot 22 based on the candidate positions for each of the two or more candidate positions. The simulation is a process of virtually representing the operations of these robots on a computer instead of actually operating the first robot 21 and the second robot 22 arranged in the work space 9. The positioning unit 14 is a functional module that determines, based on the result of the simulation, one of the two or more candidate positions as the holding position. The robot control unit 15 is a functional module that controls at least the first robot 21 based on the holding position.

[0021] The control support system 1 can be implemented using any type of computer. This computer may be a general-purpose computer such as a personal computer or a business server, or it may be incorporated into a dedicated device that performs specific processing.

[0022] Figure 3 shows an example of the hardware configuration of a computer 100 used for the control support system 1. In this example, the computer 100 comprises a main unit 110, a monitor 120, and an input device 130.

[0023] The main unit 110 is a device having a circuit 160. The circuit 160 has a processor 161, memory 162, storage 163, input / output port 164, and communication port 165. The number of each hardware component may be one or two or more. The storage 163 stores programs for configuring each functional module of the main unit 110. The storage 163 is a computer-readable recording medium such as a hard disk, non-volatile semiconductor memory, magnetic disk, or optical disk. The memory 162 temporarily stores programs loaded from the storage 163, calculation results from the processor 161, etc. The processor 161 configures each functional module by executing programs in cooperation with the memory 162. The input / output port 164 inputs and outputs electrical signals to and from the monitor 120 or input device 130 in response to commands from the processor 161. The communication port 165 communicates data with other devices such as the robot controller 3 via the communication network N in response to commands from the processor 161.

[0024] The monitor 120 is a device for displaying information output from the main unit 110. For example, the monitor 120 is a device capable of displaying graphics, such as an LCD panel.

[0025] The input device 130 is a device for inputting information into the main unit 110. Examples of input devices 130 include operation interfaces such as keypads, mice, and operation controllers.

[0026] The monitor 120 and the input device 130 may be integrated as a touch panel. For example, the main unit 110, the monitor 120, and the input device 130 may be integrated as a single unit, similar to a tablet computer.

[0027] Each functional module of the control support system 1 is implemented by loading a control support program onto the processor 161 or memory 162 and having the processor 161 execute that program. The control support program contains code for implementing each functional module of the control support system 1. The processor 161 operates the input / output ports 164 and communication ports 165 according to the control support program and performs data reading and writing to the memory 162 or storage 163.

[0028] The control support program may be provided on a non-temporary recording medium such as a CD-ROM, DVD-ROM, or semiconductor memory. Alternatively, the control support program may be provided via a communication network as a data signal superimposed on a carrier wave.

[0029] [Control support method] As an example of the control support method relating to this disclosure, an example of processing performed by the control support system 1 will be described with reference to Figure 4. Figure 4 is a flowchart showing this example as processing flow S1. That is, the control support system 1 executes processing flow S1.

[0030] In step S11, the model acquisition unit 11 acquires model data. In one example, the model acquisition unit 11 acquires model data including a first workpiece model showing a first workpiece 81 and a first robot model showing a first robot 21 having a first end effector 21a. The model acquisition unit 11 may further acquire model data including a second workpiece model showing a second workpiece 82 and a second robot model showing a second robot having a second end effector 22a. All of these models are represented by electronic data. The second robot model is an example of an apparatus model showing a different apparatus from the first robot 21. Workpiece models, such as the first and second workpiece models, show at least the shape of the workpiece and may further show other attributes of the workpiece, such as dimensions. Robot models, such as the first and second robot models, show specifications for the robot and end effector. These specifications may include a set of parameters relating to the structure of the robot and end effector, such as shape and dimensions, and a set of parameters relating to the function of the robot and end effector, such as the range of motion of each joint and the performance of the end effector.

[0031] In one example, the model acquisition unit 11 acquires model data specified by the user of the control support system 1. The model acquisition unit 11 may read model data corresponding to the user instruction from a predetermined storage device such as the storage 163, or it may accept model data input by the user via the input device 130. In any case, when the user specifies the workpiece and robot, the model acquisition unit 11 acquires model data corresponding to that specification. For example, if the first robot 21 and the second robot 22 can process multiple types of first workpieces 81, the user can have these robots process multiple types of first workpieces 81 while changing the type of first workpiece 81. In this case, when the user specifies the first workpiece 81 to be processed next, the model acquisition unit 11 acquires the first workpiece model of the specified first workpiece 81.

[0032] In step S12, the candidate setting unit 12 sets two or more candidate positions on the first workpiece 81 based on the first work model. This process involves generating a virtual shape of the first workpiece 81 through calculations using the first work model and setting candidate positions on this virtual shape. An example of this process will be explained in detail with reference to Figure 5. Figure 5 is a flowchart showing an example of setting candidate positions.

[0033] In step S121, the candidate setting unit 12 provisionally sets multiple candidate positions on the first workpiece 81. For example, the candidate setting unit 12 sets candidate positions on each face of the first workpiece 81, which has a three-dimensional shape. The candidate setting unit 12 may set the multiple candidate positions randomly, or it may set them according to a regular pattern such as a grid. Figure 6 shows an example of provisionally set candidate positions. In this example, the candidate setting unit 12 randomly sets candidate positions 300 on the first workpiece 81.

[0034] In step S122, the candidate setting unit 12 selects one from a group of candidate positions.

[0035] In step S123, the candidate setting unit 12 calculates the degree of contact between the contact surface of the first workpiece 81 and the first end effector 21a for the selected candidate position. The contact surface refers to the surface of the end effector that may come into contact with the workpiece when the end effector holds the workpiece. The contact surface may be a flat surface, a curved surface, or a more complex shape. The degree of contact is an index that indicates how much of the end effector's contact surface comes into contact with the workpiece. The more of the end effector's contact surface that comes into contact with the workpiece, the higher the degree of contact. The candidate setting unit 12 generates a virtual contact surface based on the first robot model and calculates the degree of contact between the virtual contact surface and the virtual first workpiece 81. The candidate setting unit 12 may generate a virtual contact surface that is faithful to the shape of the actual end effector's contact surface, or it may generate a virtual contact surface in an abstract form of the actual shape.

[0036] Figure 7 shows an example of a method for calculating the degree of contact. In this example, the candidate setting unit 12 positions the center 201 of the contact surface 200 that contacts the first workpiece 81 at the selected candidate position 301. The candidate setting unit 12 then calculates the degree of contact between the contact surface 200 positioned in this way and the first workpiece 81. The candidate setting unit 12 generates a plurality of sample points 210 on the contact surface 200 of the first end effector 21a. The candidate setting unit 12 may generate the plurality of sample points 210 according to a regular pattern such as a grid or radial pattern, or it may generate the plurality of sample points 210 randomly.

[0037] Next, the candidate setting unit 12 brings the contact surface 200 closer to the first workpiece 81 along the normal direction of the contact surface 200, virtually bringing the contact surface 200 into contact with the first workpiece 81. For each of the multiple sample points 210 in this contact state, the candidate setting unit 12 calculates the distance from the sample point 210 to the surface 81a of the first workpiece 81. To measure this distance, the candidate setting unit 12 uses Ray 220, which is a virtual line from each of the multiple sample points 210 from the sample point 210 in the normal direction of the contact surface 200 to the surface 81a of the first workpiece 81. For each of the multiple sample points 210 of the contact surface 200 that is in contact with the first workpiece 81, the candidate setting unit 12 calculates the distance from the sample point to the surface 81a of the first workpiece 81. This distance is the length of Ray 220. The distance at sample point 210 in contact with surface 81a is 0, and the length of sample point 210 where Ray 220 does not reach surface 81a is infinite.

[0038] The candidate setting unit 12 calculates the degree of contact based on the distance of each of the multiple sample points. The candidate setting unit 12 may also calculate the degree of contact based on a statistical value (e.g., the average) of the distances of the multiple sample points 210. The candidate setting unit 12 may also calculate the degree of contact after replacing the infinite distance with a predetermined value. The candidate setting unit 12 calculates the degree of contact using a function in which the degree of contact increases as the statistical value decreases. Alternatively, the candidate setting unit 12 may calculate the degree of contact based on the crossing ratio, which is the proportion of sample points 210 where Ray 220 intersects the surface 81a. The candidate setting unit 12 calculates the degree of contact using a function in which the degree of contact increases as the crossing ratio increases. Since whether or not Ray 220 intersects the surface 81a is determined based on distance, calculating the degree of contact based on the crossing ratio is also an example of calculating the degree of contact based on distance. Alternatively, the candidate setting unit 12 may calculate the degree of contact based on both the statistical value of distance and the crossing ratio. As yet another example, the candidate setting unit 12 may calculate the degree of contact based on at least one of the distance statistics and the crossing ratio, and the distance from the selected candidate position to the center of gravity of the first workpiece 81. For example, the candidate setting unit 12 may further calculate the degree of contact using a function in which the degree of contact increases as the distance to the center of gravity decreases.

[0039] Return to Figure 5. As shown in step S124, the candidate setting unit 12 calculates the degree of contact for each of the multiple candidate positions that have been provisionally set. If there are unprocessed candidate positions (NO in step S124), the process returns to step S122. In the repeated step S122, the candidate setting unit 12 selects the next candidate position. In the repeated step S123, the candidate setting unit 12 calculates the degree of contact between the contact surface of the first workpiece 81 and the first end effector 21a for that candidate position.

[0040] If all candidate positions have been processed (YES in step S124), the process proceeds to step S125. In step S125, the candidate setting unit 12 selects two or more candidate positions from the multiple candidate positions based on the contact degree of each of the multiple candidate positions. The candidate setting unit 12 may select two or more candidate positions whose contact degree is above a predetermined threshold. Alternatively, the candidate setting unit 12 may sort the candidate positions in descending order of contact degree and then select the first n candidate positions (where n > 1). This selection process can be described as a process of narrowing down the provisionally set multiple candidate positions to candidate positions in which the first end effector 21a is expected to be able to reliably hold the first workpiece 81, that is, candidate positions in which stable holding is expected. The candidate setting unit 12 sets the two or more selected candidate positions for simulation.

[0041] As explained with reference to Figure 5, the candidate setting unit 12 provisionally sets multiple candidate positions on the first workpiece 81 based on the first workpiece model. Then, the candidate setting unit 12 selects two or more candidate positions from the multiple candidate positions based on each of the multiple candidate positions and the shape of the contact surface of the first end effector 21a. The candidate setting unit 12 sets the two or more selected candidate positions for simulation.

[0042] Returning to Figure 4, in step S13, the simulation unit 13 selects one of the two or more candidate positions that have been narrowed down.

[0043] In step S14, the simulation unit 13 performs a simulation based on the selected candidate positions. The simulation unit 13 performs the simulation based on at least the first work model and the first robot model. The simulation unit 13 may further perform the simulation based on at least one of the second work model and the second robot model. The simulation unit 13 uses the model data to generate a virtual space corresponding to the real workspace 9 and performs the simulation in that virtual space.

[0044] The simulation unit 13 virtually executes, through simulation, a picking process in which the first robot 21 uses the first end effector 21a to hold the first workpiece 81 at a selected candidate position, and post-processing of the first workpiece 81 held at the candidate position.

[0045] The simulation unit 13 may perform a placing process, in which the first robot 21 places the first workpiece 81 held by the first end effector 21a at a designated position, as at least part of the post-processing. In this case, the simulation unit 13 virtually performs the picking process and the placing process. The combination of the picking process and the placing process is also called the pick-and-place process. The simulation unit 13 may also perform a placing process in which the first robot 21 places the first workpiece 81, held by the first end effector 21a, at a designated position on the second workpiece 82. Alternatively, the simulation unit 13 may perform a placing process in which the first workpiece 81 is placed at a designated position on an object other than the second workpiece 82, such as a workbench or rack.

[0046] The simulation unit 13 may perform a place process and additional processing on the first workpiece 81, which is being held by the first robot 21 at a designated position, as at least part of the post-processing. In this case, the simulation unit 13 virtually performs the pick process, the place process, and the additional processing. The additional processing may be performed by the first robot 21 without using the second robot 22, or it may be a process performed collaboratively by the first robot 21 and the second robot 22. For example, the simulation unit 13 may perform a collaborative process as additional processing in which the second robot 22 works on the first workpiece 81, which is being held by the first robot 21 at a designated position. The simulation unit 13 may also perform a collaborative process (additional processing) in which the second robot 22 fixes the first workpiece 81 onto the second workpiece 82 by welding, screwing, or other methods. The simulation unit 13 may execute a process as a place process in which the first robot 21 positions the first workpiece 81 at a designated position set in the air, and a process as a collaborative process (additional process) in which the second robot 22 receives the first workpiece 81 from the first robot 21 at that designated position. In the collaborative process (additional process), the simulation unit 13 may virtually execute work on multiple work areas on the first workpiece 81 by the second robot 22. A work area refers to a part of the workpiece that is processed by the robot. Each work area may be an area defined by a point, a line, or a surface.

[0047] As described above, the simulation unit 13 can perform various types of post-processing. Post-processing may include placement processing. Alternatively, post-processing may include placement processing and additional processing. Since additional processing may be collaborative processing, post-processing may include placement processing and collaborative processing.

[0048] In any case, the simulation unit 13 virtually performs the picking and post-processing corresponding to the selected candidate positions. In this simulation, the picking and post-processing may or may not be completed. In this disclosure, "being able to complete the picking and post-processing" means that the robot is able to maintain a normal posture (i.e., operate normally) during both the picking and post-processing, no interference is detected, and as a result, the series of operations from the picking to the completion of post-processing is successful. Interference refers to the phenomenon of one object coming into contact with or colliding with another object. Note that when a robot or another device attempts to process a workpiece, contact between the robot or another device and the workpiece is not considered interference. When the other device is operating, the ability of the other device to maintain a normal posture during both the picking and post-processing is also a condition for completing the picking and post-processing.

[0049] The simulation unit 13 stores the simulation results in a predetermined storage device such as memory 162 or storage 163. For example, the simulation unit 13 may store result data that includes pairs of selected candidate positions and flag information indicating whether or not the picking process and post-processing were completed.

[0050] As shown in step S15, the simulation unit 13 performs a simulation for each of the two or more narrowed-down candidate positions. If there are any unprocessed candidate positions (NO in step S15), the process returns to step S13. In the repeated step S13, the simulation unit 13 selects the next candidate position. In the repeated step S14, the simulation unit 13 virtually performs the picking process and post-processing for that candidate position through simulation.

[0051] If all of the two or more candidate positions have been processed (YES in step S15), the process proceeds to step S16. In step S16, the position determination unit 14 determines one of the at least one candidate positions that can complete the picking and post-processing in the simulation as a retained position. That is, the position determination unit 14 determines one of the at least one candidate positions that succeeded in the simulation as a retained position.

[0052] The position determination unit 14 refers to the simulation result data to identify at least one candidate position in which the picking and post-processing can be completed. For example, the position determination unit 14 may identify at least one candidate position in which the first robot 21 operates normally and no interference is detected during both the picking and post-processing. Alternatively, the position determination unit 14 may identify at least one candidate position in which both the first robot 21 and the second robot 22 operate normally and no interference is detected for either of these two robots during both the picking and post-processing.

[0053] The position determination unit 14 may determine a candidate position selected by the user of the control support system 1 as the retained position. In one example, the position determination unit 14 displays at least one identified candidate position on the user's display device (e.g., monitor 120). The position determination unit 14 then determines one candidate position selected by the user from the at least one candidate position as the retained position.

[0054] Alternatively, the position determination unit 14 may automatically determine one of the identified candidate positions as the holding position. For example, the position determination unit 14 may determine the candidate position with the highest degree of contact among the at least one candidate position as the holding position.

[0055] Figure 8 shows an example of a series of processes for determining a holding position, with reference to a hypothetical first workpiece 81. As shown in state ST1, the candidate setting unit 12 sets a plurality of candidate positions 300 on the first workpiece 81 (step S121). The candidate setting unit 12 calculates the degree of contact for each of the plurality of candidate positions 300 (steps S122 to S124). As shown in state ST2, the magnitude of the degree of contact can be visualized by the intensity of the color of the bar drawn at each candidate position. In this example, the color becomes darker as the degree of contact increases. As shown in state ST3, the candidate setting unit 12 selects two or more candidate positions 300 from the plurality of candidate positions 300 based on the degree of contact (step S125). In this example, the candidate setting unit 12 selects candidate positions 300 whose degree of contact is greater than a predetermined threshold. As shown in state ST4, the simulation unit 13 performs a simulation based on each selected candidate position, and the position determination unit 14 identifies at least one candidate position 300 that can complete the picking and post-processing based on the results of the simulation (steps S13 to S16). As shown in state ST5, the position determination unit 14 determines one of the identified at least one candidate position 300 as the holding position 320, either based on the user's selection or automatically (step S16).

[0056] Returning to Figure 4, in step S17, the robot control unit 15 controls the actual robot based on the holding position.

[0057] The robot control unit 15 generates an action program for controlling the actual robot based on the holding position. The robot control unit 15 generates a first action program to cause the first robot 21 to perform a picking process to hold the first workpiece 81 at the holding position and post-processing on the first workpiece 81 held at that holding position. In one example, the robot control unit 15 may generate the first action program based on the simulation results corresponding to the holding position.

[0058] The motion program includes data for controlling the robot, including, for example, a path indicating the robot's trajectory. The robot's trajectory refers to the path of movement of the robot or its components. For example, the robot's trajectory may be the trajectory of the end-effector or end-effector. The first motion program includes at least code to cause the actual first robot 21 to hold the first workpiece 81 in a determined holding position. The first motion program may further include code to cause the actual first robot 21 to perform at least a portion of post-processing.

[0059] The robot control unit 15 may generate a second motion program for controlling the actual second robot 22. The second motion program may further include code to cause the actual second robot 22 to perform at least part of the post-processing.

[0060] The robot control unit 15 controls the actual robot based on the motion program. The robot control unit 15 controls the first robot 21, which is positioned in the workspace 9, to perform an actual pick operation in which the first workpiece 81 present in the workspace 9 is held in a holding position by the first end effector 21a. The robot control unit 15 outputs a first motion program to the robot controller 3, causing the robot controller 3 to control the first robot 21. The robot controller 3 operates the first robot 21 based on the first motion program. The robot control unit 15 may also output a second motion program to the robot controller 3, causing the robot controller 3 to control the second robot 22. The robot controller 3 may operate the second robot 22 based on the second motion program.

[0061] [Differentiation] The technology relating to this disclosure has been described in detail above based on various examples. However, this disclosure is not limited to the examples given above. The technology relating to this disclosure can be modified in various ways without departing from its essence.

[0062] The control support system may acquire a work model that shows a workpiece with multiple pre-set candidate positions. Alternatively, the control support system may output the holding position to another computer system, such as a robot control system, and the other computer system may control the actual robot based on that holding position. In other words, the control support system does not need to include a functional module corresponding to at least one of the candidate setting unit 12 and the robot control unit 15.

[0063] In the example above, the other device is the second robot 22, but the other device may be something other than a robot, such as a conveyor or an automatic rack.

[0064] The control support system may virtually perform picking and post-processing in an environment where no other device such as the second robot 22 exists, and determine the holding position. For example, post-processing performed by a robot such as the first robot 21 alone may be simulated.

[0065] The system's hardware configuration is not limited to a configuration in which each functional module is realized by program execution. For example, at least a portion of the above-mentioned group of functional modules may be composed of logic circuits specialized for that function, or they may be composed of an ASIC (Application Specific Integrated Circuit) that integrates such logic circuits.

[0066] The processing steps for a method executed by at least one processor are not limited to the examples above. For example, some of the steps or processes described above may be omitted, or each step may be performed in a different order. Also, any two or more of the steps described above may be combined, or some of the steps may be modified or deleted. Alternatively, other steps may be performed in addition to each of the steps described above.

[0067] When comparing the relative magnitudes of two numbers in a computer system or within a computer, either the two criteria "greater than or equal to" and "greater than" may be used, or either the two criteria "less than or equal to" and "less than" may be used.

[0068] [Note] As can be seen from the various examples above, this disclosure includes the following aspects: (Note 1) A simulation unit virtually executes, for each of two or more candidate positions on a workpiece that is held by the robot's end effector, a picking process in which the end effector holds the workpiece at the candidate position, and post-processing for the workpiece held at the candidate position, based on a workpiece model showing the shape of the workpiece and a robot model showing the robot having the end effector. A determination unit that determines one of the at least one candidate position in which the picking process and the post-processing can be completed in the simulation as the holding position, A control support system equipped with the following features. (Note 2) The post-processing includes a placing process in which the robot places the held workpiece at a designated position, and additional processing on the workpiece while it is being held by the robot at the designated position. The simulation unit virtually executes the picking process, the placing process, and the addition process. The control support system described in Appendix 1. (Note 3) The simulation unit, based on the simulation which is further based on a device model representing a separate device different from the robot, virtually executes as a post-processing step a process that the robot and the separate device perform in cooperation with the workpiece held at the candidate position. The determination unit identifies at least one candidate position that allows the picking process and post-processing to be completed without detecting interference between the robot and the other device. The control support system described in Appendix 1 or 2. (Note 4) The aforementioned separate device is a separate robot, The simulation unit virtually executes the post-processing, which includes a placement process in which the robot places the held workpiece at a designated position, and a collaborative process in which another robot works on the workpiece that is being held by the robot at the designated position. The control support system described in Appendix 3. (Note 5) The simulation unit, in the collaborative processing, virtually executes operations on multiple work areas on the workpiece by the other robot. The control support system described in Appendix 4. (Note 6) The aforementioned simulation unit, The process by which the robot places the held workpiece at the specified position on another workpiece is virtually executed as the place process. The process by which the other robot fixes the workpiece onto the other workpiece is virtually executed as the collaborative process. A control support system as described in Appendix 4 or 5. (Note 7) A control support system according to any one of appendices 1 to 6, further comprising a robot control unit that controls a robot positioned in a real workspace to perform a real picking operation in which the workpiece present in the real workspace is held in the holding position by the end effector. (Note 8) When the workpiece held by the end effector is specified, an acquisition unit acquires the workpiece model of the specified workpiece, A setting unit that sets the two or more candidate positions based on the acquired work model, A control support system described in any one of the appendices 1 to 7, further comprising the features described therein. (Note 9) The setting unit is, Based on the work model, a plurality of candidate positions are tentatively set on the work, Based on each of the plurality of candidate positions and the shape of the contact surface of the end effector that can contact the workpiece, two or more candidate positions are selected from the plurality of candidate positions. The two or more selected candidate positions are set for the simulation. The control support system described in Appendix 8. (Note 10) The setting unit is, For each of the above-mentioned candidate positions, the degree of contact between the contact surface and the workpiece is calculated when the center of the contact surface that contacts the workpiece is positioned at the candidate position. Based on the degree of contact of each of the plurality of candidate positions, two or more candidate positions are selected from the plurality of candidate positions. The control support system described in Appendix 9. (Note 11) The setting unit, for each of the plurality of candidate positions, Multiple sample points are generated on the aforementioned contact surface, For each of the aforementioned sample points, the distance from the sample point to the surface of the workpiece is calculated. The degree of contact is calculated based on the distance between each of the plurality of sample points. The control support system described in Appendix 10. (Note 12) The aforementioned determination unit, In the simulation, at least one candidate position in which the post-processing can be completed is displayed on the user's display device. One candidate position selected by the user from the aforementioned at least one candidate position is determined to be the holding position. A control support system described in any one of the appendices 1 to 11. (Note 13) A control support method performed by a control support system comprising at least one processor, For each of two or more candidate positions, which are two or more candidate positions for holding a workpiece on a workpiece held by the robot's end effector, the steps of virtually executing a picking process in which the end effector holds the workpiece at the candidate position and post-processing for the workpiece held at the candidate position are performed by a simulation based on a workpiece model showing the shape of the workpiece and a robot model showing the robot having the end effector, The steps include determining one of the at least one candidate position in which the picking process and the post-processing can be completed in the simulation as the holding position, A control support method including (Note 14) For each of two or more candidate positions, which are two or more candidate positions for holding a workpiece on a workpiece held by the robot's end effector, the steps of virtually executing a picking process in which the end effector holds the workpiece at the candidate position and post-processing for the workpiece held at the candidate position are performed by a simulation based on a workpiece model showing the shape of the workpiece and a robot model showing the robot having the end effector, The steps include determining one of the at least one candidate position in which the picking process and the post-processing can be completed in the simulation as the holding position, A control support program that instructs a computer to perform certain actions.

[0069] According to appendices 1, 13, and 14, for each of the two or more candidate positions, not only the picking process but also subsequent processing is virtually executed, and the candidate position that can complete both processes is determined as the holding position. Since a holding position that is valid not only for the picking process but also for subsequent processing is determined, there is no need to change the holding position between these two processes. In other words, this mechanism makes it possible to determine the workpiece holding position that contributes to the efficient operation of the robot.

[0070] According to Appendix 2, for each of the two or more candidate positions, not only the pick-and-place process but also subsequent processing is virtually executed, and the candidate position that can complete both processes is determined as the holding position. Since a holding position is determined that is valid not only for the process of placing the workpiece at the specified position but also for subsequent processing, there is no need to change the holding position during this series of processes. In other words, this mechanism makes it possible to determine the workpiece holding position that contributes to the efficient operation of the robot.

[0071] According to Appendix 3, the simulation is performed including the operation of other devices that cooperate with the robot performing the picking process, and the holding position in which the picking and post-processing can be completed is ultimately determined. This mechanism makes it possible to determine the position in which the workpiece should be held when processing is carried out by multiple devices.

[0072] Processing by multiple robots can be complex, making it difficult to manually determine the holding position in this process. According to Appendix 4, a simulation is performed that includes the movements of the robot performing the pick-and-place process and another robot collaborating with it, ultimately determining the holding position that allows for the completion of both the pick-and-place and collaborative processes. This mechanism enables the realization of a series of complex tasks performed by multiple robots, and allows for the determination of workpiece holding positions that contribute to the efficient operation of the robots in these complex tasks.

[0073] In scenarios where multiple robots process multiple work areas on a workpiece, more factors need to be considered than when processing a single robot or a single work area. According to Appendix 5, the holding position for multiple robots to collaboratively process multiple work areas on the workpiece is ultimately determined. Therefore, even in such complex scenarios, the workpiece holding position that contributes to the efficient operation of the robots can be determined.

[0074] According to Appendix 6, even in complex scenarios where multiple robots collaborate to secure one workpiece to another, the optimal workpiece holding position can be determined to contribute to the efficient operation of the robots.

[0075] According to Appendix 7, the end effector makes it easy to determine the position where the workpiece should be held, thereby reducing the effort required to control the actual robot more efficiently.

[0076] According to Appendix 8, when a workpiece is specified, a workpiece model of that workpiece is acquired, and two or more candidate positions are set based on that workpiece model. This mechanism allows the robot to determine an efficient holding position for each workpiece, even when the robot is processing various workpieces.

[0077] According to Appendix 9, several candidate positions are initially set, and then the number of these candidate positions is narrowed down based on the shape of the contact surface of the end effector. The simulation is then performed using the remaining candidate positions. By narrowing down the number of candidate positions simulated, the total execution time of the simulation can be reduced, and therefore the holding position can be determined more quickly.

[0078] According to Appendix 10, since the degree of contact between the end effector's contact surface and the workpiece is taken into consideration, candidate positions can be selected where the end effector is expected to reliably hold the workpiece, that is, candidate positions where the picking process is expected to be reliably performed. Therefore, simulations for two or more candidate positions can be efficiently performed, and the holding position can be determined more quickly.

[0079] According to Appendix 11, the degree of contact between the contact surface and the workpiece can be calculated more accurately by considering the distance from each sample point on the contact surface that is in contact with the workpiece to the surface of the workpiece.

[0080] According to Appendix 12, by providing a mechanism that entrusts the final determination of the holding position to the user, the holding position can be determined in a way that reflects the user's knowledge and experience. Since only candidate positions that can reliably hold the workpiece and perform post-processing are presented to the user, the effort required of the user to determine the holding position can be reduced. [Explanation of Symbols]

[0081] 1...Control support system, 3...Robot controller, 9...Workspace, 11...Model acquisition unit, 12...Candidate setting unit, 13...Simulation unit, 14...Position determination unit, 15...Robot control unit, 21...First robot, 21a...First end effector, 22...Second robot (separate device), 22a...Second end effector, 81...First workpiece, 82...Second workpiece (separate workpiece), 200...Contact surface, 210...Sample point, 300...Candidate position, 320...Holding position.

Claims

1. A setting unit that sets two or more candidate positions which are two or more candidates for holding positions on a workpiece, based on a workpiece model that shows the shape of a workpiece held by the robot's end effector, For each of the two or more candidate positions set, a simulation unit virtually executes a picking process in which the end effector holds the workpiece at the candidate position, and post-processing for the workpiece held at the candidate position, based on a workpiece model and a robot model representing the robot having the end effector. A determination unit that determines one of the at least one candidate position in which the picking process and the post-processing can be completed in the simulation as the holding position, A control support system equipped with the following features.

2. The setting unit is, Based on the work model, a plurality of candidate positions are tentatively set on the work, Based on each of the plurality of candidate positions and the shape of the contact surface of the end effector that can contact the workpiece, two or more candidate positions are selected from the plurality of candidate positions. The two or more selected candidate positions are set for the simulation. The control support system according to claim 1.

3. The setting unit is, For each of the above-mentioned candidate positions, the degree of contact between the contact surface and the workpiece is calculated when the center of the contact surface that contacts the workpiece is positioned at the candidate position. Based on the degree of contact of each of the plurality of candidate positions, two or more candidate positions are selected from the plurality of candidate positions. The control support system according to claim 2.

4. The setting unit, for each of the plurality of candidate positions, Multiple sample points are generated on the aforementioned contact surface, For each of the aforementioned sample points, the distance from the sample point to the surface of the workpiece is calculated. The degree of contact is calculated based on the distance between each of the plurality of sample points. The control support system according to claim 3.

5. The post-processing includes a placing process in which the robot places the held workpiece at a designated position, and additional processing on the workpiece while it is being held by the robot at the designated position. The simulation unit virtually executes the picking process, the placing process, and the addition process. A control support system according to any one of claims 1 to 4.

6. The simulation unit, based on the simulation which is further based on a device model representing a separate device different from the robot, virtually executes as a post-processing step a process that the robot and the separate device perform in cooperation with the workpiece held at the candidate position. The determination unit identifies at least one candidate position that allows the picking process and post-processing to be completed without detecting interference between the robot and the other device. A control support system according to any one of claims 1 to 4.

7. The aforementioned separate device is a separate robot, The simulation unit virtually executes the post-processing, which includes a placement process in which the robot places the held workpiece at a designated position, and a collaborative process in which another robot works on the workpiece that is being held by the robot at the designated position. The control support system according to claim 6.

8. The simulation unit, in the collaborative processing, virtually executes operations on multiple work areas on the workpiece by the other robot. The control support system according to claim 7.

9. The aforementioned simulation unit, The process by which the robot places the held workpiece at the specified position on another workpiece is virtually executed as the place process. The process by which the other robot fixes the workpiece onto the other workpiece is virtually executed as the collaborative process. The control support system according to claim 8.

10. The control support system according to any one of claims 1 to 4, further comprising a robot control unit that controls a robot positioned in a real workspace to perform a real picking operation in which the workpiece present in the real workspace is held in the holding position by the end effector.

11. The system further includes an acquisition unit that acquires the work model of the workpiece held by the end effector when the workpiece held by the end effector is specified, The setting unit sets the two or more candidate positions based on the acquired work model. A control support system according to any one of claims 1 to 4.

12. The aforementioned determination unit, In the simulation, at least one candidate position in which the post-processing can be completed is displayed on the user's display device. One candidate position selected by the user from the aforementioned at least one candidate position is determined to be the holding position. A control support system according to any one of claims 1 to 4.

13. A control support method performed by a control support system comprising at least one processor, The steps include setting two or more candidate positions, which are two or more candidates for the holding position on the workpiece, based on a workpiece model that shows the shape of the workpiece to be held by the robot's end effector, For each of the two or more candidate positions set, the end effector virtually executes a picking process in which it holds the workpiece at the candidate position, and post-processing for the workpiece held at the candidate position, by simulation based on the workpiece model and a robot model showing the robot having the end effector. The steps include determining one of the at least one candidate position in which the picking process and the post-processing can be completed in the simulation as the holding position, A control support method including

14. A step of setting two or more candidate positions which are two or more candidates for a holding position on a workpiece, based on a workpiece model that shows the shape of a workpiece to be held by the end effector of a robot, For each of the two or more candidate positions set, the end effector virtually executes a picking process in which it holds the workpiece at the candidate position, and post-processing for the workpiece held at the candidate position, by simulation based on the workpiece model and a robot model showing the robot having the end effector. The steps include determining one of the at least one candidate position in which the picking process and the post-processing can be completed in the simulation as the holding position, A control support program that instructs a computer to perform certain actions.

Images