Control support system, control support method, and control support program

Abstract

To determine a work-piece holding position that contributes an efficient operation of a robot.SOLUTION: A control support system comprises: a simulation part that, with respect to each of two or more candidate positions which are two or more candidates of a holding position on a work-piece held by an end effector of a robot, virtually executes pick processing such that the end effector holds the work-piece at the candidate position, and post processing for the work-piece held at the candidate position by simulation based a work model indicating a shape of the work-piece and a robot model indicating the robot having the end effector; and a determination part that determines one of at least one candidate position where the pick processing and the post processing can be completed in the simulation as the holding position.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] One aspect of the present disclosure relates to a control support system, a control support method, and a control support program.

Background Art

[0002] Citation Document 1 describes a robot system for picking up one object from a group of objects using a robot. This system includes a camera that provides an image of the object, a deep learning neutral network that generates a segmented image of the object, means for identifying a location for picking up the object using the segmented image, and means for rotating the object using the orientation of the object in the segmented image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] A mechanism that can determine a holding position of a workpiece that contributes to efficient operation of a robot is desired.

Means for Solving the Problems

[0005] A control support system according to an aspect of the present disclosure performs, for each of two or more candidate positions that are two or more candidates for a holding position on a workpiece held by an end effector of a robot, a pick process in which the end effector holds the workpiece at the candidate position, and a post-process for the workpiece held at the candidate position, by a simulation based on a workpiece model showing the shape of the workpiece and a robot model showing the robot having the end effector, and a simulation unit that virtually executes the post-process, and a determination unit that determines, as a holding position, one of at least one candidate position capable of completing the pick process and the post-process in the simulation.

[0006] A control support method according to an aspect of the present disclosure is a control support method executed by a control support system including at least one processor, the method including: for each of two or more candidate positions that are two or more candidates for a holding position on a workpiece held by an end effector of a robot, virtually executing, by a simulation based on a workpiece model showing the shape of the workpiece and a robot model showing the robot having the end effector, a pick process in which the end effector holds the workpiece at the candidate position, and a post-process for the workpiece held at the candidate position; and determining, as a holding position, one of at least one candidate position capable of completing the pick process and the post-process in the simulation.

[0007] A control support program according to an aspect of the present disclosure causes a computer to execute: for each of two or more candidate positions that are two or more candidates for a holding position on a workpiece held by an end effector of a robot, virtually executing, by a simulation based on a workpiece model showing the shape of the workpiece and a robot model showing the robot having the end effector, a pick process in which the end effector holds the workpiece at the candidate position, and a post-process for the workpiece held at the candidate position; and determining, as a holding position, one of at least one candidate position capable of completing the pick process and the post-process in the simulation.

Advantages of the Invention

[0008] According to one aspect of the present disclosure, it is possible to determine a holding position of a workpiece that contributes to efficient operation of a robot.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Mode for Carrying Out the Invention

[0010] Hereinafter, various examples in the present disclosure will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant descriptions are omitted.

[0011] [Overview of the System] The control support system according to the present disclosure is a computer system for determining a holding position on a workpiece held by an end effector of a robot. The control support system sets two or more candidate positions that are candidates for the holding position. Then, for each of the two or more candidate positions, the control support system virtually executes by simulation a process involving holding the workpiece at the candidate position by the robot. The control support system determines, based on the result of the simulation, one of the two or more candidate positions as the holding position. In the present disclosure, the robot holding the workpiece by the end effector is also simply referred to as "the robot holds the workpiece". The holding may be realized by various methods such as gripping and suction.

[0012] Conventionally, the holding position on the workpiece is determined manually. In order to determine the holding position, it is necessary to consider various factors such as the stability of the workpiece while being held and the processes performed after holding. Regarding the processes after holding, it is necessary to satisfy constraints such as the robot being able to operate for the processes and the robot not interfering with obstacles. In addition, the holding position needs to be determined for each individual type of workpiece. For a single workpiece, the holding position can vary depending on various factors such as the processes after holding and the actual surrounding environment in which the workpiece is processed. Since it is necessary to consider various factors in this way, determining the holding position is a difficult task.

[0013] The control support system determines the holding position in consideration of not only the holding of the workpiece but also the processes after holding. By adopting the holding position that also takes into account the processes after holding, it becomes unnecessary to change the holding position from the holding of the workpiece until the subsequent processes are completed. That is, the control support system can determine the holding position of the workpiece that contributes to the efficient operation of the robot.

[0014] In one example, the control support system controls the robot arranged in the actual work space to hold the workpiece at the determined holding position. For example, the control support system generates an operation program based on the holding position and controls the actual robot based on the operation program. In this way, the control support system may generate a series of operations of the robot involving holding the workpiece. By using the control support system, a series of robot operations including a pick process and post-processing following the pick process can be easily generated.

[0015] [Configuration of the System] FIG. 1 is a diagram showing an example of the application of the control support system. The control support system 1 shown in this example determines the holding position of the actual workpiece held by the actual robot in the actual work space 9. In the example of FIG. 1, the first robot 21 and the second robot 22 are arranged in the work space 9, and these robots cooperate to fix the first workpiece 81 on the second workpiece 82. The second workpiece 82 is a workpiece different from the first workpiece 81. Hereinafter, it is assumed that the first robot 21 holds the first workpiece 81. Therefore, the control support system 1 determines the holding position on the first workpiece 81 held by the first robot. The second robot 22 is an example of a different device from the first robot 21. A different device such as the second robot 22 contributes to the processing of the workpiece together with the robot (for example, the first robot 21) holding the workpiece. The control support system 1 is connected to the robot controller 3 that controls at least the first robot 21 via a communication network. The communication network may be a wired network or a wireless network. The communication network may be configured to include at least one of the Internet and the intranet. Alternatively, the communication network may be simply realized by a single communication cable.

[0016] In the example of FIG. 1, the second workpiece 82 is an H-beam, and the first workpiece 81 is a substantially T-shaped steel material (T-shaped steel) welded to the H-beam. The first robot 21 fits the T-shaped steel material onto the H-beam by pick-and-place processing on the T-shaped steel material and continues to hold the T-shaped steel material until welding is completed. The second robot 22 welds the T-shaped steel material fitted to the H-beam to fix the T-shaped steel material to the H-beam. At least one of the first robot 21 and the second robot 22 may move on a rail provided to extend along the H-beam. Alternatively, at least one of the first robot 21 and the second robot 22 may be an autonomous mobile robot (AMR) or may be supported by an automated guided vehicle (AGV). The first robot 21 and the second robot 22 may operate in cooperation with a positioner that holds the workpiece.

[0017] The first robot 21 is a device that receives power and performs a predetermined operation according to the purpose to execute useful work. In one example, the first robot 21 includes a plurality of joints, an arm, and a first end effector 21a attached to the tip of the arm. The first robot 21 holds the workpiece using the first end effector 21a. Examples of the first end effector 21a include a gripper, a suction hand, and a magnetic hand. A joint axis is set for each of the plurality of joints. Some components of the first robot 21, such as the arm and the swivel part, rotate around the joint axis. As a result, the first robot 21 can change the position and posture of the first end effector 21a within a predetermined range. In one example, the first robot 21 is a multi-axis serial-link type vertical articulated robot. The first robot 21 may be a 6-axis vertical articulated robot or a 7-axis vertical articulated robot with one redundant axis added to the 6 axes. As described above, the first robot 21 may be a self-propelled mobile robot, for example, an autonomous mobile robot (AMR) or a robot supported by an automated guided vehicle (AGV). Alternatively, the first robot 21 may be a stationary robot fixed at a predetermined location.

[0018] In one example, the second robot 22 has the same configuration as the first robot 21. The second end effector 22a of the second robot 22 has a function for fixing the first workpiece 81 to the second workpiece 82. Examples of the second end effector 22a include a welding gun, a screw tightening device, and the like.

[0019] The robot controller 3 is a device that controls at least the first robot 21 according to a pre-generated operation program. In one example, the robot controller 3 calculates a joint angle target value (the angle target value of each joint of the first robot 21) for matching the position and orientation of the end effector with the target value indicated in the operation program, and controls the first robot 21 according to the angle target value.

[0020] FIG. 2 is a diagram showing an example of the functional configuration of the control support system 1. In this example, the control support system 1 includes a model acquisition unit 11, a candidate setting unit 12, a simulation unit 13, a positioning unit 14, and a robot control unit 15 as functional components. The model acquisition unit 11 is a functional module that acquires model data used for simulation. The candidate setting unit 12 is a functional module that sets two or more candidate positions that are candidates for the holding position of the first workpiece 81. The simulation unit 13 is a functional module that virtually executes, by simulation, the processes executed by the first robot 21 and the second robot 22 based on the candidate positions for each of the two or more candidate positions. The simulation is a process of virtually expressing the operations of these robots on a computer instead of actually operating the first robot 21 and the second robot 22 arranged in the work space 9. The positioning unit 14 is a functional module that determines, based on the result of the simulation, one of the two or more candidate positions as the holding position. The robot control unit 15 is a functional module that controls at least the first robot 21 based on the holding position.

[0021] The control support system 1 can be implemented by any type of computer. The computer may be a general-purpose computer such as a personal computer or a business server, or may be incorporated into a dedicated device that executes specific processing.

[0022] FIG. 3 is a diagram showing an example of the hardware configuration of the computer 100 used for the control support system 1. In this example, the computer 100 includes a main body 110, a monitor 120, and an input device 130.

[0023] The main body 110 is a device having a circuit 160. The circuit 160 has a processor 161, a memory 162, a storage 163, an input / output port 164, and a communication port 165. The number of each hardware component may be 1 or 2 or more. The storage 163 records programs for configuring each functional module of the main body 110. The storage 163 is a computer-readable recording medium such as a hard disk, a non-volatile semiconductor memory, a magnetic disk, or an optical disk. The memory 162 temporarily stores programs loaded from the storage 163, calculation results of the processor 161, and the like. The processor 161 constitutes each functional module by executing a program in cooperation with the memory 162. The input / output port 164 inputs and outputs electrical signals to and from the monitor 120 or the input device 130 in accordance with a command from the processor 161. The communication port 165 performs data communication with other devices such as the robot controller 3 via the communication network N in accordance with a command from the processor 161.

[0024] The monitor 120 is a device for displaying information output from the main body 110. For example, the monitor 120 is a device capable of graphic display such as a liquid crystal panel.

[0025] The input device 130 is a device for inputting information to the main body 110. Examples of the input device 130 include operation interfaces such as a keypad, a mouse, and an operation controller.

[0026] The monitor 120 and the input device 130 may be integrated as a touch panel. For example, like a tablet computer, the main body 110, the monitor 120, and the input device 130 may be integrated.

[0027] Each functional module of the control support system 1 is realized by causing the processor 161 to load a control support program onto the processor 161 or the memory 162 and execute the program. The control support program includes code for realizing each functional module of the control support system 1. The processor 161 operates the input / output port 164 and the communication port 165 according to the control support program, and reads and writes data in the memory 162 or the storage 163.

[0028] The control support program may be provided after being recorded on a non-transitory recording medium such as a CD-ROM, a DVD-ROM, or a semiconductor memory. Alternatively, the control support program may be provided via a communication network as a data signal superimposed on a carrier wave.

[0029] [Control Support Method] As an example of the control support method according to the present disclosure, an example of the process executed by the control support system 1 will be described with reference to FIG. 4. FIG. 4 is a flowchart showing the example as a process flow S1. That is, the control support system 1 executes the process flow S1.

[0030] In step S11, the model acquisition unit 11 acquires model data. In one example, the model acquisition unit 11 acquires model data including a first work model representing the first work 81 and a first robot model representing the first robot 21 having the first end effector 21a. The model acquisition unit 11 may further acquire model data including a second work model representing the second work 82 and a second robot model representing the second robot having the second end effector 22a. All of these models are represented by electronic data. The second robot model is an example of a device model indicating a device different from the first robot 21. A work model such as the first work model and the second work model may at least indicate the shape of the work and may further indicate other attributes of the work such as dimensions. A robot model such as the first robot model and the second robot model indicates the specifications regarding the robot and the end effector. The specifications may include a group of parameters regarding the structure of the robot and the end effector, such as shape and dimensions, and a group of parameters regarding the functions of the robot and the end effector, such as the movable range of each joint and the performance of the end effector.

[0031] In one example, the model acquisition unit 11 acquires the model data specified by the user of the control support system 1. The model acquisition unit 11 may read the model data corresponding to the user instruction from a predetermined storage device such as the storage 163, or may receive the model data input by the user via the input device 130. In any case, when the work and the robot are specified by the user, the model acquisition unit 11 acquires the model data corresponding to the specification. For example, when the first robot 21 and the second robot 22 can process a plurality of types of the first work 81, the user can cause these robots to process the plurality of types of the first work 81 while changing the type of the first work 81. In this case, when the user specifies the first work 81 to be processed next, the model acquisition unit 11 acquires the first work model of the specified first work 81.

[0032] In step S12, the candidate setting unit 12 sets two or more candidate positions on the first workpiece 81 based on the first work model. This process means generating a virtual shape of the first workpiece 81 by calculation using the first work model and setting candidate positions on this virtual shape. An example of this process will be described in detail with reference to FIG. 5. FIG. 5 is a flowchart showing an example of setting candidate positions.

[0033] In step S121, the candidate setting unit 12 temporarily sets a plurality of candidate positions on the first workpiece 81. For example, the candidate setting unit 12 sets candidate positions on each surface of the first workpiece 81 which is a three-dimensional shape. The candidate setting unit 12 may set a plurality of candidate positions randomly, or may set them according to a regular pattern such as a grid pattern. FIG. 6 is a diagram showing an example of the temporarily set candidate positions. In this example, the candidate setting unit 12 randomly sets a candidate position 300 on the first workpiece 81.

[0034] In step S122, the candidate setting unit 12 selects one from the plurality of candidate positions.

[0035] In step S123, the candidate setting unit 12 calculates the degree of contact between the selected candidate position and the contact surface between the first workpiece 81 and the first end effector 21a. The contact surface refers to the surface of the end effector that may come into contact with the workpiece when the end effector holds the workpiece. The contact surface may be a flat surface, may include a curved surface, or may exhibit a more complex shape. The degree of contact is an index indicating how much of the contact surface of the end effector comes into contact with the workpiece. The higher the degree of contact, the more parts of the contact surface of the end effector come into contact with the workpiece. The candidate setting unit 12 generates a virtual contact surface based on the first robot model and calculates the degree of contact between the virtual contact surface and the virtual first workpiece 81. The candidate setting unit 12 may generate the virtual contact surface so as to be faithful to the shape of the contact surface of the actual end effector, or may generate the virtual contact surface in an abstracted form of the actual shape.

[0036] FIG. 7 is a diagram showing an example of a method for calculating the degree of contact. In this example, the candidate setting unit 12 positions the center 201 of the contact surface 200 that contacts the first workpiece 81 at the selected candidate position 301. Then, the candidate setting unit 12 calculates the degree of contact between the contact surface 200 positioned in this way and the first workpiece 81. The candidate setting unit 12 generates a plurality of sample points 210 on the contact surface 200 of the first end effector 21a. The candidate setting unit 12 may generate a plurality of sample points 210 according to a regular pattern such as a grid pattern or a radial pattern, or may generate a plurality of sample points 210 randomly.

[0037] Subsequently, the candidate setting unit 12 moves the contact surface 200 closer to the first workpiece 81 along the normal direction of the contact surface 200 to virtually contact the contact surface 200 with the first workpiece 81. For each of the plurality of sample points 210 in this contact state, the candidate setting unit 12 calculates the distance from the sample point 210 to the surface 81a of the first workpiece 81. To measure this distance, the candidate setting unit 12 uses, for each of the plurality of sample points 210, a virtual line Ray220 that extends from the sample point 210 in the normal direction of the contact surface 200 to reach the surface 81a of the first workpiece 81. The candidate setting unit 12 calculates the distance from each of the plurality of sample points 210 of the contact surface 200 that has contacted the first workpiece 81 to the surface 81a of the first workpiece 81. This distance is the length of Ray220. The distance at the sample point 210 that is in contact with the surface 81a is 0, and the length of the sample point 210 where Ray220 does not reach the surface 81a is infinite.

[0038] The candidate setting unit 12 calculates the degree of contact based on the distance of each of a plurality of sample points. The candidate setting unit 12 may calculate the degree of contact based on a statistical value (for example, an average) of the distances of the plurality of sample points 210. The candidate setting unit 12 may calculate the degree of contact after replacing a distance that is infinite with a predetermined value. The candidate setting unit 12 calculates the degree of contact using a function in which the smaller the statistical value, the larger the degree of contact. Alternatively, the candidate setting unit 12 may calculate the degree of contact based on the intersection ratio, which is the ratio of the sample points 210 at which the Ray 220 intersects the surface 81a. The candidate setting unit 12 calculates the degree of contact using a function in which the larger the intersection ratio, the larger the degree of contact. Since whether the Ray 220 intersects the surface 81a is specified based on the distance, the calculation of the degree of contact based on the intersection ratio is also an example of the calculation of the degree of contact based on the distance. Alternatively, the candidate setting unit 12 may calculate the degree of contact based on both the statistical value of the distance and the intersection ratio. As yet another example, the candidate setting unit 12 may calculate the degree of contact based on at least one of the statistical value of the distance and the intersection ratio and the distance from the selected candidate position to the center of gravity of the first workpiece 81. For example, the candidate setting unit 12 further calculates the degree of contact using a function in which the smaller the distance to the center of gravity, the larger the degree of contact.

[0039] Return to FIG. 5. As shown in step S124, the candidate setting unit 12 calculates the degree of contact for each of the plurality of temporarily set candidate positions. If there are unprocessed candidate positions (NO in step S124), the process returns to step S122. In the repeated step S122, the candidate setting unit 12 selects the next candidate position. In the repeated step S123, the candidate setting unit 12 calculates the degree of contact between the first workpiece 81 and the contact surface of the first end effector 21a at that candidate position.

[0040] When all candidate positions have been processed (YES in step S124), the process proceeds to step S125. In step S125, the candidate setting unit 12 selects two or more candidate positions from the plurality of candidate positions based on the contact degree of each of the plurality of candidate positions. The candidate setting unit 12 may select two or more candidate positions whose contact degree is equal to or greater than a predetermined threshold value. Alternatively, the candidate setting unit 12 may sort the candidate positions in descending order of the contact degree and then select the top n candidate positions (where n > 1). Such a selection process can be said to be a process of narrowing down the plurality of temporarily set candidate positions to candidate positions where it is expected that the first end effector 21a can surely hold the first work 81, that is, candidate positions where stable holding is expected. The candidate setting unit 12 sets the selected two or more candidate positions for simulation.

[0041] As described with reference to FIG. 5, the candidate setting unit 12 temporarily sets a plurality of candidate positions on the first work 81 based on the first work model. Then, the candidate setting unit 12 selects two or more candidate positions from the plurality of candidate positions based on each of the plurality of candidate positions and the shape of the contact surface of the first end effector 21a. The candidate setting unit 12 sets the selected two or more candidate positions for simulation.

[0042] Returning to FIG. 4, in step S13, the simulation unit 13 selects one from the two or more narrowed-down candidate positions.

[0043] In step S14, the simulation unit 13 executes a simulation based on the selected candidate position. The simulation unit 13 executes a simulation based on at least the first work model and the first robot model. The simulation unit 13 may further execute a simulation based on at least one of the second work model and the second robot model. The simulation unit 13 uses the model data to generate a virtual space corresponding to the actual work space 9 and executes a simulation in that virtual space.

[0044] The simulation unit 13 virtually executes, by simulation, a pick process in which the first robot 21 holds the first workpiece 81 at a selected candidate position using the first end effector 21a, and a post-process for the first workpiece 81 held at the candidate position.

[0045] The simulation unit 13 may execute, as at least part of the post-process, a place process in which the first robot 21 places the first workpiece 81 held by the first end effector 21a at a specified position. In this case, the simulation unit 13 virtually executes the pick process and the place process. The combination of the pick process and the place process is also referred to as a pick-and-place process. The simulation unit 13 may execute, as the place process, a process in which the first robot 21 places the first workpiece 81 held by the first end effector 21a at a specified position on the second workpiece 82. Alternatively, the simulation unit 13 may execute, as the place process, a process in which the first workpiece 81 is placed at a specified position on an object different from the second workpiece 82, such as a workbench or a rack.

[0046] The simulation unit 13 may execute, as at least part of the post-processing, placement processing and additional processing on the first workpiece 81 held by the first robot 21 at a specified position. In this case, the simulation unit 13 virtually executes pick processing, placement processing, and additional processing. The additional processing may be processing performed by the first robot 21 without using the second robot 22, or may be processing executed by the cooperation of the first robot 21 and the second robot 22. For example, the simulation unit 13 may execute, as additional processing, cooperative processing in which the second robot 22 works on the first workpiece 81 held by the first robot 21 at a specified position. The simulation unit 13 may execute, as cooperative processing (additional processing), processing in which the second robot 22 fixes the first workpiece 81 on the second workpiece 82 by means such as welding or screwing. The simulation unit 13 executes, as placement processing, processing in which the first robot 21 positions the first workpiece 81 at a specified position set in the air, and may execute, as cooperative processing (additional processing), processing in which the second robot 22 receives the first workpiece 81 from the first robot 21 at the specified position. In the cooperative processing (additional processing), the simulation unit 13 may virtually execute work in a plurality of work areas on the first workpiece 81 by the second robot 22. A work area refers to a part of the workpiece processed by the robot. Each work area may be an area defined by a point, a line, or a plane.

[0047] As described above, the simulation unit 13 can execute various types of post-processing. The post-processing may include placement processing. Alternatively, the post-processing may include placement processing and additional processing. Since the additional processing may be cooperative processing, the post-processing may include placement processing and cooperative processing.

[0048] In any case, the simulation unit 13 virtually executes pick processing and post-processing corresponding to the selected candidate position. In this simulation, it may or may not be possible to complete the pick processing and the post-processing. In the present disclosure, "being able to complete the pick processing and the post-processing" means that in both the pick processing and the post-processing, the robot can maintain a normal posture (i.e., operate normally), no interference is detected, and as a result, a series of operations from the pick processing to the completion of the post-processing are successful. Interference refers to the phenomenon in which one object contacts or collides with another object. It should be noted that when a robot or another device attempts to process a certain workpiece, the contact between the robot or the other device and the workpiece is not interference. When the other device operates, it is also a condition for being able to complete the pick processing and the post-processing that the other device can maintain a normal posture in both the pick processing and the post-processing.

[0049] The simulation unit 13 stores the result of the simulation in a predetermined storage device such as the memory 162 and the storage 163. For example, the simulation unit 13 may store result data including a pair of the selected candidate position and flag information indicating whether the pick processing and the post-processing have been completed.

[0050] As shown in step S15, the simulation unit 13 executes a simulation for each of the two or more candidate positions that have been narrowed down. If there are unprocessed candidate positions (NO in step S15), the process returns to step S13. In the repeated step S13, the simulation unit 13 selects the next candidate position. In the repeated step S14, the simulation unit 13 virtually executes pick processing and post-processing for that candidate position by simulation.

[0051] When all of the candidate positions of 2 or more have been processed (YES in step S15), the process proceeds to step S16. In step S16, the positioning unit 14 determines, as a holding position, one of at least one candidate position that can complete the pick process and the post-process in the simulation. That is, the positioning unit 14 determines, as a holding position, one of at least one candidate position that has been successful in the simulation.

[0052] The positioning unit 14 refers to the result data of the simulation and identifies at least one candidate position that can complete the pick process and the post-process. For example, the positioning unit 14 may identify at least one candidate position where the first robot 21 operates normally and no interference is detected in both the pick process and the post-process. Alternatively, the positioning unit 14 may identify at least one candidate position where both the first robot 21 and the second robot 22 operate normally and no interference is detected for both of these two robots in both the pick process and the post-process.

[0053] The positioning unit 14 may determine, as a holding position, a candidate position selected by the user of the control support system 1. In one example, the positioning unit 14 displays the identified at least one candidate position on the user's display device (e.g., the monitor 120). Then, the positioning unit 14 determines, as a holding position, one candidate position selected by the user from the at least one candidate position.

[0054] Alternatively, the positioning unit 14 may automatically determine, as a holding position, one of the identified at least one candidate positions. For example, the positioning unit 14 determines, as a holding position, the candidate position with the highest degree of contact among the at least one candidate positions.

[0055] FIG. 8 is a diagram showing an example of a series of processes for determining a holding position with reference to a virtual first workpiece 81. As shown in state ST1, a candidate setting unit 12 sets a plurality of candidate positions 300 on the first workpiece 81 (step S121). The candidate setting unit 12 calculates a degree of contact for each of the plurality of candidate positions 300 (steps S122 to S124). As shown in state ST2, the magnitude of the degree of contact can be visualized by the darkness of the color of the bar drawn at each candidate position. In this example, the darker the color, the greater the degree of contact. As shown in state ST3, the candidate setting unit 12 selects two or more candidate positions 300 from the plurality of candidate positions 300 based on the degree of contact (step S125). In this example, the candidate setting unit 12 selects candidate positions 300 whose degree of contact is greater than a predetermined threshold value. As shown in state ST4, a simulation unit 13 executes a simulation based on each selected candidate position, and a positioning unit 14 identifies at least one candidate position 300 at which pick processing and post-processing can be completed based on the result of the simulation (steps S13 to S16). As shown in state ST5, the positioning unit 14 determines, based on the user's selection or automatically, one of the at least one identified candidate positions 300 as a holding position 320 (step S16).

[0056] Returning to FIG. 4, in step S17, a robot control unit 15 controls a real robot based on the holding position.

[0057] The robot control unit 15 generates an operation program for controlling the real robot based on the holding position. The robot control unit 15 generates a first operation program for causing the first robot 21 to perform pick processing for holding the first workpiece 81 at the holding position and post-processing for the first workpiece 81 held at the holding position. In one example, the robot control unit 15 may generate the first operation program based on the simulation result corresponding to the holding position.

[0058] The operation program contains data for controlling the robot, and includes, for example, a path indicating the trajectory of the robot. The trajectory of the robot refers to the path of the movement of the robot or its components. For example, the trajectory of the robot can be the trajectory of the tip or the end effector. The first operation program includes at least code for causing the actual first robot 21 to hold the first workpiece 81 at the determined holding position. The first operation program may further include code for causing the actual first robot 21 to execute at least a part of the post-processing.

[0059] The robot control unit 15 may generate a second operation program for controlling the actual second robot 22. The second operation program may further include code for causing the actual second robot 22 to execute at least a part of the post-processing.

[0060] The robot control unit 15 controls the actual robot based on the operation program. The robot control unit 15 controls the first robot 21 disposed in the work space 9 so as to execute an actual pick-up process of holding the first workpiece 81 existing in the work space 9 by the first end effector 21a at the holding position. The robot control unit 15 outputs the first operation program to the robot controller 3 to cause the robot controller 3 to control the first robot 21. The robot controller 3 operates the first robot 21 based on the first operation program. The robot control unit 15 may output the second operation program to the robot controller 3 to cause the robot controller 3 to control the second robot 22. The robot controller 3 may operate the second robot 22 based on the second operation program.

[0061] [Modification Example] As described above, the technology according to the present disclosure has been described in detail based on various examples. However, the present disclosure is not limited to the above examples. Various modifications are possible for the technology according to the present disclosure without departing from the gist thereof.

[0062] The control support system may acquire a work model indicating a work with a plurality of candidate positions set in advance. Alternatively, the control support system may output the holding position to another computer system such as a robot control system, and the other computer system may control the actual robot based on the holding position. That is, the control support system may not include a functional module corresponding to at least one of the candidate setting unit 12 and the robot control unit 15.

[0063] In the above example, the other device is the second robot 22, but the other device may be a device other than a robot, such as a conveyor or an automatic rack.

[0064] The control support system may virtually execute the pick process and the post-process in an environment where there is no other device such as the second robot 22 by simulation to determine the holding position. For example, the post-process performed alone by a robot such as the first robot 21 may be simulated.

[0065] The hardware configuration of the system is not limited to the mode of realizing each functional module by executing a program. For example, at least a part of the above-described functional module group may be configured by a logic circuit specialized for the function, or may be configured by an ASIC (Application Specific Integrated Circuit) integrating the logic circuit.

[0066] The processing procedures of the method executed by at least one processor are not limited to the above example. For example, some of the above-described steps or processes may be omitted, or each step may be executed in a different order. Also, any two or more of the above-described steps may be combined, or some of the steps may be modified or deleted. Alternatively, other steps may be executed in addition to the above-described steps.

[0067] When comparing the magnitude relationship between two numerical values in a computer system or a computer, either of the two criteria of "greater than or equal to" and "greater than" may be used, and either of the two criteria of "less than or equal to" and "less than" may be used.

[0068] [Appendix] As can be understood from the various examples above, the present disclosure includes the aspects shown below. (Appendix 1) For each of two or more candidate positions that are two or more candidates for a holding position on a workpiece held by an end effector of a robot, a pick process in which the end effector holds the workpiece at the candidate position and a post-process for the workpiece held at the candidate position are virtually executed by simulation based on a workpiece model showing the shape of the workpiece and a robot model showing the robot having the end effector, a determination unit that determines, as the holding position, one of at least one of the candidate positions that can complete the pick process and the post-process in the simulation; A control support system comprising: (Appendix 2) The post-process includes a place process in which the robot places the held workpiece at a designated position and an additional process for the workpiece held by the robot at the designated position, The simulation unit virtually executes the pick process, the place process, and the additional process. The control support system according to Appendix 1. (Appendix 3) The simulation unit virtually executes, as the post-process, a process that the robot and the separate device cooperate to execute on the workpiece held at the candidate position by simulation based on an apparatus model showing a separate device different from the robot, The determination unit identifies at least one of the candidate positions that can complete the pick process and the post-process without detecting interference for both the robot and the separate device. The control support system described in Supplementary Note 1 or 2. (Supplementary Note 4) wherein the other device is another robot, the simulation unit virtually executes the post-processing including a placement process in which the robot places the held workpiece at a specified position and a cooperation process in which the other robot works on the workpiece held by the robot at the specified position. The control support system described in Supplementary Note 3. (Supplementary Note 5) in the cooperation process, the simulation unit virtually executes the work of the other robot in a plurality of work areas on the workpiece. The control support system described in Supplementary Note 4. (Supplementary Note 6) the simulation unit virtually executes, as the placement process, a process in which the robot places the held workpiece at the specified position on another workpiece, and virtually executes, as the cooperation process, a process in which the other robot fixes the workpiece on the other workpiece. The control support system described in Supplementary Note 4 or 5. (Supplementary Note 7) The control support system according to any one of Supplementary Notes 1 to 6, further comprising a robot control unit that controls a robot disposed in the real work space so as to execute a real pick-up process of holding the workpiece existing in the real work space by the end effector at the holding position. (Supplementary Note 8) an acquisition unit that acquires the work model of the specified workpiece when the workpiece held by the end effector is specified; a setting unit that sets the two or more candidate positions based on the acquired work model; The control support system according to any one of Supplementary Notes 1 to 7, further comprising the above. (Supplementary Note 9) the setting unit virtually sets a plurality of the candidate positions on the workpiece based on the work model. selecting the two or more candidate positions from the plurality of candidate positions based on each of the plurality of candidate positions and the shape of the contact surface of the end effector capable of contacting the workpiece; setting the selected two or more candidate positions for the simulation; The control support system according to Supplementary Note 8. (Supplementary Note 10) The setting unit For each of the plurality of candidate positions, calculate the degree of contact between the contact surface and the workpiece when the center of the contact surface that has contacted the workpiece is positioned at the candidate position; select the two or more candidate positions from the plurality of candidate positions based on the degree of contact of each of the plurality of candidate positions; The control support system according to Supplementary Note 9. (Supplementary Note 11) The setting unit, for each of the plurality of candidate positions, generate a plurality of sample points on the contact surface; for each of the plurality of sample points, calculate the distance from the sample point to the surface of the workpiece; calculate the degree of contact based on the distances of each of the plurality of sample points; The control support system according to Supplementary Note 10. (Supplementary Note 12) The determination unit display, on the user's display device, the at least one candidate position at which the post-processing can be completed in the simulation; determine, as the holding position, one candidate position selected by the user from the at least one candidate position; The control support system according to any one of Supplementary Notes 1 to 11. (Supplementary Note 13) A control support method executed by a control support system including at least one processor, For each of two or more candidate positions, which are two or more candidates for a holding position on a workpiece held by an end effector of a robot, a pick process in which the end effector holds the workpiece at the candidate position and a post-process for the workpiece held at the candidate position are virtually executed by simulation based on a workpiece model showing the shape of the workpiece and a robot model showing the robot having the end effector, Determining, as the holding position, one of at least one of the candidate positions that can complete the pick process and the post-process in the simulation; A control support method including the above. (Appendix 14) For each of two or more candidate positions, which are two or more candidates for a holding position on a workpiece held by an end effector of a robot, a pick process in which the end effector holds the workpiece at the candidate position and a post-process for the workpiece held at the candidate position are virtually executed by simulation based on a workpiece model showing the shape of the workpiece and a robot model showing the robot having the end effector, Determining, as the holding position, one of at least one of the candidate positions that can complete the pick process and the post-process in the simulation; A control support program that causes a computer to execute the above.

[0069] According to Appendices 1, 13, and 14, for each of two or more candidate positions, not only the pick process but also the subsequent process are virtually executed, and a candidate position that can complete both processes is determined as the holding position. Instead of a holding position that is only effective in the pick process, a holding position that is also effective in the subsequent process is determined, so there is no need to change the holding position between these two processes. That is, this mechanism can determine a holding position of a workpiece that contributes to efficient operation of the robot.

[0070] According to Supplementary Note 2, for each of two or more candidate positions, not only the pick-and-place process but also subsequent processes are virtually executed, and then a candidate position where both processes can be completed is determined as the holding position. Instead of a holding position that is only effective in the process of placing the workpiece at a specified position, a holding position that is also effective in subsequent processes is determined, so there is no need to change the holding position during this series of processes. That is, this mechanism can determine the holding position of the workpiece that contributes to the efficient operation of the robot.

[0071] According to Supplementary Note 3, a simulation is executed including the operation of another device that cooperates with the robot executing the pick process, and finally a holding position where the pick process and post-processing can be completed is determined. With this mechanism, it is possible to determine the position where the workpiece should be held in the process by multiple devices.

[0072] Since the processing by multiple robots can become complex, it is not easy to manually determine the holding position in this processing. According to Supplementary Note 4, a simulation is executed including the operation of another robot that cooperates with the robot executing the pick-and-place process, and finally a holding position where the pick-and-place process and cooperative processing can be completed is determined. By using this mechanism, a series of complex operations by multiple robots can be realized, and also the holding position of the workpiece that contributes to the efficient operation of the robot in such complex operations can be determined.

[0073] In a scenario where multiple robots process multiple work areas on the workpiece, more factors need to be considered than in the processing related to a single robot or a single work area. According to Supplementary Note 5, the holding position for multiple robots to cooperate in processing multiple work areas on the workpiece is finally determined. Therefore, even in such a complex scenario, the holding position of the workpiece that contributes to the efficient operation of the robot can be determined.

[0074] According to Supplementary Note 6, even in a complex scenario where multiple robots cooperate to fix the workpiece to another workpiece, the holding position of the workpiece that contributes to the efficient operation of the robot can be determined.

[0075] According to Supplementary Note 7, the end effector can easily determine the position where the work should be held, and thus the labor for more efficiently controlling the actual robot can be reduced accordingly.

[0076] According to Supplementary Note 8, when a work is specified, a work model of the work is acquired, and two or more candidate positions are set based on the work model. With this mechanism, even in a scenario where the robot processes various works, the holding position at which the robot can operate efficiently can be determined for each work.

[0077] According to Supplementary Note 9, first, a plurality of candidate positions are temporarily set, then the number of the candidate positions is narrowed down based on the shape of the contact surface of the end effector, and a simulation is executed with the remaining candidate positions. By reducing the number of candidate positions to be simulated, the total execution time of the simulation can be shortened, and thus the holding position can be determined more quickly.

[0078] According to Supplementary Note 10, since the degree of contact between the contact surface of the end effector and the work is considered, a candidate position that is expected to reliably hold the work by the end effector, that is, a candidate position that is expected to reliably execute at least the pick-up process can be selected. Therefore, the simulation for two or more candidate positions can be efficiently executed, and the holding position can be determined more quickly.

[0079] According to Supplementary Note 11, by considering the distance from each sample point on the contact surface in contact with the work to the surface of the work, the degree of contact between the contact surface and the work can be calculated more accurately.

[0080] According to Supplementary Note 12, by preparing a mechanism that entrusts the user with the final determination of the holding position, the holding position can be determined in a form that also reflects the knowledge and experience of the user. Since only candidate positions that can reliably hold the work and execute post-processing are presented to the user, the labor of the user for determining the holding position can be reduced.

Explanation of Signs

[0081] 1... Control support system, 3... Robot controller, 9... Workspace, 11... Model acquisition unit, 12... Candidate setting unit, 13... Simulation unit, 14... Position determination unit, 15... Robot control unit, 21... First robot, 21a... First end effector, 22... Second robot (separate device), 22a... Second end effector, 81... First workpiece, 82... Second workpiece (separate workpiece), 200... Contact surface, 210... Sample point, 300... Candidate position, 320... Holding position.

Claims

1. For each of two or more candidate positions, which are two or more candidates for a holding position on a workpiece held by an end effector of a robot, a pick process in which the end effector holds the workpiece at the candidate position and a post-process for the workpiece held at the candidate position are virtually executed by simulation based on a workpiece model showing the shape of the workpiece and a robot model showing the robot having the end effector; a simulation unit; a determination unit that determines, as the holding position, one of the at least one candidate position in which the pick process and the post-process can be completed in the simulation; A control support system comprising:

2. The post-process includes a place process in which the robot places the held workpiece at a designated position and an additional process for the workpiece held by the robot at the designated position, The simulation unit virtually executes the pick process, the place process, and the additional process. The control support system according to claim 1.

3. The simulation unit virtually executes, as the post-process, a process in which the robot and the other device cooperate to execute on the workpiece held at the candidate position by simulation based on an apparatus model showing an apparatus different from the robot; The determination unit identifies the at least one candidate position in which the pick process and the post-process can be completed without detecting interference for both the robot and the other device. The control support system according to claim 1.

4. The other device is another robot, The simulation unit virtually executes the post-process including a place process in which the robot places the held workpiece at a designated position and a cooperation process in which the other robot works on the workpiece held by the robot at the designated position. The control support system according to claim 3.

5. In the cooperation process, the simulation unit virtually executes work in a plurality of work areas on the workpiece by the other robot. The control support system according to claim 4.

6. The simulation unit virtually executes, as the place process, a process in which the robot places the held workpiece at the designated position on another workpiece. virtually execute, as the collaborative process, the process in which the other robot fixes the workpiece on the other workpiece The control support system according to claim 5

7. The control support system according to any one of claims 1 to 6, further comprising a robot control unit that controls a robot arranged in the actual work space so as to execute an actual pick-up process of holding the workpiece existing in the actual work space by the end effector at the holding position

8. When the workpiece held by the end effector is specified, an acquisition unit that acquires the work model of the specified workpiece, A setting unit that sets the two or more candidate positions based on the acquired work model The control support system according to any one of claims 1 to 6, further comprising

9. The setting unit temporarily sets a plurality of the candidate positions on the workpiece based on the work model, selects the two or more candidate positions from the plurality of candidate positions based on each of the plurality of candidate positions and the shape of the contact surface of the end effector that can contact the workpiece, sets the selected two or more candidate positions for the simulation The control support system according to claim 8

10. The setting unit for each of the plurality of candidate positions, calculates the degree of contact between the contact surface and the workpiece when the center of the contact surface that has contacted the workpiece is positioned at the candidate position, selects the two or more candidate positions from the plurality of candidate positions based on the degree of contact of each of the plurality of candidate positions The control support system according to claim 9

11. The setting unit, for each of the plurality of candidate positions generates a plurality of sample points on the contact surface, for each of the plurality of sample points, calculates the distance from the sample point to the surface of the workpiece, calculates the degree of contact based on the distance of each of the plurality of sample points The control support system according to claim 10

12. The determination unit displays, on a display device of the user, the at least one candidate position at which the post-processing can be completed in the simulation, determines, as the holding position, one candidate position selected by the user from the at least one candidate position The control support system according to any one of claims 1 to 6

13. A control support method executed by a control support system including at least one processor, for each of two or more candidate positions which are two or more candidates for a holding position on a workpiece held by an end effector of a robot, virtually executing, by simulation based on a workpiece model showing the shape of the workpiece and a robot model showing the robot having the end effector, a pick process in which the end effector holds the workpiece at the candidate position and a post-process for the workpiece held at the candidate position; determining, as the holding position, one of at least one of the candidate positions that can complete the pick process and the post-process in the simulation; A control support method including the above.

14. for each of two or more candidate positions which are two or more candidates for a holding position on a workpiece held by an end effector of a robot, virtually executing, by simulation based on a workpiece model showing the shape of the workpiece and a robot model showing the robot having the end effector, a pick process in which the end effector holds the workpiece at the candidate position and a post-process for the workpiece held at the candidate position; determining, as the holding position, one of at least one of the candidate positions that can complete the pick process and the post-process in the simulation; A control support program causing a computer to execute the above.

Images