Processing device, processing method, and recording medium

Abstract

This processing device comprises: a determination means that, in a series of operations for each of one or more robots including a plurality of operations for executing work, determines an optimization range indicating up to which operation among the plurality of operations is to be optimized; and a calculation means that performs optimization calculation that satisfies a constraint condition in the execution of the work including at least the execution of the work in the optimization range, and minimizes an objective function indicating the time required for the work.

Description

Processing device, processing method, and recording medium

[0001] The present disclosure relates to a processing device, a processing method, and a recording medium.

[0002] Techniques for optimizing a program that controls the operation of a controlled object such as a robot are known. Patent Document 1 discloses, as a related technique, a method for optimizing the non-linear control characteristics related to a controlled object for each operating state of the controlled object.

[0003] Japanese Patent Application Laid-Open No. 2004-030269

[0004] In the fields of control such as work by a plurality of robots related to Patent Document 1, autonomous driving of a plurality of automobiles, joint transportation by a plurality of drones, cooperative transportation by a plurality of transport devices such as AGVs (Automated Guided Vehicles), joint work by a plurality of construction machines that operate autonomously, and cooperation of a plurality of manufacturing devices, there is a need for a technology that can efficiently operate each of a plurality of controlled objects without each of the plurality of controlled objects having an adverse effect on other controlled objects.

[0005] One of the purposes of each aspect of the present disclosure is to provide a processing device, a processing method, a recording medium, etc. that can solve the above problems.

[0006] According to one aspect of the present disclosure, a processing device includes: a determination means for determining an optimization range indicating up to which operation among the plurality of operations is to be optimized in a series of operations for each of one or more robots including a plurality of operations for executing a task; and a calculation means for performing an optimization calculation that satisfies a constraint condition in the execution of the task including at least the execution of the task within the optimization range and minimizes an objective function indicating the time required for the task.

[0007] According to another aspect of the present disclosure, the processing method includes determining an optimization range that indicates which of the multiple actions to be optimized in a series of actions for each of one or more robots, which include multiple actions for performing a task, and performing an optimization calculation that satisfies constraints in the execution of the task, which includes at least the execution of the task within the optimization range, and minimizes an objective function that indicates the time required for the task.

[0008] According to another aspect of this disclosure, the recording medium stores a program that causes a computer to perform the following: determine an optimization range indicating which of a plurality of actions to be optimized in a series of actions for each of one or more robots, which include a plurality of actions for performing an action; and perform an optimization calculation that satisfies constraints in the execution of the action, which includes at least the execution of the action within the optimization range, and minimizes an objective function indicating the time required for the action.

[0009] According to the above configurations, each of the multiple controlled objects can operate efficiently without adversely affecting any of the other controlled objects.

[0010] This figure shows an example of the configuration of an optimization device according to some embodiments of the present disclosure. This figure shows an example of the configuration of an optimization device according to some embodiments of the present disclosure. This figure shows an example of operation definition information according to some embodiments of the present disclosure. This figure shows an example of optimization range definition information according to some embodiments of the present disclosure. This figure shows an example of the processing flow of a processing system according to some embodiments of the present disclosure. This figure shows an example of operation definition information according to some embodiments of the present disclosure. This figure shows an example of the configuration of an optimization device according to some embodiments of the present disclosure. This figure shows an example of the processing flow of a processing system according to some embodiments of the present disclosure. This figure shows an example of the calculation result of the operation plan in updating the optimization range definition information according to some embodiments of the present disclosure. This figure shows an example of the extraction result in updating the optimization range definition information according to some embodiments of the present disclosure. This figure shows an example of the time range for extraction in updating the optimization range definition information according to some embodiments of the present disclosure. This figure shows an example of the configuration of a processing device according to some embodiments of the present disclosure. This figure shows an example of the processing flow of a processing device according to some embodiments of the present disclosure. This is a schematic block diagram showing the configuration of a computer according to at least one embodiment.

[0011] The embodiments will be described in detail below with reference to the drawings. In each embodiment of this disclosure, "optimization" means identifying and adopting the best result from among several results obtained in the process. However, strictly speaking, there may be a better result than the "optimization" itself. In other words, there may be a more suitable result other than the several results obtained in the process.

[0012] <Embodiment> A processing system 1 according to one embodiment of the present disclosure will be described. The processing system 1 according to one embodiment of the present disclosure is a system that narrows the range of motion planning to reduce the cost of motion planning when controlling multiple objects to be controlled. Examples of the processing system 1 include a robot system that controls multiple robots, an autonomous driving system that automatically drives multiple automobiles, a transport system that performs joint transport by multiple drones or coordinated transport by multiple AGVs (Automatic Guided Vehicles), a collaborative work system by multiple autonomously operating construction machines, and a collaborative system in which multiple manufacturing machines work together. In the following description, a robot system will be described as a specific example of the processing system 1. However, the processing system 1 is not limited to a robot system. For example, as described above, the processing system 1 may be a robot system that controls multiple robots, an autonomous driving system that automatically drives multiple automobiles, a transport system that performs joint transport by multiple drones or coordinated transport by multiple AGVs, a collaborative work system by multiple autonomously operating construction machines, or a collaborative system in which multiple manufacturing machines work together.

[0013] (Configuration of the Processing System) Figure 1 is a diagram showing an example of the configuration of a processing system 1 according to some embodiments of the present disclosure. The processing system 1 comprises robots 10a1, 10a2, ..., 10aM, an optimization device 20, and trays 30a1, 30a2, ..., 30aN. The robots 10a1, 10a2, ..., 10aM are sometimes collectively referred to as robot 10. The trays 30a1, 30a2, ..., 30aN are sometimes collectively referred to as tray 30. M and N are integers of 2 or more. The processing system 1, for example, uses multiple (i.e., M) arm robots 10 to perform the task of aligning items (not shown) placed on the source tray 30a1 to the destination tray 30a2.

[0014] Furthermore, if the items placed in the source tray 30a1 run out, or if there is no longer enough space to arrange items in the destination tray 30a2, the processing system 1 may be configured to replace the source tray 30a1 or destination tray 30a2 with another tray 30 as needed. This allows the processing system 1 to continue operations continuously according to the site conditions.

[0015] The following describes a specific example of the processing system 1 when M and N are both 2, and tray 30a1 is the source tray 30, and tray 30a2 is the destination tray 30.

[0016] Robot 10 is equipped with a hand 101 for picking or placing items. Examples of the types of hands 101 include suction type, two-fingered type, multi-fingered type with three or more fingers, suction type, and combinations of two-fingered and multi-fingered types. However, the type of hand 101 is not limited to these types. Robot 10 is, for example, an arm robot. Examples of arm robots include vertical articulated arm robots with six or seven joints, and horizontal articulated arm robots with four joints. However, arm robots are not limited to these types. Furthermore, each arm robot 10 may be equipped with the same type of hand 101, or it may be equipped with different types of hands 101.

[0017] Robots 10a1 and 10a2 each pick up items placed on tray 30a1 and arrange and place the items on tray 30a2.

[0018] The optimization device 20 calculates and determines the operation procedure and control input for each robot 10 so that the operation of each robot 10 is efficient. Specifically, for example, the optimization device 20 calculates and determines the operation procedure and control input for each robot 10 so as to minimize the time it takes for the robots 10 to complete the item alignment work without interfering with each other or colliding with each of the trays 30.

[0019] Figure 2 shows an example of the configuration of an optimization device 20 according to some embodiments of the present disclosure. As shown in Figure 2, the optimization device 20 includes an operation input unit 201, a communication interface unit 202, an arithmetic processing unit 203, a storage unit 204, and a control unit 205.

[0020] The operation input unit 201 includes an operation input device 2011. Examples of the operation input device 2011 include a keyboard and a mouse. The operation input unit 201 detects operations performed by the operator on the optimization device 20. The operation input unit 201 then outputs information indicating the detected operation to the arithmetic processing unit 203.

[0021] The communication interface unit 202 includes a data communication line 2021. An example of the data communication line 2021 is Ethernet. The communication interface unit 202 performs data communication with external devices connected via the data communication line 2021.

[0022] The memory unit 204 stores at least operation definition information I1, optimization range definition information I2, constraint information I3, objective function information I4, and program I5. Examples of memory units 204 include information storage devices such as HDDs (Hard Disk Drives), SSDs (Solid State Drives), DVDs (Digital Versatile Disks), and RAMs (Random Access Memory).

[0023] The memory unit 204 stores operation definition information I1 that has been acquired in advance using methods such as receiving input via the operation input unit 201 or receiving input from an external device via the communication interface unit 202.

[0024] The motion definition information I1 includes information that defines the operation procedure of the robot 10. Specifically, for example, the motion definition information I1 is information that associates the state of each robot 10 with next action information that indicates the next action that each robot 10 should take in that state.

[0025] Figure 3 shows an example of operation definition information I1 according to some embodiments of the present disclosure. In the example shown in Figure 3, the storage unit 204 stores as operation definition information I1 the definition of the operation procedure for the robot 10's pick-up and place operation, in which the robot 10 picks up an item, transports it to a designated location, and then places the item. That is, when the robot 10 starts operation from an initial state, the operation procedure is defined as follows: reach (bring the end-effector close to the vicinity of the item), pick (approach) (bring the end-effector further close to the item until it is in a pickable state), pick (grasp) (operate the hand to pick up the item), transport (bring the end-effector close to the vicinity of the place to place the item while still holding the item), place (approach) (bring the end-effector further close to the place to place until it is in a placeable state), and place (place) (operate the hand to place the item). The operation procedure is defined so that each operation is performed as the next operation when the preceding operation is completed, and when the place (place) operation is completed, the reach operation is performed again as the next operation.

[0026] The memory unit 204 stores the optimization range definition information I2 that has been acquired in advance using methods such as receiving input via the operation input unit 201 or receiving input from an external device via the communication interface unit 202.

[0027] The optimization range definition information I2 includes information that defines the optimization range, indicating which state transitions in a series of operation procedures for the robot 10 to perform a task should be optimized. Specifically, for example, the optimization range definition information I2 is information that associates each state of the robot 10 with the optimization range information for each state of the robot 10.

[0028] Figure 4 shows an example of optimization range definition information I2 according to some embodiments of the present disclosure. As shown in Figure 4, for example, when each of the robots 10 is in its initial state (i.e., ID 1), the optimization range for robot 10a1 is until transport is completed, and the optimization range for robot 10a2 is until pick (approach) is completed. Also, as shown in Figure 4, for example, when robot 10a1 has completed transport and robot 10a2 has completed pick (approach) (i.e., ID 21), the optimization range for robot 10a1 is until reach is completed, and the optimization range for robot 10a2 is until place (approach) is completed.

[0029] Furthermore, by comparing the optimization range definition information I2 with the operation definition information I1, it can be seen that, for example, if each of the robots 10 is in its initial state, the series of operations of reaching, picking (approaching), picking (grasping), and transporting should be optimized for robot 10a1, and the series of operations of reaching and picking (approaching) should be optimized for robot 10a2.

[0030] The constraint information I3 includes information that defines the constraint conditions in the optimization calculation. The constraint information I3 is acquired in advance by methods such as receiving input via the operation input unit 201 or receiving input from an external device via the communication interface unit 202, and is stored in the storage unit 204.

[0031] Constraint information I3 includes constraints for robots 10a1 and 10a2 when performing pick-and-place operations. Specific examples of constraints include those expressed mathematically, such as "the end-effectors of robots 10a1 and 10a2 must not interfere with each other" and "the end-effector of robot 10a1 must be above tray 30a1, and the end-effector of robot 10a2 must be above tray 30a2." For example, the constraint "the end-effectors of robots 10a1 and 10a2 must not interfere with each other" can be expressed as shown in equation (1), where p0(t) and p1(t) are the three-dimensional coordinates indicating the positions of the end-effectors of robots 10a1 and 10a2 at time t, Δp is the distance required to avoid interference, and the time range for optimization is 0 to T.

[0032]

[0033] Furthermore, in addition to the Euclidean distance (L2 norm), the L1 norm and L∞ norm can be used to express the coordinate distance between two points. However, constraints are not limited to those expressed using mathematical formulas.

[0034] The memory unit 204 stores objective function information I4 that has been acquired in advance using methods such as receiving input via the operation input unit 201 or receiving input from an external device via the communication interface unit 202.

[0035] As shown in Figure 2, the arithmetic processing unit 203 includes an operating state acquisition unit 2031, an optimization range determination unit 2032, an optimization condition information creation unit 2033, and an optimization calculation unit 2034. Each of the operating state acquisition unit 2031, the optimization range determination unit 2032, the optimization condition information creation unit 2033, and the optimization calculation unit 2034 can be realized, for example, by an arithmetic device such as the CPU (Central Processing Unit) of the arithmetic processing unit 203 reading program I5 from the storage unit 204 and executing program I5.

[0036] The arithmetic processing unit 203 may, instead of the CPU described above, include at least one of the following: GPU (Graphic Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating point number Processing Unit), PPU (Physics Processing Unit), TPU (Tensor Processing Unit), quantum processor, and microcontroller. Furthermore, if the arithmetic processing unit 203 includes two or more of the GPU, DSP, MPU, FPU, PPU, TPU, quantum processor, and microcontroller, they may cooperate in processing.

[0037] The operation state acquisition unit 2031 acquires the state information of each robot 10 at the time the optimization calculation is started. Specifically, for example, if each robot 10 is in the state before starting work, it acquires state information indicating that each robot 10 is in its initial state. Also, if each robot 10 is performing work and the optimization calculation is to be performed for the next action of each robot 10 after the execution of a certain action in the work, it acquires the state information of each robot 10 at the time the work is completed.

[0038] The optimization range determination unit 2032 determines the optimization range (i.e., up to which state transitions in a series of operation procedures for the robot 10 to perform a task will be optimized) for each robot 10 based on the state information of each robot 10. In this embodiment, the optimization range determination unit 2032 determines the optimization range by referring to the optimization range definition information I2. However, the method of determining the optimization range performed by the optimization range determination unit 2032 is not limited to this. Specifically, for example, the optimization range determination unit 2032 may maintain information on the time required for each operation in the task performed by each robot 10, and then determine the optimization range based on the state information and time required information of each robot 10.

[0039] The optimization condition information creation unit 2033 creates control variables, constraints, and an objective function for the optimization calculation based on the constraint information I3, the state information of each robot 10, and the optimization range. Specific examples of problem settings for the optimization calculation include mixed-integer programming problems and nonlinear programming problems. In creating the control variables, constraints, and objective function, the optimization condition information creation unit 2033 sets, for example, the change per unit time of the end-effector velocity of each robot 10 (accelerometer of the end-effector position) or the change per unit time of the joint angular velocity (angular acceleration of the joints) as control variables. The optimization condition information creation unit 2033 also creates constraints based on the state information of each robot 10 and the optimization range, which include at least the following conditions: that the initial state of each robot 10 is the state indicated by the state information; that each robot 10 performs all operations that should be completed within the target time of the optimization calculation; and that each operation of each robot 10 is performed in a predetermined order.

[0040] Furthermore, the optimization condition information creation unit 2033 may formulate, as a constraint, that each robot 10 does not come into contact with itself or other robots 10 or surrounding obstacles, and may formulate, as an objective function, at least one of the following two: minimizing the time required to complete the operations required for all robots 10, and minimizing the sum of the squares of the control inputs to all robots 10. In addition, if multiple objective functions are set, the optimization condition information creation unit 2033 may formulate minimizing the weighted sum of each of the multiple objective functions.

[0041] The optimization calculation unit 2034 performs optimization calculations based on the formulated constraints and objective function. Specific examples of the optimization calculation unit 2034 include optimization solvers, such as mixed-integer programming solvers.

[0042] The control unit 205 controls each of the robots 10 based on the results of the optimization calculation performed by the optimization calculation unit 2034. Specifically, the control unit 205 controls each of the robots 10 so that the constraints are satisfied within the optimization range and the objective function is minimized.

[0043] Note that the processing performed by the processing system 1 according to an embodiment of the present disclosure is not limited to the above-described processing. For example, the processing system 1 may perform the processing described below.

[0044] (Processing performed by the processing system) FIG. 5 is a diagram showing an example of a processing flow of the processing system 1 according to some embodiments of the present disclosure. Next, the processing performed by the processing system 1 will be described with reference to FIG. 5.

[0045] First, the operation state acquisition unit 2031 acquires the operation state of each of the robots 10 (10a1, 10a2) (step S1). In the present embodiment, it is assumed that through the operation of the operation state acquisition unit 2031, information indicating that all of the robots 10 are in the "initial state", that is, the state before the start of work, is acquired.

[0046] Next, the optimization range determination unit 2032 determines the optimization range of each of the robots 10 (that is, up to which state transition each of the robots 10 optimizes) based on the operation state of each of the robots 10 (step S2). More specifically, for example, the optimization range determination unit 2032 refers to the optimization range definition information I2 and acquires information indicating the optimization range of each of the robots 10 using the operation states of both robots 10 as keys. Here, the optimization range determination unit 2032 refers to the optimization range definition information I2 shown in FIG. 4. In the case of the example described here, since the state of the robot 10a1 is the "initial state" and the state of the robot 10a2 is the "initial state", the optimization range determination unit 2032 refers to the state of "transportation completed" as the optimization range of the robot 10a1 and refers to the state of "pick (approach) completed" as the optimization range of the robot 10a2.

[0047] After that, the optimization condition information creation unit 2033 acquires the operations and operation procedures included in the optimization range (step S3). For example, the optimization condition information creation unit 2033 refers to the operation definition information I1. Then, the optimization condition information creation unit 2033 acquires the operations and operation procedures necessary for the robot 10 to transition from its respective operation states to the states indicated in the optimization range. Here, the optimization condition information creation unit 2033 refers to the operation definition information I1 shown in FIG. 3 stored in the storage unit 204. For the robot 10a1, the optimization condition information creation unit 2033 acquires the next operations indicating the operations and operation procedures necessary to transition from the "initial state" to the "transport completion" in the operation definition information I1, that is, the four operations of "reach", "pick (approach)", "pick (grip)", and "transport" and their order. Similarly, for the robot 10a2, the optimization condition information creation unit 2033 acquires the next operations indicating the operations and operation procedures necessary to transition from the "initial state" to the "pick (approach) completion" in the operation definition information I1, that is, the two operations of "reach" and "pick (approach)" and their order.

[0048] Next, the optimization condition information creation unit 2033 creates optimization conditions (i.e., control variables, constraints, and objective function) based on the acquired operations and operation procedures, and constraint information I3 (step S4). In this embodiment, for example, the optimization calculation problem is defined as a mixed integer programming problem. The optimization condition information creation unit 2033 also sets the change per unit time of the time-series end-effector velocity of each robot 10 (10a1, 10a2) as control variables. Furthermore, the optimization condition information creation unit 2033 formulates constraints for robot 10a1, including that the four necessary operations, "reach," "pick (approach)," "pick (grasp)," and "transport," are each executed, that "pick (approach)" starts after the completion of "reach," that "pick (grasp)" starts after the completion of "pick (approach)," and that "transport" starts after the completion of "pick (grasp)." Furthermore, the optimization condition information creation unit 2033 formulates constraints that include the following: for robot 10a2, the two necessary operations, "reach" and "pick (approach)," are performed, and "pick (approach)" begins after "reach" is completed. Furthermore, the optimization condition information creation unit 2033 formulates constraints that include: at the start and end of each operation, the end-effector position, end-effector velocity, and gripping state of each robot 10 are at predetermined positions, predetermined velocities, and predetermined states, respectively; for each robot 10, predetermined relationships are established between the time-series end-effector position, end-effector velocity, and the change in end-effector velocity per unit time; and the end-effector positions of each robot 10 do not interfere with each other. Furthermore, the optimization condition information creation unit 2033 formulates an objective function that includes minimizing the time required to complete all necessary operations for each robot 10. The optimization condition information creation unit 2033 formulates that at least the constraints and objective function listed in the explanation of the process in step S4 are included.

[0049] Then, the optimization calculation unit 2034 performs an optimization calculation that satisfies the constraints formulated in step S4 and minimizes the objective function (step S5). In this embodiment, the optimization calculation unit 2034 calculates time-series information of the change per unit time of the end-effector velocity of each robot 10 through the optimization calculation.

[0050] Next, the optimization calculation unit 2034 outputs time-series information of the change per unit time of the end-effector velocity of each robot 10, which is the result of the optimization calculation including the calculated time-series information (step S6).

[0051] The control unit 205 controls each of the robots 10 based on the results of the optimization calculation calculated by the optimization calculation unit 2034 (step S7). Specifically, the control unit 205 controls each of the robots 10 that satisfies the constraints within the optimization range and minimizes the objective function.

[0052] (Advantages) The processing system 1 according to one embodiment of the present disclosure has been described above. In the optimization device 20 (an example of a processing device) of the processing system 1, the optimization range determination unit 2032 (an example of a determination means) determines an optimization range that indicates which of the multiple operations to be optimized in a series of operations for each robot 10 (an example of one or more robots) that includes multiple operations for performing a task. The optimization calculation unit 2034 (an example of a calculation means) performs an optimization calculation that satisfies constraints in the execution of the task, which includes at least the execution of the task within the optimization range, and minimizes an objective function that indicates the time required for the task.

[0053] The optimization device 20 of this processing system 1 enables each of the multiple controlled objects to operate efficiently without each object negatively affecting the other controlled objects.

[0054] <First Modification of Embodiment> Next, a processing system 1 according to a first modification of one embodiment of the present disclosure will be described. The processing system 1 according to the first modification of one embodiment of the present disclosure comprises robots 10a1, 10a2, ..., 10aM, an optimization device 20, and trays 30a1, 30a2, ..., 30aN, similar to the processing system 1 according to one embodiment of the present disclosure.

[0055] The optimization device 20, like the optimization device 20 according to one embodiment of the present disclosure, includes an operation input unit 201, a communication interface unit 202, an arithmetic processing unit 203, a storage unit 204, and a control unit 205. The storage unit 204, like the storage unit 204 according to one embodiment of the present disclosure, stores at least operation definition information I1, optimization range definition information I2, constraint information I3, objective function information I4, and a program I5.

[0056] However, in the processing system 1 according to one embodiment of the present disclosure, the operating state of the robot 10 and the next operation are associated in the operation definition information I1. In contrast, in the processing system 1 according to the first modified example of one embodiment of the present disclosure, information related to operating conditions is added to the operation definition information I1, and among the associations between the operating state of the robot 10 and the next operation, some are associated by operating conditions. Furthermore, the operation state acquisition unit 2031 according to the first modified example of one embodiment of the present disclosure acquires information on the next operation based on the operating state and operating conditions of the robot 10. In other words, the processing system 1 according to the first modified example of one embodiment of the present disclosure differs from the processing system 1 according to one embodiment of the present disclosure in that it adds operating conditions to the operation definition information I1, and the operation state acquisition unit 2031 acquires information on the next operation based on the operating state and operating conditions.

[0057] Figure 6 shows an example of operation definition information I1 according to some embodiments of the present disclosure. As shown in Figure 6, operation definition information I1 is an addition of operation conditions to the operation definition information I1 shown in Figure 3. In addition, operation definition information I1 adds the presence or absence of a next operation to the operation condition for completion of place. For example, if there is a next operation, a reach operation is defined as the next operation. If there is no next operation, an initialization operation is defined as the next operation.

[0058] The operation definition information I1 shown in Figure 6 indicates that when the operation state is "placement complete," the operation state acquisition unit 2031 determines whether there is a next operation or not. If the operation state acquisition unit 2031 determines that there is a next operation, it acquires the "reach" operation as the next operation. If the operation state acquisition unit 2031 determines that there is no next operation, it acquires the "initialization" operation as the next operation.

[0059] (Advantages) A ​​processing system 1 according to a first modification of one embodiment of the present disclosure has been described. The processing system 1 can appropriately handle cases where the transition of operations differs depending on the execution state and operating conditions of the work. That is, even in cases where the transition of the next operation of the robot 10 differs depending on the operating conditions in addition to the operating state of the robot 10, the processing system 1 can be applied to prevent a significant increase in the calculation time for optimization and to maintain a high work efficiency of the robot 10.

[0060] <Second Modification of the Present Disclosure> Next, a processing system 1 according to a second modification of one embodiment of the present disclosure will be described. The processing system 1 according to the second modification of one embodiment of the present disclosure, like the processing system 1 according to one embodiment of the present disclosure and the processing system 1 according to the first modification of one embodiment of the present disclosure, comprises robots 10a1, 10a2, ..., 10aM, an optimization device 20, and trays 30a1, 30a2, ..., 30aN.

[0061] Figure 7 shows an example of the configuration of an optimization device 20 according to several embodiments of the present disclosure. As shown in Figure 7, the optimization device 20 includes an operation input unit 201, a communication interface unit 202, an arithmetic processing unit 203, a storage unit 204, and a control unit 205, similar to the optimization device 20 according to one embodiment of the present disclosure and a first modified example of one embodiment of the present disclosure.

[0062] As shown in Figure 7, the arithmetic processing unit 203 includes an operating state acquisition unit 2031, an optimization range determination unit 2032, an optimization condition information creation unit 2033, and an optimization calculation unit 2034, similar to the arithmetic processing unit 203 in one embodiment of the present disclosure and the first modified example of one embodiment of the present disclosure. Furthermore, as shown in Figure 7, the arithmetic processing unit 203 also includes an optimization range definition generation unit 2035.

[0063] The optimization range definition generation unit 2035 creates or updates optimization range definition information I2, which indicates which state transitions in a series of operation procedures for each robot 10 to perform a task should be optimized. Specifically, for each of the operation states and conditions of the multiple robots 10, the optimization range definition generation unit 2035 determines which tasks each robot 10 must complete before including them in the optimization calculation range.

[0064] Figure 8 shows an example of the processing flow of the processing system 1 according to several embodiments of the present disclosure. The processing flow shown in Figure 8 shows the operation of updating the optimization range definition information I2 by the optimization range definition generation unit 2035 according to a second modification of one embodiment of the present disclosure. Next, the process by which the processing system 1 updates the optimization range definition information I2 will be described.

[0065] First, the optimization range definition generation unit 2035 acquires the initial state, which is the operating state of each robot 10 (step S11). For example, the optimization range definition generation unit 2035 refers to the operation definition information I1. Then, the optimization range definition generation unit 2035 acquires the operating state and operating conditions that each robot 10 can take from the operation definition information I1.

[0066] Next, the optimization range definition generation unit 2035 repeatedly performs the following operations for each combination of operating states of each robot 10. For example, if there are four possible operating states for two robots 10, there are 4 x 4 = 16 possible combinations of operating states. Therefore, the optimization range definition generation unit 2035 repeatedly performs the following operations for all 16 possible combinations of operating states.

[0067] The optimization range definition generation unit 2035 calculates an operation plan for the task to be calculated, using the combination of operating states of each robot 10 as the initial state (step S12). At this time, the work range that the optimization range definition generation unit 2035 targets for operation planning only needs to be long enough to define the optimization range. Specifically, for example, the work range that the optimization range definition generation unit 2035 targets for operation planning may be defined as the range until each robot 10 completes two operations.

[0068] Next, the optimization range definition generation unit 2035 extracts the time at which a state transition occurred for each of the robots 10 from the calculation results of the motion plan (step S13). The time at which a state transition occurred here is the time at which the transition to the "next motion" occurred in the state defined in the motion definition information I1.

[0069] Next, the optimization range definition generation unit 2035 extracts locations where the occurrence times of state transitions are close together for all robots 10 that are the target of the work (step S14). The details of the extraction operation for extracting close locations will be explained below with reference to Figures 9 to 11.

[0070] Figure 9 shows an example of the calculation result of the motion plan in updating the optimization range definition information I2 according to some embodiments of the present disclosure. Figure 9 shows an example of the calculation result of the motion plan when each of the robots 10 is in its initial state.

[0071] Figure 10 shows an example of the extraction results in updating the optimization range definition information I2 according to some embodiments of the present disclosure. In the specific example shown in Figure 10, in the motion plan, the optimization range definition generation unit 2035 extracts a set of state transitions for each robot 10 when the time slot 401 in which state transitions occur for all robots 10 is smallest. In this embodiment, when each robot 10 is in its initial state, the optimization range definition generation unit 2035 extracts the state transition from "Place (approach)" to "Place (attach)" for robot 10a1, and the state transition from "Pick (grasp)" to "Transport" for robot 10a2.

[0072] The optimization range definition generation unit 2035 then defines the state of each robot 10 corresponding to the extraction location before the transition as the optimization range corresponding to each operating state (step S15). For example, for the extraction location shown in Figure 10, if the state of each robot 10 is in the initial state, the optimization range for robot 10a1 is defined as "Place (approach) complete", and the optimization range for robot 10a2 is defined as "Pick (grasp) complete". The optimization range definition generation unit 2035 then updates the optimization range definition information I2.

[0073] Subsequently, the optimization range definition generation unit 2035 repeats the operations from step S12 to step S15 for each combination of possible operating states for each robot 10, thereby completing the update operation.

[0074] (Advantages) The processing system 1 according to the second modification of this disclosure can determine a suitable optimization range according to the work and the robot configuration, and perform optimization calculations based on this range. By determining the optimization range so that the work of multiple robots is completed at the same time as much as possible, the processing system 1 can minimize the time that the robots are stopped between the completion of one task and the start of the next task. Therefore, it is possible to prevent a significant increase in the calculation time for optimization and to maintain a high level of robot work efficiency.

[0075] In step S14, which extracts locations where the occurrence times of state transitions are close together for all robots 10 that are the target of work, the optimization range definition generation unit 2035 may pre-determine the target time range. Figure 11 is a diagram showing an example of the time range extracted in updating the optimization range definition information I2 according to some embodiments of the present disclosure. For example, the optimization range definition generation unit 2035 may extract locations where the occurrence times of state transitions are close together within the range of the time slot 402 shown in Figure 11.

[0076] This processing system 1 makes it possible to determine a suitable optimization range while keeping the time required for optimization calculations within an appropriate range. However, if the point where the state transition occurs most closely is too close to the start of the work, the frequency of optimization calculations may increase, which is undesirable. Conversely, if the point is too far from the start of the work, the time required for optimization calculations may increase, which is also undesirable. By limiting the extraction target to a predetermined range of time slots, processing system 1 can keep the time step to the optimization range within a predetermined range and appropriately control the time required for optimization calculations.

[0077] A processing system 1 according to some embodiments of the present disclosure will be described. Figure 12 is a diagram showing an example of the configuration of a processing device 500 according to some embodiments of the present disclosure. As shown in Figure 12, the processing device 500 according to some embodiments of the present disclosure comprises a determination means 501 and a calculation means 502.

[0078] The determination means 501 determines an optimization range that indicates which of the multiple actions to be optimized in a series of actions for each of one or more robots, which include multiple actions for performing a task. The calculation means 502 performs an optimization calculation that satisfies constraints in the execution of the task, which includes at least the execution of the task within the optimization range, and minimizes an objective function that indicates the time required for the task.

[0079] The determination means 501 can be implemented, for example, using the functions of the optimization range determination unit 2032 illustrated in Figure 2. The calculation means 502 can be implemented, for example, using the functions of the optimization calculation unit 2034 illustrated in Figure 2.

[0080] Next, the processing performed by the processing apparatus 500 according to some embodiments of the present disclosure will be described. Figure 13 is a diagram showing an example of the processing flow of the processing apparatus 500 according to some embodiments of the present disclosure. Here, the processing performed by the processing apparatus 500 will be described with reference to Figure 13.

[0081] The determination means 501 determines an optimization range that indicates which of the multiple actions to be optimized in a series of actions for each of one or more robots, which include multiple actions for performing the task (step S101). The calculation means 502 performs an optimization calculation that satisfies constraints in the execution of the task, which includes at least the execution of the task within the optimization range, and minimizes an objective function that indicates the time required for the task (step S102).

[0082] Such a processing device 500 can identify the state of an object that is specularly reflected.

[0083] In addition, the order of processing in the embodiments of this disclosure may be changed, as long as appropriate processing is performed.

[0084] Although embodiments of this disclosure have been described, the processing system 1, robot 10, optimization device 20, and other control devices described above may have a computer device inside. The processing steps described above are stored in the form of a program on a computer-readable recording medium, and the processing is performed when the computer reads and executes this program. A specific example of a computer is shown below.

[0085] Figure 14 is a schematic block diagram showing the configuration of a computer according to at least one embodiment. As shown in Figure 14, the computer 5 comprises a CPU (Central Processing Unit) 6, main memory 7, storage 8, and interface 9. For example, the processing system 1, robot 10, optimization device 20, and other control devices are each implemented in the computer 5. The operation of each processing unit described above is stored in the storage 8 in the form of a program. The CPU 6 reads the program from the storage 8, loads it into the main memory 7, and executes the above processing according to the program. The CPU 6 also allocates storage areas in the main memory 7 corresponding to each of the storage units described above according to the program.

[0086] Examples of storage 8 include HDD (Hard Disk Drive), SSD (Solid State Drive), magnetic disk, magneto-optical disk, CD-ROM (Compact Disc Read Only Memory), DVD-ROM (Digital Versatile Disc Read Only Memory), and semiconductor memory. Storage 8 may be an internal medium directly connected to the bus of the computer 5, or an external medium connected to the computer 5 via interface 9 or a communication line. Furthermore, if this program is distributed to the computer 5 via a communication line, the computer 5 that receives the distribution may expand the program into main memory 7 and execute the above processing. In at least one embodiment, storage 8 is a tangible storage medium that is not temporary.

[0087] Furthermore, the above program may implement some of the functions described above. Moreover, the above program may be a file that can implement the above functions in combination with a program already recorded on the computer device, a so-called differential file (differential program).

[0088] While several embodiments of this disclosure have been described, these embodiments are illustrative and do not limit the scope of the disclosure. These embodiments may be modified in various ways, without departing from the gist of the disclosure.

[0089] Furthermore, some or all of the above embodiments may also be described as follows, but are not limited to these.

[0090] (Note 1) A processing apparatus comprising: determination means for determining an optimization range that indicates which of the multiple actions to be optimized in a series of actions for each of one or more robots that include multiple actions for performing an action; and calculation means for performing an optimization calculation that satisfies constraints in the execution of the action, which includes at least the execution of the action within the optimization range, and minimizes an objective function that indicates the time required for the action.

[0091] (Appendix 2) The processing apparatus according to Appendix 1, comprising acquisition means for acquiring the operating state of each of the one or more robots at the start of the optimization calculation.

[0092] (Note 3) The processing apparatus according to Note 2, wherein the determination means determines the optimization range based on optimization range definition information that uniquely associates the operating state of each of the one or more robots with the optimization range of each of the one or more robots.

[0093] (Appendix 4) The processing apparatus according to Appendix 3, comprising a generation means for generating the optimization range definition information.

[0094] (Note 5) The processing apparatus according to Note 4, wherein the generation means performs optimization calculations in advance for the work to be calculated and the operating state of each of the one or more robots, and generates the optimization range definition information based on the operating transitions when the times of the operating transitions in which all of the work of the one or more robots are completed within a predetermined time difference are close together.

[0095] (Note 6) The processing apparatus according to Note 5, wherein the generation means generates the optimization range definition information based on the fact that the operation transition is included within a predetermined time range.

[0096] (Note 7) The processing apparatus according to any one of Notes 1 to 6, wherein the determination means determines the optimization range such that, when operation is started from the operating state of each of the one or more robots, all of the work of the one or more robots is completed within a predetermined time difference.

[0097] (Note 8) The processing apparatus according to any one of Notes 1 to 7, wherein the determination means determines the optimization range based on the operating conditions of each of the one or more robots.

[0098] (Note 9) The processing apparatus according to any one of Notes 1 to 8, comprising control means for controlling one or more robots based on the optimization calculation performed by the calculation means.

[0099] (Note 10) A processing method comprising: determining an optimization range that indicates which of the multiple actions to be optimized in a series of actions for each of one or more robots that include multiple actions for performing a task; and performing an optimization calculation that satisfies constraints in the execution of the task, which includes at least the execution of the task within the optimization range, and minimizes an objective function that indicates the time required for the task.

[0100] (Note 11) The processing method according to Note 10, which includes obtaining the operating state of each of the one or more robots at the start of the optimization calculation.

[0101] (Appendix 12) The processing method according to Appendix 11, which includes determining the optimization range based on optimization range definition information that uniquely associates the operating state of each of the one or more robots with the optimization range of each of the one or more robots.

[0102] (Note 13) The processing method according to Note 12, which includes generating the optimization range definition information.

[0103] (Note 14) The processing method according to Note 13, which includes performing an optimization calculation in advance for the work to be calculated and the operating state of each of the one or more robots, and generating the optimization range definition information based on the operating transitions when the times of the operating transitions in which all of the work of the one or more robots are completed within a predetermined time difference are close together.

[0104] (Note 15) The processing method according to Note 14, which includes generating the optimization range definition information based on the fact that the operation transition falls within a predetermined time range.

[0105] (Note 16) A processing method according to any one of Notes 10 to 15, which includes determining the optimization range so that when operation is started from the operating state of each of the one or more robots, all of the work of the one or more robots is completed within a predetermined time difference.

[0106] (Note 17) A processing method according to any one of Notes 10 to 16, which includes determining the optimization range based on the operating conditions of each of the one or more robots.

[0107] (Note 18) A processing method according to any one of Notes 10 to 17, which includes controlling one or more robots based on the optimization calculation.

[0108] (Note 19) A recording medium storing a program that causes a computer to perform the following: determining an optimization range that indicates which of the multiple actions to be optimized in a series of actions for each of one or more robots that include multiple actions for performing a task; and performing an optimization calculation that satisfies constraints in the execution of the task, which includes at least the execution of the task within the optimization range, and minimizes an objective function that indicates the time required for the task.

[0109] (Note 20) A recording medium described in Note 19, which stores a program that causes the computer to perform the following actions at the start of the optimization calculation: acquiring the operating state of each of the one or more robots.

[0110] (Note 21) A recording medium according to Note 20, which stores a program that causes the computer to perform the following: determine the optimization range based on optimization range definition information that uniquely associates the operating state of each of the one or more robots with the optimization range of each of the one or more robots.

[0111] (Note 22) The recording medium described in Note 21, which stores a program that causes the computer to perform the task of generating the optimization range definition information.

[0112] (Note 23) A recording medium according to Note 22 that stores a program that causes the computer to perform optimization calculations in advance for the work to be calculated and the operating state of each of the one or more robots, and to generate the optimization range definition information based on the operating transitions when the times of the operating transitions in which all the work of the one or more robots are completed within a predetermined time difference are close together.

[0113] (Note 24) The recording medium described in Note 23, which stores a program that causes the computer to perform the following actions: generating the optimization range definition information based on the fact that the operation transition falls within a predetermined time range.

[0114] (Note 25) A recording medium described in any one of Notes 19 to 24, which contains a program that causes the computer to perform the following: when operation is started from the operating state of each of the one or more robots, determine the optimization range so that all of the tasks of the one or more robots are completed within a predetermined time difference.

[0115] (Note 26) A recording medium described in any one of Notes 17 to 25, which stores a program that causes the computer to perform the operation of determining the optimization range based on the operating conditions of each of the one or more robots.

[0116] (Note 27) A recording medium described in any one of Notes 17 to 26, which stores a program that causes the computer to perform the operation of controlling the one or more robots based on the optimization calculation.

[0117] According to each aspect of this disclosure, each of the multiple controlled objects can be operated efficiently without each of the multiple controlled objects adversely affecting the other controlled objects.

[0118] 1... Processing system 10a1, 10a2, 10aM... Robot 20... Optimization device 30... Tray 201... Operation input unit 202... Communication interface unit 203... Calculation processing unit 2031... Operation status acquisition unit 2032... Optimization range determination unit 2033... Optimization condition information creation unit 2034... Optimization calculation unit 204... Storage unit I1... Operation definition information I2... Optimization range definition information I3... Constraint information I4... Objective function information I5... Program

Claims

1. A processing apparatus comprising: a determination means for determining an optimization range that indicates which of the multiple actions to be optimized in a series of actions for each of one or more robots, which include multiple actions for performing a task; and a calculation means for performing an optimization calculation that satisfies constraints in the execution of the task, which includes at least the execution of the task within the optimization range, and minimizes an objective function that indicates the time required for the task.

2. The processing apparatus according to claim 1, comprising acquisition means for acquiring the operating state of each of the one or more robots at the start of the optimization calculation.

3. The processing apparatus according to claim 2, wherein the determination means determines the optimization range based on optimization range definition information that uniquely associates the operating state of each of the one or more robots with the optimization range of each of the one or more robots.

4. The processing apparatus according to claim 3, comprising a generation means for generating the optimization range definition information.

5. The processing apparatus according to claim 4, wherein the generation means performs optimization calculations in advance for the work to be calculated and the operating state of each of the one or more robots, and generates the optimization range definition information based on the operating transitions when the times of the operating transitions in which all of the work of the one or more robots are completed within a predetermined time difference are close together.

6. The processing apparatus according to claim 5, wherein the generation means generates the optimization range definition information based on the fact that the operation transition is included within a predetermined time range.

7. The processing apparatus according to any one of claims 1 to 6, wherein the determination means determines the optimization range such that, when operation is started from the operating state of each of the one or more robots, all of the work of the one or more robots is completed within a predetermined time difference.

8. The processing apparatus according to any one of claims 1 to 7, comprising: control means for controlling one or more robots based on the optimization calculation performed by the calculation means.

9. A processing method comprising: determining an optimization range that indicates which of the multiple actions to be optimized in a series of actions for each of one or more robots, which include multiple actions for performing a task; and performing an optimization calculation that satisfies constraints in the execution of the task, which includes at least the execution of the task within the optimization range, and minimizes an objective function that indicates the time required for the task.

10. A recording medium containing a program that causes a computer to perform the following: determine an optimization range indicating which of the multiple actions to be optimized in a series of actions for each of one or more robots, which include multiple actions for performing a task; and perform an optimization calculation that satisfies constraints in the execution of the task, which includes at least the execution of the task within the optimization range, and minimizes an objective function indicating the time required for the task.

Images