Robot system, operation planning method, and program

Abstract

The present invention contributes to reducing calculation time over the entirety of generation of an operation plan for robot control. This robot system comprises: an abstraction unit that creates a first model by aggregating tasks from an initial model created by abstracting information necessary for generating an operation plan for a robot, the information including a task to be executed by the robot; a first generation unit for generating a first operation plan using the first model; and a plan generation unit for generating an operation plan using the initial model from the first operation plan.

Description

Robot system, motion planning method and program

[0001] The present disclosure relates to a robot system, a motion planning method, and a program.

[0002] There are techniques for reducing the amount of calculations required to plan the operation of a robot and shortening the calculation time. For example, Patent Document 1 discloses a technique in which a control device for controlling the operation of a work robot that uses a manipulator to perform work within a target area includes a trajectory information acquisition unit that acquires N-1 or N pieces of trajectory information indicating N-1 or N trajectories that connect N work areas in which the work robot performs a series of tasks in the order of the series of tasks, a classification unit that classifies the N-1 or N trajectories into trajectories that require correction and trajectories that do not require correction, and a trajectory planning unit that plans a trajectory of the tip of the manipulator between two work areas associated with each trajectory, for each of the one or more trajectories classified by the classification unit as trajectories that require correction.

[0003] Special Publication No. 2020-062690

[0004] The following analysis was performed by the inventors of the present disclosure.

[0005] In the calculation of a robot's motion plan, the technology described in Patent Document 1 reduces the amount of calculation by narrowing the scope of calculation when recalculating if any changes are made. However, there is no description of the calculation of the motion plan that is initially generated, and a huge amount of calculation is still required.

[0006] The present disclosure has been made in consideration of the above circumstances, and one of its objectives is to provide a technology that contributes to reducing calculation time across the entire generation of motion plans for robot control.

[0007] According to a first aspect of the present disclosure, there is provided a robot system including: an abstraction unit that aggregates tasks from an initial model created by abstracting information including tasks to be executed by a robot, the initial model being information necessary for generating an operation plan for the robot, to create a first model; a first generation unit that generates a first operation plan using the first model; and a plan generation unit that generates an operation plan from the first operation plan using the initial model.

[0008] According to a second aspect of the present disclosure, there is provided a motion planning method executed by a computer mounted on a robot system, the motion planning method comprising: generating a first model by aggregating the tasks from an initial model created by abstracting information including tasks to be executed by the robot and necessary for generating a motion plan for the robot; generating a first motion plan using the first model; and generating a motion plan from the first motion plan using the initial model.

[0009] According to a third aspect of the present disclosure, there is provided a program for causing a computer to execute the following steps: creating a first model by aggregating the tasks from an initial model created by abstracting information including tasks to be executed by a robot and necessary for generating an operation plan for the robot; generating a first operation plan using the first model; and generating an operation plan from the first operation plan using the initial model.

[0010] These programs can be recorded on a computer-readable storage medium. The storage medium can be a non-transitory medium such as a semiconductor memory, a hard disk, a magnetic recording medium, or an optical recording medium. The present disclosure can also be embodied as a computer program product.

[0011] The present disclosure can contribute to reducing calculation time across the entire generation of motion plans for robot control.

[0012] 1A is a diagram of an example of functional blocks of a robot system according to the present disclosure, (b) is a diagram for explaining an overview of motion plan generation by a task motion planner according to the present disclosure, and (c) is a diagram of an example of functional blocks of a task motion planner according to the present disclosure. (a) and (b) are diagrams for explaining an example of task aggregation according to the present disclosure. (b) is a diagram for explaining an example of first motion plan division according to the present disclosure. (c) is a flowchart of motion plan generation processing according to the present disclosure. (a) is a diagram of an example of the configuration of a robot system according to the present disclosure and functional blocks of a task motion planner, and (b) is a diagram for explaining an example of data stored in a storage device of a robot system according to the present disclosure. (a) is a diagram for explaining an example of use of a robot system according to the present disclosure, and (b) is a diagram for explaining an example of a motion plan. (a) is a diagram for explaining an example of a task motion planner, and (b) and (c) are diagrams for explaining an example of modeling. (a) is a diagram for explaining an example of an initial model according to the present disclosure, (b) is a diagram for explaining an example of a first model according to the present disclosure, and (c) is a diagram for explaining another example of the first model according to the present disclosure. (a) and (b) are diagrams for explaining an example of operation plan generation according to the present disclosure. (a) to (c) are diagrams for explaining an example of constraint conditions according to the present disclosure. A flowchart of operation plan generation processing according to the present disclosure. (a) and (b) are diagrams for explaining a modified example of a technique for dividing a first operation plan according to the present disclosure. (a) and (b) are diagrams for explaining a modified example of a technique for generating an operation plan according to the present disclosure. A hardware configuration diagram of an example of a hardware configuration of a robot system according to the present disclosure.

[0013] An overview of one embodiment of the present disclosure (hereinafter referred to as the present embodiment) will be described below with reference to the drawings. Note that reference numerals in the drawings are assigned to each element for convenience as an example to facilitate understanding, and are not intended to limit the present disclosure to the illustrated form. Furthermore, connecting lines between blocks in the drawings and the like referred to in the following description include both bidirectional and unidirectional lines. Unidirectional arrows are used to schematically indicate the flow of a main signal (data) and do not exclude bidirectionality.

[0014] In addition, although there are ports and interfaces at the connection points of input and output of each block in the figure, they are not shown. In the following explanation, "A and / or B" means either A or B, or A and B.

[0015] First Embodiment An overview of this embodiment will be described. A robot system of this embodiment automatically generates optimal work procedures and control commands for achieving a target state by simply providing a target state for the robot to execute.

[0016] 1A is a functional block diagram of a robot system 100 according to this embodiment. When a user provides a target state S1, the robot system 100 generates a motion plan S3 for the robot 600 using pre-stored data S2. The robot system 100 then generates a control instruction sequence S4 from the motion plan and supplies the control instruction sequence S4 to the robot 600. Hereinafter, the pre-stored data will be simply referred to as stored data S2.

[0017] The motion plan S3 may further include sensor data. The sensor data is data within the workspace collected by sensors installed in the workspace or sensors included in the robot system 100.

[0018] To achieve this, the robot system 100 of this embodiment includes a task motion planner 200 and a robot controller 400.

[0019] When the goal state S1 is given, the task motion planner 200 generates a motion plan S3 using previously stored information (stored data S2) necessary for generating a motion plan for the robot 600. The motion plan S3 includes task allocation and task execution procedures for realizing the goal state, as well as motion trajectories of the position and orientation of the end effector of the robot 600. In other words, the motion plan S3 includes a task plan and a motion plan.

[0020] A task plan is a sequence (task sequence) that describes, for each predetermined time step, which tasks to assign to each robot 600 and the order in which the assigned tasks are to be executed by each robot 600 when a goal state is to be achieved by multiple robots 600. A motion plan is a sequence (motion sequence) that describes, for each time step, the movements of the end effectors of each robot 600 according to the task plan. Hereinafter, the end effectors are also referred to as hands.

[0021] The robot controller 400 generates, from the motion plan S3, a control instruction sequence S4 for controlling the motion of the robot 600. The robot controller 400 obtains information on the position and posture of the hand of the robot 600 from the motion plan in the motion plan S3, and generates the control instruction sequence S4 based on this information. Then, the robot controller 400 outputs control signals to the robot 600 in accordance with the generated control instruction sequence S4. The robot 600 performs its motion in accordance with the control signals output from the robot system 100.

[0022] When generating the motion plan S3, the task motion planner 200 creates a model that abstracts the information necessary for generating the motion plan into a calculable form. Information necessary for generating the motion plan includes, for example, the shape of the object, the task execution point, and the point of action. Hereinafter, creating an abstract model will also be referred to as modeling. Note that objects to be modeled include hands, obstacles, workpieces, etc. Modeling is performed using the retained data S2. The task motion planner 200 performs optimization calculations to generate the motion plan S3. The initial model is an abstraction of the state of each component so that it can be defined as a constraint equation for the optimization problem during optimization calculations. Note that the motion plan is generated in a predetermined time unit. Hereinafter, the time unit for generating the motion plan will be referred to as the time step length.

[0023] The lower the level of abstraction (degree of abstraction), the greater the number of elements that make up the model. Furthermore, the more elements that make up the model and the shorter the time step length, the greater the amount of calculation required. The degree of abstraction may refer to the degree to which the amount of data is reduced by abstraction. Data that can be used to create a model that can generate a motion plan with the desired accuracy is stored as retained data S2. However, as tasks become more complex and the number of robots to be controlled increases, the amount of calculation required to obtain a motion plan with the desired accuracy becomes enormous.

[0024] In order to reduce the amount of calculation required to generate the motion plan S3, the task motion planner 200 of this embodiment creates a simplified model (hereinafter referred to as a first model) at a higher level of abstraction from a model (hereinafter referred to as an initial model) created using the stored data S2. The first model is then used to generate an overall motion plan (hereinafter referred to as a first motion plan). The scope of the first motion plan is then limited, and a motion plan is generated for each limited scope using the initial model. Finally, the generated motion plans are combined to generate the motion plan to be output.

[0025] An overview will be explained using FIG. 1(b). As shown in this figure, the task motion planner 200 first creates an initial model from the retained data S2. Then, for example, by aggregating tasks in the initial model, the initial model is abstracted to create a first model. At this time, the time step length may be increased. A first motion plan is then generated using the first model. That is, as shown in FIG. 1(b), instead of generating a motion plan to achieve a goal state using an initial model with a low level of abstraction, the task motion planner 200 generates a first motion plan as an overall motion plan using a first model with a higher level of abstraction than the initial model. The generated first motion plan is then divided, and each divided motion plan is recalculated using the tasks included in the motion plan before aggregation as target tasks. The recalculated motion plans are then combined to generate a motion plan.

[0026] To achieve this, the task motion planner 200 of this embodiment includes an abstraction unit 210, a first generation unit 220, and a plan generation unit 230, as shown in FIG.

[0027] The abstraction unit 210 aggregates the tasks to be executed by the robot 600 from an initial model that abstracts the retained data S2, which is information necessary for generating an operation plan for the robot 600, and creates a first model by aggregating the tasks.

[0028] The retained data S2 includes the shapes of the hand, obstacle, and workpiece, the point of application of the hand, the working point of the task, the initial position of the workpiece, the target position of the workpiece, environmental information such as a map of the workspace, constraints, the degree of abstraction used when creating the initial model, the time step length used, etc. As described above, the retained data S2 is set so that a motion plan with desired accuracy can be obtained using the created initial model. Furthermore, this information is stored in advance in a storage device of the robot system 100. Note that this information may also be stored in an external storage device accessible by the robot system 100.

[0029] An initial model that abstracts this information is created in advance. Abstraction is the process of replacing information with less data. In other words, the creation of the initial model simplifies the complex object shapes and physical behaviors handled by the robot system 100 to make them easier to handle.

[0030] The abstraction unit 210 of this embodiment aggregates tasks to create a first model that further enhances the level of abstraction of the created initial model. That is, the abstraction unit 210 aggregates tasks to enhance the level of abstraction. At this time, the abstraction unit 210 aggregates tasks according to predetermined rules. For example, multiple tasks whose inter-workpoint distances are within a predetermined threshold are aggregated into a single task. The rules and thresholds are stored in a storage device. In addition, information such as the required time and execution order of the tasks before aggregation within each aggregated task is also stored in the storage device.

[0031] A specific example is shown in Fig. 2(a). Here, the explanation will be given taking as an example tasks with work points 811 to 816. Of these tasks, the tasks with work points 811 and 812 are aggregated into one task, which is designated as work point 821. Furthermore, the four tasks with work points 813 to 816 are aggregated into one task, which is designated as work point 822. The abstraction unit 210 performs this processing based on the position information of the work points of each task that is stored in advance in a storage device.

[0032] As shown in FIG. 2( a), classification based only on distance may result in a bias in the allocation of tasks to a particular robot 600. Therefore, in order to reduce the bias in task allocation, weighting may be applied to each task or an upper limit on the number of tasks to be aggregated may be set. For example, as shown in FIG. 2( b), tasks may be aggregated taking into consideration the balance of task execution time. In this case, the execution time of each task, the upper limit on the number of tasks to be aggregated at the time of aggregation, etc. are stored in a storage device.

[0033] The first generation unit 220 generates a first motion plan using the first model. The task motion planner 200 of this embodiment has a function of formulating each element constituting the model using a linear constraint equation including integer variables and solving a mixed integer linear problem to generate a motion plan. In this embodiment, the first generation unit 220 uses this function to formulate each element constituting the first model and generate a first motion plan.

[0034] The plan generating unit 230 generates an operation plan from the first operation plan using an initial model. In this embodiment, for example, as shown in FIG. 3 , the obtained first operation plan is divided in the time direction to obtain divided periods 700-1, 700-2, and 700-3. Hereinafter, unless a particular distinction is required, the divided period will be represented by the divided period 700. Then, for each first operation plan (hereinafter referred to as a partial plan) for each divided period 700, the plan generating unit 230 recalculates an operation plan using the initial model based on the partial plan. In this embodiment, for the pre-aggregation tasks included in the partial plans for each divided period 700, an operation plan for that divided period 700 is calculated. In other words, in this embodiment, the plan generating unit 230 limits the time range and target tasks and recalculates the operation plan at the abstraction level of the initial model. Finally, an operation plan is generated by combining the recalculated operation plans in the chronological order of the original divided period 700.

[0035] FIG. 3 illustrates an example of generating an operation plan for two robots 600 (hand 1 and hand 2). FIG. 3 also shows only a task plan (task sequence) in which the tasks to be executed by each robot 600 are arranged along the time axis. Here, tasks A to F are respectively aggregated tasks (hereinafter referred to as aggregated tasks), i.e., first model tasks. Furthermore, tasks a to e are initial model tasks aggregated into task B (hereinafter referred to as pre-aggregation tasks). Tasks f to k are pre-aggregation tasks aggregated into task E.

[0036] For example, the partial plan for divided period 700-2 includes a portion of post-aggregation task B to be executed by hand 1 and a portion of post-aggregation task E to be executed by hand 2. Post-aggregation task B in divided period 700-2 includes pre-aggregation tasks a to d, and post-aggregation task E includes pre-aggregation tasks h to k. For divided period 700-2, the plan generation unit 230 recalculates the motion plans for pre-aggregation tasks a to d and h to k included in this period using the initial model.

[0037] The division method of the first motion plan is not limited to this. When recalculating using the initial model, the division may be performed in real time. The division direction is not limited to the time direction, and may be performed spatially, for example.

[0038] [Motion Plan Generation Processing] The flow of the motion plan generation processing of this embodiment is shown in Fig. 4. As shown in this figure, the task motion planner 200 of this embodiment starts the motion plan generation processing in response to an instruction from the user.

[0039] The abstraction unit 210 creates a first model by aggregating tasks from a previously created initial model in accordance with a predetermined rule (step S1101).

[0040] The first generator 220 generates a first operation plan using the created first model (step S1102).

[0041] The plan generating unit 230 generates an operation plan from the first operation plan using the initial model (step S1103), outputs it to the robot controller 400, and ends the process.

[0042] As described above, the robot system 100 of this embodiment aggregates tasks to be executed by the robot 600 from an initial model created using pre-stored information, which is information necessary for planning the robot 600's motion, to create a first model that further abstracts the initial model. A rough motion plan is then generated using the first model with a higher level of abstraction. Then, for each partial plan obtained by dividing the rough motion plan, a detailed plan is recalculated using the initial model for the pre-aggregation tasks included in the partial plan. Finally, the obtained detailed plans are combined to create a motion plan. Based on the motion plan output from the task motion planner 200, the robot controller 400 generates a command sequence for controlling the robot 600 and operates the robot 600.

[0043] As described above, according to this embodiment, first, an overall motion plan is generated using a model with a higher level of abstraction than the initial one. Then, a detailed motion plan is generated using the model with the initial level of abstraction by limiting the target range and target task. In this way, the motion plan for the robot 600 is generated in two stages. In generating the motion plan at each stage, a mixed integer linear problem is solved with fewer constraints, so the amount of calculation can be reduced at each stage. This also shortens the calculation time for the overall motion plan.

[0044] That is, according to this embodiment, it is possible to reduce the calculation time for generating an overall motion plan for controlling a robot.

[0045] <<Second Embodiment>> Next, a second embodiment to which the present disclosure is applied will be described. This embodiment is an embodiment that embodies the first embodiment in more detail. In this embodiment, configurations with the same names as those in the first embodiment basically have the same functions as those in the first embodiment. Hereinafter, this embodiment will be described, focusing on the differences from the first embodiment.

[0046] As shown in Fig. 5(a), the robot system 100 of this embodiment includes a task motion planner 200, a robot controller 400, and an input device 310. The configuration of the robot system 100 is basically the same as that of the first embodiment. The functions of the task motion planner 200 and the robot controller 400 are also the same as those of the first embodiment. The robot system 100 may further include a sensor.

[0047] The robot system 100 of this embodiment is used to control a robot 600 that performs a task of welding a plurality of locations (welding locations: work points 630) on a vehicle body 620, as shown in FIG. 6A, for example. A plurality of robots 600 may work in cooperation with each other. A hand 601 is attached to the tip of each robot 600. Here, as an example, a case where two robots 600_#1 and 600_#2 are used is illustrated. The number of robots 600 is not limited to this. When there is no need to distinguish between the two, they are represented by robot 600.

[0048] As in the first embodiment, the task motion planner 200 of this embodiment generates a motion plan from a given target state and information (retained data S2) necessary for generating a motion plan for the robot 600. That is, the task motion planner 200 first performs modeling, abstracting the target state S1 and retained data S2 into a form that can be handled mathematically. The retained data S2 is the same as in the first embodiment. The task motion planner 200 then formulates the created model using linear constraint equations including integer variables. Then, under the constraint conditions expressed by the obtained constraint equations, the task motion planner 200 solves a mixed integer linear problem and outputs a solution that optimizes the objective function as a motion plan.

[0049] As in the first embodiment, the robot controller 400 generates a control instruction sequence for controlling the robot 600 based on the received operation plan.

[0050] The input device 310 receives instructions from a user. In this embodiment, for example, the input device 310 receives input of a goal state from the user.

[0051] Prior to a detailed description of the task motion planner 200, a functional overview of the task motion planner 200 will be given. As shown in FIG. 7A, the task motion planner 201 includes a model creation unit 202 and a motion plan generation unit 203.

[0052] The model creation unit 202 abstracts the target state S1 input by the user and the retained data S2 including the time step length and object model information to create a model. The retained data S2 is set so that an initial model can be generated that can obtain a motion plan with desired accuracy. The time step length is the unit time when outputting as a motion plan. Hereinafter, the model created from the target state and the retained data S2 will be referred to as the initial model. Note that in the following description of this embodiment, the initial model will be described as including the initial time step length retained as the retained data S2. The initial time step length will be referred to as the initial time step length.

[0053] During abstraction, the model creation unit 202 determines a goal logical formula from the goal state. The goal logical formula is described, for example, in a temporal logic expression. Specifically, when a goal state is presented in which tasks a to c are desired to be performed, this goal state is expressed in temporal expression as "◇ (task a completion state) ∧◇ (task b completion state) ∧◇ (task c completion state)." Here, ◇ is a logical operator meaning that it will eventually become true, and ∧ is a logical AND.

[0054] The model creation unit 202 also abstracts tasks. Here, the execution status of the tasks, the execution procedures of the tasks in each robot 600, and the allocation of tasks to each robot 600 are abstracted. At this time, the model creation unit 202 uses object models stored as object model information. The object models used include, for example, a model of the hand 601 of the robot 600, a model of the workpiece, a working point 630, and a model of an obstacle (for example, the car body 620 in FIG. 6A). These are represented by three-dimensional figures such as a rectangular parallelepiped, a cube, a sphere, a cone, etc., and / or two-dimensional figures such as a polygon, an ellipse, etc.

[0055] An example of these modeling is shown in Figure 7(b). Here, as a task execution state model, an example of a hand-only task, which is a task performed by a hand alone, such as welding, and an example of a workpiece handling task, such as picking up, in which a workpiece is handled by a hand 601, are shown. Also shown are task procedures and assignment models for two robots 600_#1 and 600_#2.

[0056] As an execution state, for example, if the task is welding, a state in which the tip of the model 691 of the hand 601 is away from the welding position, which is a modeled working point (hereinafter referred to as working point model 692), is considered to be unexecuted. Also, a state in which the tip of the model 691 of the hand 601 overlaps with the working point model 692 is considered to be in execution. Also, if the task is picking up, a state in which the tip of the model 691 of the hand 601 and the workpiece model 693 are in the same position is considered to be picking up. Note that whether a task is unexecuted or in execution is represented, for example, by using an execution state variable.

[0057] The model creation unit 202 also models the dynamic characteristics of the robot 600. An example of these models is shown in Fig. 7(c). Here, the dynamic characteristics include a motion model of the hand 601 (model 691) and a collision avoidance model. Reference numeral 694 denotes a model of an obstacle.

[0058] The motion plan generating unit 203 expresses (formulates) the created model as, for example, a linear mathematical expression including integer variables, and then uses the formula as a constraint equation to solve a mixed integer linear problem and generate a motion plan.

[0059] 6B shows an example of the motion plan S3 generated by the motion plan generating unit 203. The motion plan indicates, for example, the tasks to be executed for each robot 600 and for each time step, the position and posture of the hand 601, etc.

[0060] [Function Blocks] Next, detailed functions of the task motion planner 200 of this embodiment will be described. The task motion planner 200 of this embodiment first generates a motion plan (first motion plan) with a long time step length using a model (first model) with a higher level of abstraction than the initial model. Next, the time range and target task are limited, and motion plans are recalculated with an initial time step length using the initial model. The recalculated motion plans are then combined to generate a motion plan to be output to the robot controller 400. Here, a high level of abstraction means that the amount of data (number of components) has been reduced by abstraction compared to the initial model.

[0061] 5A, the task motion planner 200 of this embodiment includes an initial model creation unit 240, an abstraction unit 210, a first generation unit 220, a plan generation unit 230, and a storage device 330. The plan generation unit 230 also includes a division unit 231, a second generation unit 232, and a combination unit 233.

[0062] The storage device 330 stores data necessary for the task motion planner 200 to generate a motion plan, data generated when the motion plan is generated, etc. The storage device 330 may be provided outside the task motion planner 200.

[0063] The initial model creation unit 240 creates an initial model. Here, the initial model is a model created by the model creation unit 202 of the task motion planner 201. That is, similar to the first embodiment, the initial model is information including tasks to be executed by the robot 600, and is a model created by abstracting information necessary for generating a motion plan for the robot 600. Therefore, the function of the initial model creation unit 240 is the same as that of the model creation unit 202. The initial model creation unit 240 stores data of the created initial model in the storage device 330.

[0064] As in the first embodiment, the abstraction unit 210 performs a conversion process to create a first model from an initial model by aggregating tasks in accordance with predetermined creation rules. The creation rules used when creating the first model are stored in advance in the storage device 330. For example, the user registers the creation rules in the storage device 330 via the input device 310.

[0065] As in the first embodiment, aggregating tasks refers to the process of reducing the number of tasks that make up a model, as shown in Figures 2(a) and 2(b). For example, multiple tasks whose inter-operation point distances are within a predetermined threshold may be aggregated into one task, or tasks may be aggregated in consideration of the balance of task execution times.

[0066] In this embodiment, not only the tasks but also the time step length, the shape of the object, candidate patterns of the object pose, etc. may be simplified. Specific examples of this case are shown in Figures 8(a) and 8(b).

[0067] Fig. 8(a) shows an example of an initial model. Here, the initial model includes an initial time step length (initial TSL) of Δt, a model 621 of the shape of an obstacle, a model 611 of the shape of a hand 601, a model 631 of the working point (execution point) of the task, and a model 641 showing the point of action of the hand 601. The obstacle corresponds to the car body 620 shown in Fig. 6(a), and its shape is modeled as a combination of multiple rectangular parallelepipeds and truncated pyramids, etc. The shape of the hand 601 approximates a welding hand, and is modeled as multiple rectangular parallelepipeds.

[0068] 8(b) is an example of a first model with a higher level of abstraction than the initial model. Here, the first model includes a first time step length (first TSL) of ΔT, a model 622 of the shape of the obstacle, a model 612 of the shape of the hand, a model 632 of the working point (execution point) of the task, and a model 641 showing the point of action on the hand. The shape of the obstacle is modeled using a single rectangular parallelepiped. The shape of the hand is modeled using a single rectangular parallelepiped.

[0069] As shown in this figure, the first time step length ΔT used when generating the first plan is set to be longer than the initial time step length Δt. Here, an example is shown where ΔT = Δt × 3. Furthermore, while the shapes of the obstacle and hand are represented by multiple rectangular parallelepipeds in the initial model, they are represented by a single rectangular parallelepiped in the first model. By representing the hand 601 by multiple rectangular parallelepipeds in the initial model, multiple posture patterns can also be expressed. However, in the first model, the posture pattern is a single fixed pattern.

[0070] The abstraction unit 210 may perform these abstractions in addition to the aggregation of tasks described above. That is, it performs a conversion process on the modeling target to reduce the number of constraint equations, and creates a first model. These processes reduce the number of components of the model and the number of constraint equations in the mixed integer linear problem to be solved when generating an action plan. This reduces the amount of calculation.

[0071] The abstraction unit 210 may store information about the aggregated tasks, together with their required times, execution order, etc., as aggregated task information in the storage device 330. The abstraction unit 210 may also store, in the storage device 330, other conversion information between the first model and the initial model, such as conversion information about the shape and information about the time step length.

[0072] In addition, for example, when the work is pick-and-place, the abstraction unit 210 may simplify the shape of the workpiece to be worked on. The workpieces may also be aggregated. As shown in FIG. 8C, workpiece aggregation can be achieved by combining a group of workpieces whose initial positions and target positions are close to each other and whose initial postures and target postures are similar, similar to task aggregation. In this case, the shape surrounding the aggregated multiple workpieces is defined as the aggregated workpiece shape.

[0073] As in the first embodiment, the first generator 220 generates a first operation plan using the created first model.

[0074] As described above, the motion plan generation unit 203 of the task motion planner 201 formulates an initial model and uses it as a constraint equation to solve a mixed integer linear problem and calculate a motion plan. The function of the first generation unit 220 of this embodiment is basically the same as that of the motion plan generation unit 203. However, instead of the initial model, a first model is formulated and used as a constraint equation to generate a first motion plan.

[0075] As in the first embodiment, the plan generation unit 230 generates an operation plan from a first operation plan using an initial model. In this embodiment, the first operation plan is divided into multiple parts in the time direction, each of which is a partial plan. Then, for each partial plan, a detailed plan is recalculated using the initial model. Finally, the detailed plans are combined to generate an operation plan.

[0076] That is, the dividing unit 231 divides the first operation plan in the time direction to generate partial plans. Furthermore, the second generating unit 232 generates a detailed plan for each partial plan using the initial model. Here, the second generating unit 232 generates, as the detailed plan, an operation plan for executing tasks included in the partial plan within the time range of the partial plan. The combining unit 233 combines the detailed plans to generate an operation plan.

[0077] [Outline of Operation Plan Generation Method] Here, the processing of the plan generation unit 230 of this embodiment will be explained using a specific example. Hereinafter, the task of task point X will be simply referred to as task X. As shown in Fig. 9(a) , in the first model, task group a to f of the initial model are aggregated into two tasks (tasks A and B). Task A is an aggregate of task groups a and b, and task B is an aggregate of task groups c to f.

[0078] 9(b) is a diagram illustrating examples of a first operation plan 710, a partial plan 720, a detailed plan 730, and an operation plan 740 generated by this first model. Here, Δt is the initial time step length, ΔT is the time step length (first time step length) used when generating the first operation plan 710, and ΔD is the division period.

[0079] 9(b) shows a first operation plan 710 generated using the first model in which task groups a to f are aggregated as described above. Here, as an example, only the execution sequence of the tasks of one robot 600 is shown. As shown in this figure, in the first operation plan 710, task A is executed between ΔT and 2ΔT, and task B is executed between 3ΔT and 6ΔT.

[0080] 9B, the dividing unit 231 divides the first operation plan 710 by a predetermined time interval ΔD to generate partial plans 720 (720a and 720b). Here, an example is shown in which the predetermined time interval ΔD is 3ΔT and the first operation plan 710 is divided into two partial plans 720.

[0081] As shown in FIG. 9B , the second generation unit 232 recalculates the motion plan using the initial model for each partial plan 720. Here, the second generation unit 232 generates a motion plan with an initial time step length Δt to execute tasks a and b within the time range (0 to 3ΔT) of partial plan 720a, resulting in detailed plan 730a. Furthermore, the second generation unit 232 generates a motion plan with an initial time step length Δt to execute tasks c to f within the time range (3ΔT to 6ΔT) of partial plan 720b, resulting in detailed plan 730b. The second generation unit 232 generates these detailed plans 730a and 730b using information related to the conversion stored in the storage device 330. Note that unless there is a particular need to distinguish between them, they will be represented as detailed plan 730.

[0082] As shown in the bottom row of Fig. 9(b), the combining unit 233 combines the detailed plans 730a and 730b to generate an operation plan 740. When combining, the combining unit 233 combines the detailed plans 730a and 730b while maintaining the execution order of the partial plans 720 that were the basis for generating each detailed plan 730. That is, the detailed plan 730b is combined after the detailed plan 730a. At this time, the combining unit 233 sets the configuration so that the final state of the object in the immediately preceding detailed plan 730 becomes the initial state of the next detailed plan 730. The state of the object is, for example, the position, posture, and whether or not the hand is attached, etc.

[0083] [Data in Storage Device] Here, a brief description will be given of the data stored in the storage device 330. As shown in FIG. 5B, the storage device 330 of this embodiment stores retained data S2, an initial model 331, creation rules 332, aggregated task information 333, and conversion information 334. The creation rules 332 are rules that the abstraction unit 210 follows when creating a first model from the initial model 331. The aggregated task information 333 is information on tasks aggregated by the abstraction unit 210. The conversion information 334 is information on shapes and the like that are converted when creating a first model from the initial model 331.

[0084] The retained data S2 includes abstract state designation information, constraint condition information, operation limit information, task information, abstract model information, map information, and the like in addition to the time step length and object model information described above.

[0085] The abstract state designation information is information that designates an abstract state that needs to be defined in order to assign a task to each robot 600. For example, the state of an object according to the type and content of the task is indicated in the form of an abstract state.

[0086] The constraint information indicates constraints for executing a task, and includes, for example, the positional relationship between the robot 600 and obstacles, the definition of the above-mentioned in-execution and unexecuted states, and the like.

[0087] Examples of constraint information are shown in FIGS. 10(a) to 10(c). Here, as shown in FIG. 10(a), an example of constraints is shown in which the shapes of the hand, obstacle, and working area are represented by multiple two-dimensional rectangles. Here, for each set of rectangles, a constraint equation, either Equation 1 or Equation 2, shown in FIG. 10(b), is set. Here, the constraint equation is set so that the object does not pass through the area corresponding to the obstacle. Furthermore, as shown in FIG. 10(c), constraints are defined between adjacent times (t and t+1) for all time steps.

[0088] The motion limit information is information relating to the motion limits of the robot 600. For example, it is the maximum speed and maximum acceleration of the hand 601, requirements for obstacle avoidance, and the like.

[0089] The task information is specific actions required to execute each task. For example, if the task is welding, it includes moving, joining, etc. The task information may include the time required to join each part. It may also include constraints on the execution order between tasks. Furthermore, it may include constraints when multiple robots 600 work together. For example, the task information may be registered in association with information indicating the type or content of the task.

[0090] Abstract model information is information about a model that abstracts dynamics in a workspace. Dynamics here refers to items whose values ​​can change in a model, such as the state of an object. A model that abstracts dynamics in a workspace is also called an abstract model. Abstract model information may be registered in association with information indicating the type or content of a task.

[0091] The object model information is information about a model representing an object in the workspace, and includes, for example, information necessary to recognize the type, position, and / or orientation of each object, as well as three-dimensional shape information such as CAD (Computer Aided Design) data for recognizing the three-dimensional shape of each object.

[0092] The map information is information that indicates a map of the workspace, and more specifically, information that indicates the positions of objects within the workspace.

[0093] [Motion Plan Generation Processing] Here, the flow of the motion plan generation processing of this embodiment will be described. Fig. 11 shows the processing flow of the motion plan generation processing of this embodiment. The task motion planner 200 receives instructions from the user and starts the motion plan generation processing. The instructions from the user may be, for example, setting (inputting) a target state.

[0094] The initial model creating unit 240 creates an initial model using the above method (step S2101).

[0095] Thereafter, the abstraction unit 210 creates a first model using the above method (step S2102). The abstraction unit 210 aggregates tasks and sets the first time step length to be longer than the initial time step length. Furthermore, the shape of the object may be simplified compared to the initial model. Work may also be aggregated.

[0096] The first generation unit 220 generates a first operation plan 710 using the first model (step S2103). Here, task sequences and motion sequences that specify the execution timing, execution order, execution allocation, hand positions, postures, etc. of the aggregated tasks using a first time step length are generated as the first operation plan 710. Note that these sequences are generated for each cooperating robot 600.

[0097] The dividing unit 231 divides the first operation plan 710 by a predetermined time interval ΔD (step S2104). Here, the first operation plan 710 is divided into N pieces (N is an integer equal to or greater than 1) by the time interval ΔD. Furthermore, each of the divided first operation plans 710 is called a partial plan 720, and in the order of processing, it is called the nth partial plan 720. Note that n is an integer equal to or greater than 1 and equal to or less than N.

[0098] The second generator 232 uses the initial model to generate a detailed plan 730 for each partial plan 720. That is, for each n from 1 to N, the second generator 232 repeatedly generates an n-th detailed plan 730 based on the n-th partial plan 720 (steps S2105 to S2108).

[0099] The combining unit 233 combines the N detailed plans 730 to generate an operation plan 740 (step S2109), and then ends the process. At this time, the configuration is set so that the final state of the nth detailed plan 730 becomes the initial state of the (n+1)th detailed plan.

[0100] As described above, this embodiment has the same configuration as the first embodiment, and therefore has the same effects as the first embodiment.

[0101] Furthermore, according to this embodiment, the first operation plan 710 is divided in the time direction, and the detailed plan 730 is generated using an initial model for each divided partial plan 720. By generating the detailed plan 730 using the partial plans 720 divided in the time direction, it is possible to generate a detailed operation plan in a spatially non-fixed environment. This is particularly useful when there are multiple robots 600 (devices to be controlled).

[0102] According to this embodiment, the calculation time can be reduced throughout the generation of an operation plan for controlling a robot by taking advantage of the cooperation of all the devices that are the object of the operation plan.

[0103] <Modification 1> In the above embodiment, the dividing unit 231 divides the first operation plan 710 by a predetermined time interval ΔD. However, the time interval for division is not limited to this. For example, division may be performed according to the ratio of the execution times of the tasks before aggregation.

[0104] An example of this case is shown in Figure 12(a). In the above embodiment, the task is divided by a time width ΔD. However, this results in the task being divided in the middle of task d before it is consolidated. To avoid this, the division line 721 is moved to the position of division line 721a and then divided.

[0105] Furthermore, tasks with execution order constraints are divided so that they are in the same group. Specifically, as shown in Fig. 12(b), if there is an execution order constraint that task e must be executed after task d, the tasks are divided so that the two are not separated. As shown in Fig. 12(b), the position of the division line 721 is moved to 721c and 721d to divide the tasks.

[0106] The information on the tasks before and after aggregation is obtained from the aggregated task information 333 stored in the storage device 330 .

[0107] <Modification 2> In the above embodiment, the processing of the plan generating unit 230 is described using an example in which there is one robot 600, but this is not limiting. When recalculating each partial plan, the recalculation is performed taking into account multiple robots 600.

[0108] As an example, an example in which two robots 600 are used is shown in FIG. 13( a ) and FIG. 13( b ).

[0109] As shown in FIG. 13(a), in the first model, task groups a and b of the initial model are designated as task A, task group c to f are designated as task B, and task group g to i are designated as task C.

[0110] 13(b), for example, assume that a first operation plan 710 is generated so that tasks A and B are executed by robot 600#1 and task C is executed by robot 600#2. Subsequently, the plan is divided into partial plans 720, and recalculation may result in a result in which, of tasks a and b included in aggregate task A, task a is executed by robot 600#1 and task b is executed by robot 600#2. An example of an operation plan 740 in this case is shown below.

[0111] In this way, as in this embodiment, by recalculating the motion plan using the initial model for each partial plan 720 divided in the time direction, an optimal solution can be obtained during recalculation even when multiple robots 600 cooperate with each other. For example, when dividing the space and recalculating a detailed motion plan, the spatial range is limited, and the spatial allocation is fixed, making it impossible to consider effective cooperation between the robots 600.

[0112] <Modification 3> In the above embodiment and modification, an example has been described in which an operation plan is generated in two stages, but the present invention is not limited to this. An operation plan may be generated in multiple stages of three or more stages.

[0113] For example, the task motion planner 200 creates a second model that is more abstract than the first model (by aggregating tasks), and then generates a second motion plan using the second model. Next, the second motion plan is divided into predetermined ranges, and a first detailed plan is generated for each divided range using the first model.

[0114] For example, in the example of FIG. 2A , task 821 and task 822 are aggregated in the second model. Then, an operation plan for the aggregated tasks is generated as a second operation plan. After that, operation plans for individual tasks 821 and 822 within the divided ranges are generated as first detailed plans. After that, each first detailed plan is divided into predetermined ranges, and for each divided range, a detailed plan 730 is generated using the initial model, and these are combined using the above method.

[0115] The first detailed plans may be combined using the above method to generate the first operation plan 710. In this case, the subsequent processing is the same as in the above embodiments and modifications.

[0116] In the above embodiment and modification, the case where the control target of the robot system 100 is the robot 600 has been described as an example, but the control target is not limited to this. For example, the control target may be the operation of various types of transportation means such as an automobile or an airplane.

[0117] [Hardware Configuration] An example of the hardware configuration of the robot system 100 according to each of the above embodiments and modifications will be described. Fig. 14 is a diagram showing an example of the hardware configuration of the robot system 100. The robot system 100 includes a CPU (Central Processing Unit) 191, a main storage device (memory) 192, an auxiliary storage device 193, and an interface (I / F) 194. These are connected to each other via a bus so as to be able to communicate with each other.

[0118] The CPU 191 realizes the above functions and comprehensively controls the entire device by, for example, loading a program stored in the auxiliary storage device 193 into the main storage device 192 and executing it. Note that one or more processors such as an MPU (Micro Processing Unit) may be used instead of the CPU 191.

[0119] The main storage device 192 is a memory such as a RAM (Random Access Memory), and is a work area used by the CPU 191 when processing programs executed by the device.

[0120] The auxiliary storage device 193 is, for example, a read-only memory (ROM), a hard disk drive (HDD), or a solid state drive (SSD). The auxiliary storage device 193 stores various programs executed by the device. The auxiliary storage device 193 may include storage media such as a flexible disk, a hard disk, an optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, or a DVD. The storage device 330 may be constructed on the auxiliary storage device 193, for example.

[0121] The programs stored in the auxiliary storage device 193 can be provided as program products recorded on non-transitory computer-readable recording media. The auxiliary storage device 193 can be used to store various programs recorded on non-transitory computer-readable recording media for the medium to long term.

[0122] The I / F 194 is an interface for electrically connecting the robot system 100 to another device. The other device may be, for example, an input device 310, a sensor, or an external storage device. For example, the other device may be an interface compliant with USB (Universal Serial Bus), SATA (Serial AT Attachment), or the like. The robot system 100 may also be connected to a communication device via the I / F 194. By connecting to the communication device, the robot system 100 can input and output signals and data via the communication device via a wired or wireless connection.

[0123] The above-mentioned functions of each device are realized by the CPU 191 loading a program stored in the auxiliary storage device 193 into the main storage device 192 and executing it.

[0124] The robot system 100 may also include at least one of the input device 310, the storage device 330, and a sensor, or a part thereof. The robot system 100 may also be realized by a general-purpose information processing device.

[0125] The hardware configuration of the robot system 100 is not limited to this. For example, an integrated circuit (IC) dedicated to each function, an application specific integrated circuit (ASIC), a system on chip (SOC), a field programmable gate array (FPGA), or the like may be implemented.

[0126] Furthermore, the task motion planner 200 and the robot controller 400 that constitute the robot system 100 may each be realized by independent hardware.

[0127] In addition, a program for realizing each of the above functions can be recorded on a computer-readable storage medium. The storage medium can be a non-transitory medium such as a semiconductor memory, a hard disk, a magnetic recording medium, or an optical recording medium. The present disclosure can also be embodied as a computer program product.

[0128] In the process flow used in the above explanation, multiple steps (processes) are described in order, but the order in which each step is performed is not limited to the order described. For example, the order of the steps shown in the figure can be changed to the extent that the content is not affected, such as by performing each process in parallel.

[0129] Although the embodiments and modifications of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments and can be modified in various ways that would be understandable to a person skilled in the art. Each embodiment and modification can be combined with other embodiments as appropriate. Furthermore, for example, the network configurations and configurations of each element shown in the drawings are examples intended to aid in understanding the present disclosure and are not limited to the configurations shown in these drawings.

[0130] Finally, preferred embodiments of the present disclosure will be summarized. Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. (Supplementary Note 1) A robot system includes: an abstraction unit that creates a first model by aggregating tasks from an initial model created by abstracting information including tasks to be executed by the robot and necessary for generating a motion plan for the robot; a first generation unit that generates a first motion plan using the first model; and a plan generation unit that generates a motion plan from the first motion plan using the initial model. (Supplementary Note 2) In the robot system described in Supplementary Note 1, it is preferable that the plan generation unit includes: a division unit that divides the first motion plan to generate partial plans; a second generation unit that generates a detailed plan from the partial plans using the initial model for each partial plan; and a combination unit that combines the detailed plans to generate the motion plan. (Supplementary Note 3) In the robot system described in Supplementary Note 2, it is preferable that the division unit divides the first motion plan in the time direction to generate the partial plans. (Supplementary Note 4) In the robot system described in Supplementary Note 3, it is desirable that the dividing unit divides the first operation plan according to a ratio of the execution times of the tasks included in the first operation plan. (Supplementary Note 5) In the robot system described in Supplementary Note 3 or 4, it is desirable that the dividing unit divides the first operation plan so that the task group having constraints on the execution order is included in the same partial plan. (Supplementary Note 6) In the robot system described in any of Supplements 3 to 5, it is desirable that the partial plans include a first partial plan, and that the second generation unit generates the detailed plan by setting the state of the robot indicated by the first partial plan as the initial state of a partial plan immediately after the first partial plan.(Supplementary Note 7) In the robot system described in any one of Supplements 1 to 6, it is preferable that the initial model further includes an object model that abstracts the shape of an object in the workspace of the robot and a time step length that is a time unit for generating the motion plan, and that the abstraction unit creates the first model by at least one of increasing the level of abstraction of the object model compared to the initial model and making the time step length longer than the time step length of the initial model. (Supplementary Note 8) In the robot system described in any one of Supplements 1 to 7, it is preferable that the robot system further includes a robot controller that generates an instruction sequence for controlling the robot based on the motion plan. (Supplementary Note 9) A motion planning method executed by a computer mounted on a robot system includes: generating a first model by aggregating tasks from an initial model that is information including tasks to be executed by the robot and that is created by abstracting information necessary for generating a motion plan for the robot; generating a first motion plan using the first model; and generating a motion plan from the first motion plan using the initial model. (Supplementary Note 10) The program causes a computer to execute the following steps: creating a first model by aggregating the tasks from an initial model, the initial model including information including tasks to be executed by the robot, and created by abstracting information necessary for generating a motion plan for the robot; generating a first motion plan using the first model; and generating a motion plan from the first motion plan using the initial model. (Supplementary Note 11) The robot system described in any of Supplements 1 to 8 preferably further includes an initial model creation unit that creates the initial model from pre-stored information necessary for generating a motion plan for the robot. Note that Supplements 9 and 10 can be expanded into Supplements 2-8 and 11, similar to Supplementary Note 1.

[0131] The disclosures of the above-mentioned patent documents and other documents are incorporated herein by reference. Within the scope of this disclosure (including the claims), modifications and adjustments of the embodiments and variations are possible based on the basic technical concepts. Furthermore, within the scope of this disclosure, various combinations and selections of the various disclosed elements (including each element of each claim, each element of each embodiment or variation, each element of each drawing, etc.) are possible. In other words, this disclosure naturally includes various modifications and alterations that would be possible by a person skilled in the art based on the entire disclosure, including the claims, and the technical concepts. In particular, with regard to the numerical ranges described herein, any numerical value or subrange included within the range should be construed as being specifically described, even if not otherwise specified.

[0132] 100: Robot system, 191: CPU, 192: Main storage device, 193: Auxiliary storage device, 194: I / F, 200: Task motion planner, 201: Task motion planner, 202: Model creation unit, 203: Motion plan generation unit, 210: Abstraction unit, 220: First generation unit, 230: Plan generation unit, 231: Division unit, 232: Second generation unit, 233: Combination unit, 240: Initial model creation unit, 310: Input device, 330: Storage device, 331: Initial model, 332: Creation rules, 333: Aggregated task information, 334: Conversion information, 400: Robot controller, 600: robot, 601: hand, 611: hand model, 612: hand model, 620: body (obstacle), 621: obstacle model, 622: obstacle model, 630: working point, 631: working point model, 632: working point model, 641: action point model, 691: hand model, 692: working point model, 693: workpiece model, 694: obstacle model, 700: division period, 710: first operation plan, 720: partial plan, 720a: partial plan, 720b: partial plan, 721: division line, 721a: division line, 721c: division line, 721d: division line, 730: detailed plan, 730a: detailed plan, 730b: detailed plan, 740: operation plan, 811: working point, 812: working point, 813: working point, 814: working point, 815: working point, 816: working point, 821: working point, 822: working point, S1: target state, S2: retained data, S3: motion plan, S4: control instruction sequence, ΔD: time width, ΔT: first time step length, Δt: initial time step length

Claims

1. A robot system comprising: an abstraction unit that aggregates tasks from an initial model created by abstracting information including tasks to be executed by a robot and necessary for generating an operation plan for the robot, to create a first model; a first generation unit that generates a first operation plan using the first model; and a plan generation unit that generates an operation plan from the first operation plan using the initial model.

2. A robot system according to claim 1, wherein the plan generation unit comprises: a division unit that divides the first operation plan to generate partial plans; a second generation unit that generates, for each partial plan, a detailed plan from the partial plan using the initial model; and a combination unit that combines the detailed plans to generate the operation plan.

3. A robot system according to claim 2, wherein the division unit divides the first motion plan in the time direction to generate the partial plans.

4. A robot system according to claim 3, wherein the dividing unit divides the first operation plan in accordance with the ratio of the execution times of the tasks included in the first operation plan.

5. A robot system according to claim 3, wherein the dividing unit divides the first operation plan so that the task group having constraints on the execution order is included in the same partial plan.

6. A robot system according to claim 3, wherein the partial plan includes a first partial plan, and the second generation unit generates the detailed plan by setting the state of the robot indicated by the first partial plan as the initial state of the partial plan immediately following the first partial plan.

7. A robot system according to claim 1, wherein the initial model further includes an object model that abstracts the shape of an object in the robot's workspace and a time step length that is a time unit for generating the motion plan, and the abstraction unit creates the first model by performing at least one of increasing the degree of abstraction of the object model compared to the initial model and making the time step length longer than the time step length of the initial model.

8. A robot system according to claim 1, further comprising a robot controller that generates an instruction sequence for controlling the robot based on the motion plan.

9. A motion planning method executed by a computer installed in a robot system, comprising: generating a first model by aggregating the tasks from an initial model created by abstracting information including tasks to be executed by the robot and necessary for generating a motion plan for the robot; generating a first motion plan using the first model; and generating a motion plan from the first motion plan using the initial model.

10. A program that causes a computer to execute the following steps: creating a first model by aggregating the tasks from an initial model created by abstracting information including tasks to be executed by a robot and necessary information for generating an operation plan for the robot; generating a first operation plan using the first model; and generating an operation plan from the first operation plan using the initial model.

Images