Learning device

Abstract

This learning device comprises: an extraction unit that extracts, from trajectory information including state information for each control period along a time series, information that may change with a period longer than the control period; and a generation unit that generates a learning trajectory that can accommodate changes with a period longer than the control period, on the basis of the information extracted by the extraction unit and the trajectory information.

Description

Learning Device

[0001] The present disclosure relates to a learning device, a generation method, and a recording medium.

[0002] It is known to control a robot such as an arm robot according to a trajectory plan.

[0003] For example, Patent Literature 1 describes a machine learning device including a processor that executes a program and a storage device that stores the program. According to Patent Literature 1, the storage device stores learning data that associates a gripping point position with a trajectory planning ease, which is a value that quantifies the ease of trajectory planning for planning a trajectory to the gripping point of a robot arm, for a recognized image in which a group of items and the box containing the group of items are recognized through an opening of the box. The processor also executes an acquisition process that acquires a first image of the group of items and the box through the opening of the box, a calculation process that calculates a predicted position of the gripping point position and a predicted value of the trajectory planning ease using parameters based on image features of the first image acquired by the acquisition process, and an update process that updates the parameters based on the gripping point position and the trajectory planning ease, which are the learning data stored in the storage device, and the calculation results calculated by the calculation process.

[0004] Japanese Patent Application Laid-Open No. 2018-202550

[0005] Qinsheng Zhang et al. "DiffCollage: Parallel Generation of Large Content with Diffusion Models," retrieved September 1, 2023, Internet <URL: https: / / arxiv.org / abs / 2303.17076>

[0006] One type of trajectory planning is Task and Motion Planning (TAMP), which performs time-series task planning along with motion planning to move a robot from an initial state to a final state. Such trajectory plans can be generated using, for example, a pre-trained diffusion model.

[0007] Furthermore, for the purpose of reducing the computational costs of learning and generation, a technique called DiffCollage (Non-Patent Document 1) is sometimes used, which generates high-dimensional trajectories by simultaneously using multiple diffusion models that generate low-dimensional trajectories. In this case, if the learning target is simply extracted and trained to have a low dimensionality, such as by extracting and training two adjacent states (motion trajectories) from the training trajectory, it becomes impossible to capture the medium- to long-term structure. As a result, it may become impossible to perform training that captures the medium- to long-term structure inherent in the trajectory. Thus, when generating high-dimensional trajectories using multiple diffusion models that generate low-dimensional trajectories, there has been a problem in that it may become impossible to perform training that captures the medium- to long-term structure.

[0008] Therefore, one of the objects of the present disclosure is to provide a learning device, a generation method, and a recording medium that can solve the above-mentioned problems.

[0009] In order to achieve this purpose, the learning device disclosed herein has a configuration including: an extraction unit that extracts information that may change at periods longer than the control period from trajectory information that includes state information for each control period along a time series; and a generation unit that generates a learning trajectory that can respond to changes at periods longer than the control period based on the information extracted by the extraction unit and the trajectory information.

[0010] Furthermore, the generation method in the present disclosure is configured such that an information processing device extracts information that may change at periods longer than the control period from trajectory information including state information for each control period along a time series, and generates a learning trajectory that can respond to changes at periods longer than the control period based on the extracted information and the trajectory information.

[0011] Furthermore, the recording medium in the present disclosure is a computer-readable recording medium having recorded thereon a program for causing an information processing device to perform the following process: extract information that may change at periods longer than the control period from trajectory information including state information for each control period along a time series; and generate a learning trajectory that can respond to changes at periods longer than the control period based on the extracted information and the trajectory information.

[0012] According to the above-described configurations, it becomes possible to perform learning using the generated learning trajectory, thereby reducing the risk of being unable to perform learning that captures the medium- to long-term structure.

[0013] FIG. 1 is a diagram for explaining an overview of a control system in the present disclosure. FIG. 1 is a diagram for explaining an example of a trajectory. FIG. 1 is a diagram for explaining an example of a trajectory. FIG. 2 is a diagram for explaining an example of a trajectory. FIG. 2 is a diagram for explaining an example of a configuration of an entire control system. FIG. 3 is a block diagram for explaining an example of a configuration of a learning device. FIG. 4 is a diagram for explaining an example of a process of a long-period state extraction unit. FIG. 5 is a diagram for explaining an example of a process of a learning trajectory generation unit. FIG. 6 is a diagram for explaining an example of a process of the learning trajectory generation unit. FIG. 7 is a block diagram for explaining an example of a configuration of a control device. FIG. 8 is a diagram for explaining an example of a process when a control device generates a trajectory. FIG. 9 is a diagram for explaining an example of a process when a control device generates a trajectory. A flowchart showing an example of an operation of a learning device. A flowchart showing an example of an operation of a control device. A diagram showing an example of a hardware configuration of a second learning device in the present disclosure. A block diagram for explaining an example of a configuration of a learning device. A flowchart showing an example of an operation of a learning device.

[0014] [First Embodiment] A configuration example of a control system 100 according to the present disclosure will be described with reference to Figs. 1 to 14. Fig. 1 is a diagram for explaining an overview of the control system 100. Figs. 2 and 3 are diagrams for explaining an example of a trajectory. Fig. 4 is a diagram showing an example of the configuration of the entire control system 100. Fig. 5 is a block diagram showing an example of the configuration of a learning device 200. Fig. 6 is a diagram for explaining an example of processing by the long-period state extraction unit 252. Figs. 7 to 9 are diagrams for explaining an example of processing by the learning trajectory generation unit 253. Fig. 10 is a block diagram showing an example of the configuration of a control device 300. Figs. 11 and 12 are diagrams for explaining an example of processing when the control device 300 generates a trajectory. Fig. 13 is a flowchart showing an example of operation of the learning device 200. Fig. 14 is a flowchart showing an example of operation of the control device 300.

[0015] In a first embodiment of the present disclosure, a control system 100 is described that generates a trajectory (trajectory plan) for causing a robot to perform a predetermined process and controls the robot or the like according to the generated trajectory. For example, as illustrated in FIG. 1 , assume that the robot is to execute a process of moving objects A and B to the left end and then placing object B on object A. In such a case, if object B is moved first, object B will be placed where object A should be, making it impossible to place object B on object A. Therefore, rather than simply generating a trajectory for transporting object A and object B, it is necessary to generate a trajectory that captures a medium- to long-term structure, in which a series of processes is performed on object A followed by a series of processes on object B.

[0016] Therefore, the control system 100 described in the present disclosure acquires a learning trajectory and extracts information from the trajectory to be learned that may change at periods longer than the control period. Then, the control system 100 generates a low-dimensional learning trajectory that can accommodate changes at periods longer than the control period based on the extracted information and the trajectory. This allows the control system 100 to perform learning using a learning trajectory that corresponds to a medium- to long-term structure while reducing computational costs. Note that "low-dimensional" refers to the number of state vectors included in the trajectory being less than a predetermined number, such as two. For example, the control system 100 can generate a learning trajectory that includes two state vectors that correspond to a medium- to long-term structure based on the extracted information and the trajectory.

[0017] As will be described later, the control system 100 not only performs learning according to a medium- to long-term structure, but also performs learning according to a short-term structure by extracting and learning two adjacent state vectors from the learning trajectory as a motion trajectory. When generating a trajectory using a trained model, the control system 100 performs hierarchical trajectory generation, first generating a trajectory using a model generated by training according to a medium- to long-term structure, and then generating a trajectory using a model generated by training according to a shorter-term structure. This makes it possible to generate appropriate trajectories while adapting to the medium- to long-term structure.

[0018] In the present disclosure, a trajectory refers to information including a time-series state vector. Fig. 2 shows an example of a state vector at time t = 0 according to a certain control period. For example, referring to Fig. 2, the state vector includes information indicating the position of a robot such as an arm robot performing processing, information indicating the position of an object that is an object to be controlled such as object A or object B, and information indicating a skill to be executed by the robot at that timing (or a skill currently being executed; hereinafter, referred to as an "executed skill").

[0019] As shown in Fig. 3, the trajectory includes a plurality of state vectors in time series, such as those illustrated in Fig. 2. For example, the trajectory can include a state vector for each control period.

[0020] 2 and 3 illustrate one-dimensional cases for simplicity. However, the state vector may include information indicating a position in two or three dimensions. Furthermore, the execution skills may include, for example, approach, which indicates approaching an object, pick, which indicates starting to grasp an object, hold, which indicates carrying an object, place, which indicates releasing the object, and push, which indicates pushing an object. The execution skills may include any action other than those exemplified above.

[0021] Fig. 4 shows an example of the overall configuration of the control system 100. Referring to Fig. 4, the control system 100 has a learning device 200 and a control device 300. The control system 100 may also include a robot 400, such as an arm robot, controlled by the control device 300. As shown in Fig. 4, the learning device 200 and the control device 300 are connected by wire or wirelessly so as to be able to communicate with each other. The control device 300 and the robot 400 are also connected by wire or wirelessly so as to be able to communicate with each other.

[0022] The control system 100 may have a configuration other than that illustrated in Fig. 4. For example, the control system 100 may be configured from an information processing device that has both the functions of the learning device 200 and the control device 300. Furthermore, the learning device 200 and the control device 300 may be configured from multiple information processing devices.

[0023] The learning device 200 is an information processing device that learns a diffusion model for generating a trajectory according to a learning trajectory. The learning device 200 can learn multiple diffusion models that perform low-dimensional generation. FIG. 5 shows an example configuration of the learning device 200. Referring to FIG. 5, the learning device 200 has, as main components, for example, an operation input unit 210, a screen display unit 220, a communication I / F unit 230, a storage unit 240, and a calculation processing unit 250.

[0024] 5 illustrates an example in which the functions of learning device 200 are realized using a single information processing device. However, at least some of the functions of learning device 200 may be realized using multiple information processing devices, for example, on the cloud. Learning device 200 may not include some of the components illustrated above, such as not having operation input unit 210 or screen display unit 220, or may have components other than those illustrated above.

[0025] Operation input unit 210 is made up of operation input devices such as a keyboard, a mouse, etc. Operation input unit 210 detects operations of the operator operating learning device 200 and outputs the operations to calculation processing unit 250.

[0026] The screen display unit 220 is composed of a screen display device such as a liquid crystal display, an organic electroluminescence (EL) display, etc. The screen display unit 220 can display various information stored in the storage unit 240 on the screen in response to instructions from the arithmetic processing unit 250.

[0027] The communication I / F unit 230 includes a data communication circuit, etc. The communication I / F unit 230 performs data communication with an external device connected via a communication line.

[0028] The storage unit 240 is a storage device such as a hard disk or memory. The storage unit 240 stores processing information and a program 242 required for various processes in the arithmetic processing unit 250. The program 242 is read into the arithmetic processing unit 250 and executed to realize various processing units. The program 242 is read in advance from an external device or recording medium via a data input / output function such as the communication I / F unit 230, and is stored in the storage unit 240. Main information stored in the storage unit 240 includes, for example, orbit information 241.

[0029] The trajectory information 241 includes a plurality of trajectories for learning. As described above, the trajectories include a state vector for each control period. The trajectory information 241 is updated in response to the trajectory information acquisition unit 251 acquiring a trajectory for learning, etc.

[0030] The learning trajectory refers to, for example, a prepared trajectory for handling one or more objects using an arbitrary skill. For example, the learning trajectory may include any trajectory, such as a trajectory for the robot 400 to push object A to a predetermined location, or a trajectory for the robot 400 to grasp object B, transport it to a predetermined location, and then release the grasp.

[0031] The arithmetic processing unit 250 has an arithmetic device such as a CPU (Central Processing Unit) and its peripheral circuits. The arithmetic processing unit 250 reads and executes a program 242 from the storage unit 240, thereby realizing various processing functions by causing the above hardware and the program 242 to cooperate with each other. Major processing units realized by the arithmetic processing unit 250 include, for example, a trajectory information acquisition unit 251, a long-period state extraction unit 252, a learning trajectory generation unit 253, a task trajectory learning unit 254, a skill trajectory learning unit 255, a motion trajectory learning unit 256, and an output unit 257.

[0032] In addition, the arithmetic processing unit 250 may have a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination of these, instead of the above-mentioned CPU.

[0033] The trajectory information acquisition unit 251 acquires a trajectory for learning. For example, the trajectory information acquisition unit 251 can acquire the trajectory for learning by receiving information input using the operation input unit 210 or by receiving information from an external device or the like via the communication I / F unit 230. Furthermore, the trajectory information acquisition unit 251 stores the acquired trajectory as trajectory information 241 in the storage unit 240.

[0034] The long-period state extraction unit 252 extracts, from the learning trajectory, information that may change at a period longer than the control period as information indicating a medium- to long-term structure. For example, the long-period state extraction unit 252 extracts at least one of information indicating a task state, information indicating a skill state, etc. as information that may change at a period longer than the control period.

[0035] FIG. 6 is a diagram illustrating an example of processing by the long-period state extraction unit 252. Referring to FIG. 6, for example, the long-period state extraction unit 252 extracts information indicating a task state as information that may change at a period longer than the control period. Here, the information indicating the task state refers to information indicating a target object of the execution skill from the state vector corresponding to each control period included in the trajectory. For example, in the example illustrated in FIG. 6, the state vector at t=0 includes "Approach A" as information indicating the execution skill. Therefore, the long-period state extraction unit 252 extracts "A," which indicates the target object of the execution skill, from the state vector at t=0. For example, the long-period state extraction unit 252 can extract information indicating the task state by extracting information indicating each target object from the state vector corresponding to each control period. Note that, in cases where the state vector does not include information indicating the execution skill, the long-period state extraction unit 252 may be configured to extract, as information indicating the target object, the name of the object closest to the robot 400 in each control period.

[0036] Furthermore, the long-period state extraction unit 252 extracts information indicating a skill state as information that may change at a period longer than the control period. Here, the information indicating a skill state refers to information indicating an execution skill from among the state vectors corresponding to each control period included in the trajectory. For example, in the example shown in FIG. 6 , the state vector at t=0 includes "approach A" as information indicating an execution skill. Therefore, the long-period state extraction unit 252 extracts "approach A," which indicates an execution skill, from the state vector at t=0. For example, the long-period state extraction unit 252 can extract information indicating a skill state by extracting information indicating each skill from the state vectors corresponding to each control period.

[0037] For example, the long-period state extraction unit 252 extracts at least a part of the information indicating the task state and the information indicating the skill state, as described above, in accordance with the learning trajectory. The long-period state extraction unit 252 may perform the extraction process described above for each trajectory included in the trajectory information 241.

[0038] The learning trajectory generation unit 253 generates a low-dimensional learning trajectory that can respond to changes in periods longer than the control period, based on the learning trajectory and the information extracted by the long-period state extraction unit 252. For example, the learning trajectory generation unit 253 generates, as the low-dimensional learning trajectory, a trajectory that includes two state vectors or fewer than a predetermined number of state vectors.

[0039] For example, the learning trajectory generation unit 253 focuses on the object indicated by the information extracted by the long-period state extraction unit 252 and generates a learning trajectory including an initial (starting point) state vector and an end (ending point) state vector among state vectors for which the object indicated by the information extracted by the long-period state extraction unit 252 is the same. Specifically, for example, the learning trajectory generation unit 253 can generate a task trajectory using a learning trajectory and information indicating the task state extracted by the long-period state extraction unit 252. Here, a task trajectory refers to a trajectory generated by focusing on a target object. A task trajectory is a low-dimensional trajectory containing a small number of state vectors, yet can capture a time structure specific to handling, i.e., the "starting point and ending point of handling an object." For example, in the case of the trajectory illustrated in FIG. 7 , the target object is the same as A from the state vector at t = 0 to the state vector at t = 5. Therefore, the learning trajectory generation unit 253 generates, as a task trajectory, a trajectory that includes the state vector at t=0 serving as the start point and the state vector at t=5 serving as the end point from a series of state vectors from t=0 to t=5 in which the target object is A. Similarly, in the case of the trajectory illustrated in FIG. 7 , the target object is B from the state vector at t=6 to the state vector at t=9. Therefore, the learning trajectory generation unit 253 generates, as a task trajectory, a trajectory that includes the state vector at t=6 serving as the start point and the state vector at t=9 serving as the end point from a series of state vectors from t=6 to t=9 in which the target object is B. The learning trajectory generation unit 253 can repeat the same process for each time-series state vector included in each learning trajectory.

[0040] The learning trajectory generation unit 253 generates a skill trajectory using the learning trajectory and the information indicating the skill state extracted by the long-period state extraction unit 252. Here, the skill trajectory refers to a trajectory generated with a focus on the executed skill. The skill trajectory is a low-dimensional trajectory containing a small number of state vectors, yet can capture the time structure of the "starting point and ending point of the skill." For example, in the case of the trajectory illustrated in FIG. 8 , a skill called "Approach A" is executed from the state vector at t=0 to the state vector at t=1. Therefore, the learning trajectory generation unit 253 generates, as the skill trajectory, a trajectory including the state vector at t=0 serving as the starting point and the state vector at t=1 serving as the ending point, among a series of state vectors from t=0 to t=1 in which the skill is the same as "Approach A." ​​The learning trajectory generation unit 253 can repeat the same process for each time-series state vector contained in each learning trajectory.

[0041] The learning trajectory generation unit 253 may generate a task trajectory or a skill trajectory that includes only a state vector serving as a starting point and a state vector serving as an end point, or may generate a trajectory that includes state vectors extracted by any method from among state vectors existing between the starting point and the end point. For example, the learning trajectory generation unit 253 may generate a learning trajectory that includes a state vector serving as a starting point and a state vector serving as an end point. In the example shown in FIG. 7 , state vectors at t=1, t=2, t=3, and t=4 exist between the state vector at t=0 serving as the starting point and the state vector at t=5 serving as the end point. The learning trajectory generation unit 253 may extract some of the intermediate state vectors using any method and include them in the learning trajectory, such as the task trajectory or the skill trajectory, that is generated.

[0042] Furthermore, the learning trajectory generation unit 253 can generate, as a motion trajectory, a trajectory that includes two state vectors that are adjacent in time series from among the trajectories for learning. For example, as illustrated in Fig. 9, the learning trajectory generation unit 253 can generate, as a motion trajectory, a trajectory that is made up of two state vectors that are adjacent in time series, such as a state vector at t = 0 and a state vector at t = 1. Note that the learning trajectory generation unit 253 may generate, as a motion trajectory, a trajectory that corresponds to the number of state vectors included in the task trajectory or the skill trajectory, such as generating a trajectory that includes three state vectors that are adjacent in time series.

[0043] The task trajectory learning unit 254 learns a diffusion model using the task trajectory generated by the learning trajectory generation unit 253. The task trajectory learning unit 254 may use the task trajectory, which is the learning trajectory, to generate multiple diffusion models that generate low-dimensional trajectories.

[0044] The task trajectory learning unit 254 can perform learning using any method used to learn a diffusion model, such as the above-mentioned DiffCollage (Non-Patent Document 1). For example, the task trajectory learning unit 254 learns the diffusion model by learning a process of adding noise a predetermined number of times, such as 100 times, to the task trajectory generated by the learning trajectory generation unit 253. As a result, when generating a trajectory, the task trajectory learning unit 254 can generate a trajectory (task trajectory) by gradually removing noise from an arbitrary trajectory, etc., by tracing the reverse process of the learned process.

[0045] The skill trajectory learning unit 255 learns a diffusion model using the skill trajectory generated by the learning trajectory generation unit 253. The skill trajectory learning unit 255 may use the skill trajectory that is the learning trajectory to generate a plurality of diffusion models that generate low-dimensional trajectories.

[0046] Like the task trajectory learning unit 254, the skill trajectory learning unit 255 can perform learning using any method used in learning a diffusion model, such as the above-mentioned DiffCollage (Non-Patent Document 1). For example, the skill trajectory learning unit 255 learns the diffusion model by learning a process of adding noise a predetermined number of times, such as 100 times, to the skill trajectory generated by the learning trajectory generation unit 253. As a result, when generating a trajectory, the skill trajectory learning unit 255 can generate a trajectory (skill trajectory) by gradually removing noise from an arbitrary trajectory, etc., by tracing the reverse process of the learned process.

[0047] The motion trajectory learning unit 256 learns a diffusion model using the motion trajectory generated by the learning trajectory generation unit 253. The motion trajectory learning unit 256 may use the motion trajectory that is the learning trajectory to generate multiple diffusion models that generate low-dimensional trajectories.

[0048] Like the task trajectory learning unit 254 and the skill trajectory learning unit 255, the motion trajectory learning unit 256 can perform learning using any method used in learning a diffusion model, such as the above-mentioned DiffCollage (Non-Patent Document 1). For example, the motion trajectory learning unit 256 learns a diffusion model by learning a process of adding noise a predetermined number of times, such as 100 times, to the motion trajectory generated by the learning trajectory generation unit 253. As a result, when generating a trajectory, the motion trajectory learning unit 256 can generate a trajectory (motion trajectory) by gradually removing noise from an arbitrary trajectory, etc., by tracing the reverse process of the learned process.

[0049] For example, as described above, the learning device 200 performs the following: learning of a diffusion model according to a task trajectory by the task trajectory learning unit 254; learning of a diffusion model according to a skill trajectory by the skill trajectory learning unit 255; and learning of a diffusion model according to a motion trajectory by the motion trajectory learning unit 256. That is, the learning device 200 performs learning of a diffusion model for each type of learning trajectory, such as a task trajectory, a skill trajectory, and a motion trajectory. Note that the learning by the task trajectory learning unit 254, the learning by the skill trajectory learning unit 255, and the learning by the motion trajectory learning unit 256 may be performed independently. Furthermore, the number of diffusion models generated by the task trajectory learning unit 254, the number of diffusion models generated by the skill trajectory learning unit 255, and the number of diffusion models generated by the motion trajectory learning unit 256 may each be determined arbitrarily.

[0050] The output unit 257 transmits the learned models to the control device 300. For example, the output unit 257 can transmit the plurality of diffusion models learned by the task trajectory learning unit 254, the plurality of diffusion models learned by the skill trajectory learning unit 255, and the plurality of diffusion models learned by the motion trajectory learning unit 256 to the control device 300 via the communication I / F unit 230 or the like.

[0051] The above is an example of the configuration of the learning device 200.

[0052] The control device 300 is an information processing device that generates a trajectory using the diffusion model learned by the learning device 200. The control device 300 can also control the robot 400 and the like based on the generated trajectory. FIG. 10 shows an example configuration of the control device 300. Referring to FIG. 10, the control device 300 has, as main components, an operation input unit 310, a screen display unit 320, a communication I / F unit 330, a storage unit 340, and a calculation processing unit 350, for example.

[0053] 10 illustrates an example in which the functions of the control device 300 are realized using one information processing device. However, at least some of the functions of the control device 300 may be realized using multiple information processing devices, for example, on the cloud. Furthermore, the control device 300 may not include some of the components illustrated above, such as not having the operation input unit 310 or the screen display unit 320, or may have a configuration other than those illustrated above.

[0054] The configurations of the operation input unit 310 , screen display unit 320 , and communication I / F unit 330 may be similar to the operation input unit 210 , screen display unit 220 , and communication I / F unit 230 of the learning device 200 .

[0055] The storage unit 340 is a storage device such as a hard disk or memory. The storage unit 340 stores processing information and programs 342 required for various processes in the arithmetic processing unit 350. The programs 342 are read into the arithmetic processing unit 350 and executed to realize various processing units. The programs 342 are read in advance from an external device or recording medium via a data input / output function such as the communication I / F unit 330, and are stored in the storage unit 340. Main information stored in the storage unit 340 includes, for example, model information 341.

[0056] The model information 341 includes information about various models learned by the learning device 200. For example, the model information 341 includes a diffusion model learned by the task trajectory learning unit 254, a diffusion model learned by the skill trajectory learning unit 255, and a diffusion model learned by the motion trajectory learning unit 256. The model information 341 is acquired at any timing from the learning device 200 via the communication I / F unit 330 or the like, and is stored in the storage unit 340.

[0057] The arithmetic processing unit 350 includes a calculation device such as a CPU and its peripheral circuits. The arithmetic processing unit 350 reads and executes a program 342 from the storage unit 340, thereby causing the hardware and the program 342 to cooperate to realize various processing units. Major processing units realized by the arithmetic processing unit 350 include, for example, an initial information acquisition unit 351, a task trajectory generation unit 352, a skill trajectory generation unit 353, a motion trajectory generation unit 354, a control input calculation unit 355, and a robot control unit 356. Note that, like the arithmetic processing unit 250, the arithmetic processing unit 350 may include a GPU or the like instead of the CPU described above.

[0058] The initial information acquisition unit 351 acquires information that serves as a condition for generating a trajectory. The initial information acquisition unit 351 can acquire information indicating the initial position of the robot or object, information indicating a target position, and the like. The initial information acquisition unit 351 may also acquire information indicating any other condition. The initial information acquisition unit 351 can acquire the information exemplified above by accepting information input using the operation input unit 310 or by accepting information from an external device or the like via the communication I / F unit 330.

[0059] The task trajectory generation unit 352 uses the diffusion model learned by the task trajectory learning unit 254 and the information acquired by the initial information acquisition unit 351 to generate a task trajectory, which is a trajectory for transporting all objects included in the control system 100 to a target position. The task trajectory generation unit 352 may generate the task trajectory using a method similar to the above-mentioned method called DiffCollage. For example, the task trajectory generation unit 352 can generate a high-dimensional trajectory (task trajectory) by simultaneously using multiple diffusion models that generate low-dimensional data learned by the task trajectory learning unit 254.

[0060] The skill trajectory generation unit 353 generates a skill trajectory by using the diffusion model learned by the skill trajectory learning unit 255 and the task trajectory generated by the task trajectory generation unit 352, so as to interpolate between the task trajectories generated by the task trajectory generation unit 352. The skill trajectory generation unit 353 may generate a skill trajectory using a method similar to the above-mentioned method called DiffCollage. For example, the skill trajectory generation unit 353 can generate a high-dimensional trajectory (skill trajectory) by simultaneously using multiple diffusion models that generate low-dimensional data learned by the skill trajectory learning unit 255.

[0061] The motion trajectory generation unit 354 generates a motion trajectory by using the diffusion model learned by the motion trajectory learning unit 256 and the skill trajectory generated by the skill trajectory generation unit 353, so as to interpolate between the skill trajectories generated by the skill trajectory generation unit 353. The motion trajectory generation unit 354 may generate a motion trajectory using a method similar to the above-mentioned method called DiffCollage. For example, the motion trajectory generation unit 354 can generate a high-dimensional trajectory (motion trajectory) by simultaneously using multiple diffusion models that generate low-dimensional data learned by the motion trajectory learning unit 256.

[0062] For example, as described above, the control device 300 generates hierarchical trajectories by using the task trajectory generation unit 352, the skill trajectory generation unit 353, and the motion trajectory generation unit 354 to generate a trajectory corresponding to a medium- to long-term structure and then generate a trajectory corresponding to a short-term structure. For example, as illustrated in FIG. 11 , the control device 300 uses the task trajectory generation unit 352 to generate a task trajectory that can express a medium- to long-term structure. Next, the control device 300 uses the skill trajectory generation unit 353 to generate a skill trajectory that fills in the gaps between the generated task trajectories. Thereafter, the control device 300 uses the motion trajectory generation unit 354 to generate a motion trajectory that fills in the gaps between the generated skill trajectories. For example, by performing the above-described hierarchical trajectory generation, the control device 300 generates a trajectory that can accommodate both a medium- to long-term structure and a short-term structure.

[0063] The control device 300 may generate a trajectory using a method other than the above example. For example, assume that a diffusion model generates a trajectory by performing 100 steps of noise removal. In this case, the control device 300 may generate a trajectory by performing the following process: generating a task trajectory using the task trajectory generation unit 352 for steps 1 to 10, generating a skill trajectory using the skill trajectory generation unit 353 for steps 11 to 20, generating a motion trajectory using the motion trajectory generation unit 354 for steps 21 to 30, generating a task trajectory using the task trajectory generation unit 352 for steps 31 to 40, and repeating the same process thereafter up to step 100. In other words, the control device 300 may be configured to generate a trajectory by repeating a series of steps including generating a task trajectory, generating a skill trajectory, and generating a motion trajectory multiple times, as illustrated in FIG. 12 .

[0064] The control input calculation unit 355 calculates the control input to be input to the robot 400 according to the motion trajectory generated by the motion trajectory generation unit 354. For example, the control input calculation unit 355 calculates the control input to be input to the robot 400 so as to satisfy the state indicated by each state vector for each control cycle included in the motion trajectory. Note that when the control device 300 generates a trajectory by repeating trajectory generation multiple times, the control input calculation unit 355 may calculate the control input according to the motion trajectory finally generated by the motion trajectory generation unit 354.

[0065] The robot control unit 356 controls the robot in accordance with the control input calculated by the control input calculation unit 355. In other words, the robot control unit 356 controls the robot 400 so as to satisfy the states indicated by the state vectors included in the motion trajectory generated by the motion trajectory generation unit 354.

[0066] The above is an example configuration of the control device 300. Note that the control device 300 may be composed of a trajectory generation device having an initial information acquisition unit 351, a task trajectory generation unit 352, a skill trajectory generation unit 353, and a motion trajectory generation unit 354, and a control device having a control input calculation unit 355 and a robot control unit 356 that controls the robot 400 using the trajectory generated by the trajectory generation device.

[0067] Next, an example of the operation of the learning device 200 and the control device 300 will be described with reference to Fig. 13 and Fig. 14. First, an example of the operation of the learning device 200 will be described with reference to Fig. 13.

[0068] 13 is a flowchart showing an example of the operation of the learning device 200. Referring to FIG. 13, the long-period state extraction unit 252 extracts, from the learning trajectory, information that may change at a period longer than the control period as information indicating a medium- to long-term structure (step S101). For example, the long-period state extraction unit 252 extracts at least one of information indicating a task state, information indicating a skill state, etc. as information that may change at a period longer than the control period.

[0069] The learning trajectory generation unit 253 generates a low-dimensional learning trajectory that can respond to changes in periods longer than the control period, based on the learning trajectory and the information extracted by the long-period state extraction unit 252 (step S102). For example, the learning trajectory generation unit 253 generates a task trajectory using the learning trajectory and information indicating the task state extracted by the long-period state extraction unit 252. The learning trajectory generation unit 253 also generates a skill trajectory using the learning trajectory and information indicating the skill state extracted by the long-period state extraction unit 252. The learning trajectory generation unit 253 can also generate, as a motion trajectory, a trajectory that includes two state vectors that are adjacent in time series among the learning trajectories.

[0070] The task trajectory learning unit 254 learns a diffusion model using the task trajectory generated by the learning trajectory generation unit 253 (step S103). The task trajectory learning unit 254 may use the task trajectory, which is the learning trajectory, to generate multiple diffusion models that perform low-dimensional trajectory generation.

[0071] The skill trajectory learning unit 255 learns a diffusion model using the skill trajectory generated by the learning trajectory generation unit 253 (step S104). The skill trajectory learning unit 255 may use the skill trajectory that is the learning trajectory to generate multiple diffusion models that generate low-dimensional trajectories.

[0072] The motion trajectory learning unit 256 learns a diffusion model using the motion trajectory generated by the learning trajectory generation unit 253 (step S105). The motion trajectory learning unit 256 may use the motion trajectory that is the learning trajectory to generate multiple diffusion models that generate low-dimensional trajectories.

[0073] Note that the process of step S103, the process of step S104, and the process of step S105 may be executed in any order, or may be executed in parallel. Furthermore, the diffusion model learned by the task trajectory learning unit 254, the diffusion model learned by the skill trajectory learning unit 255, and the diffusion model learned by the motion trajectory learning unit 256 can be transmitted to the control device 300.

[0074] The above is an example of the operation of the learning device 200. Next, an example of the operation of the control device 300 will be described with reference to FIG.

[0075] 14 is a flowchart showing an example of the operation of the control device 300. Referring to Fig. 14, the task trajectory generation unit 352 generates a task trajectory, which is a trajectory for transporting all objects included in the control system 100 to a target position, using the diffusion model learned by the task trajectory learning unit 254 and information that serves as a condition for trajectory generation (step S201).

[0076] The skill trajectory generation unit 353 generates a skill trajectory by using the diffusion model learned by the skill trajectory learning unit 255 and the task trajectory generated by the task trajectory generation unit 352 so as to interpolate between the task trajectories generated by the task trajectory generation unit 352 (step S202).

[0077] The motion trajectory generation unit 354 generates a motion trajectory by using the diffusion model learned by the motion trajectory learning unit 256 and the skill trajectory generated by the skill trajectory generation unit 353 so as to interpolate between the skill trajectories generated by the skill trajectory generation unit 353 (step S203).

[0078] The control input calculation unit 355 calculates the control input to be input to the robot 400 according to the motion trajectory generated by the motion trajectory generation unit 354 (step S204). For example, the control input calculation unit 355 calculates the control input to be input to the robot 400 so as to satisfy the state indicated by each state vector for each control cycle included in the motion trajectory. Note that if the control device 300 generates a trajectory by repeating trajectory generation multiple times, the control input calculation unit 355 may calculate the control input according to the motion trajectory finally generated by the motion trajectory generation unit 354.

[0079] The robot control unit 356 controls the robot in accordance with the control input calculated by the control input calculation unit 355 (step S205). That is, the robot control unit 356 controls the robot 400 so as to satisfy the states indicated by the state vectors included in the motion trajectory generated by the motion trajectory generation unit 354.

[0080] The above is an example of the operation of the control device 300. Note that the control device 300 may be configured to generate a final motion trajectory by repeating the processes from step S201 to step S203 multiple times.

[0081] As described above, the learning device 200 includes the long-period state extraction unit 252 and the learning trajectory generation unit 253. With this configuration, the learning trajectory generation unit 253 can generate a low-dimensional learning trajectory that can respond to changes at periods longer than the control period, based on the learning trajectory and information that may change at periods longer than the control period extracted by the long-period state extraction unit 252. As a result, by performing machine learning using the generated learning trajectory, it becomes possible to generate a diffusion model that captures a long-term structure. This solves the problem that it may be impossible to perform learning that captures a medium- to long-term structure.

[0082] Furthermore, the long-period state extraction unit 252 is configured to extract information indicating a task state and information indicating a skill state. With this configuration, the learning trajectory generation unit 253 can generate a task trajectory focusing on a target object using the information indicating the task state, and can generate a skill trajectory focusing on an executed skill using the information indicating the skill state. As a result, by performing machine learning using the task trajectory and machine learning using the skill trajectory, learning that captures a more hierarchical structure can be performed.

[0083] The control device 300 also has a task trajectory generation unit 352, a skill trajectory generation unit 353, and a motion trajectory generation unit 354. With this configuration, the control device 300 can perform hierarchical trajectory generation using the task trajectory generation unit 352, the skill trajectory generation unit 353, and the motion trajectory generation unit 354. This makes it possible to generate an appropriately executable trajectory that corresponds to a medium- to long-term structure.

[0084] The control device 300 can be configured to generate a trajectory by repeating a series of steps including generating a task trajectory, generating a skill trajectory, and generating a motion trajectory multiple times, thereby enabling the generation of a more appropriate trajectory.

[0085] In the present disclosure, an example has been given in which the state vector includes information indicating the position of a robot, such as an arm robot, performing processing, information indicating the position of an object to be controlled, such as object A or object B, and information indicating a skill being executed by the robot at that time. However, the state vector may include only part of the information exemplified above, such as information indicating only the position of the robot. Furthermore, the state vector may include any information other than the information exemplified above, such as information indicating a subgoal, such as an object closest to the robot.

[0086] Furthermore, in the present disclosure, an example has been given in which learning device 200 performs both learning using a task trajectory and learning using a skill trajectory. However, learning device 200 may be configured to generate only one of the task trajectory and the skill trajectory and perform learning using the generated trajectory. In other words, learning device 200 does not necessarily have to perform both learning using a task trajectory and learning using a skill trajectory.

[0087] Furthermore, in the present disclosure, information indicating a task state and information indicating a skill state are given as examples of information extracted by the long-period state extraction unit 252 that may change at periods longer than the control period. However, the long-period state extraction unit 252 may extract information that may change at periods longer than the control period other than the above examples. For example, the long-period state extraction unit 252 may perform extraction processing according to information included in the state vector, such as extracting information that may change at periods longer than the control period by focusing on changes in subgoals. Similarly, the learning trajectory generation unit 253 may generate a learning trajectory according to the information extracted by the long-period state extraction unit 252, and the learning device 200 may train a diffusion model according to the learning trajectory generated by the learning trajectory generation unit 253.

[0088] Second Embodiment Next, a configuration example of a learning device 500 will be described with reference to Fig. 15 to Fig. 17. Fig. 15 is a diagram showing an example of the hardware configuration of the learning device 500. Fig. 16 is a block diagram showing an example of the configuration of the learning device 500. Fig. 17 is a flowchart showing an example of the operation of the learning device 500.

[0089] In a second embodiment of the present disclosure, a learning device 500 that is an information processing device that learns a plurality of diffusion models according to a learning trajectory will be described. FIG. 15 shows an example of the hardware configuration of the learning device 500. Referring to FIG. 15, the learning device 500 has, as an example, the following hardware configuration: a CPU (Central Processing Unit) 501 (arithmetic device); a ROM (Read Only Memory) 502 (storage device); a RAM (Random Access Memory) 503 (storage device); a group of programs 504 loaded into the RAM 503; a storage device 505 that stores the group of programs 504; a drive device 506 that reads and writes data from and to a storage medium 510 external to the information processing device; a communication interface 507 that connects to a communication network 511 external to the information processing device; an input / output interface 508 that inputs and outputs data; and a bus 509 that connects the various components.

[0090] 16 by the CPU 501 acquiring and executing the program group 504. The program group 504 is stored in advance in the storage device 505 or the ROM 502, for example, and is loaded into the RAM 503 or the like by the CPU 501 for execution as needed. The program group 504 may be supplied to the CPU 501 via the communication network 511, or may be stored in advance in the recording medium 510, with the drive device 506 reading out the program and supplying it to the CPU 501.

[0091] 15 shows an example of the hardware configuration of the learning device 500. The hardware configuration of the learning device 500 is not limited to the above. For example, the learning device 500 may be configured with only a part of the above configuration, such as not including the drive device 306. Furthermore, the CPU 501 may be a GPU, as exemplified in the first embodiment.

[0092] The extraction unit 521 extracts information that may change at a period longer than the control period from the trajectory information including state information for each control period along a time series. For example, the extraction unit 521 can extract information indicating a target object or information indicating an execution skill from the trajectory information.

[0093] The generation unit 522 generates a learning trajectory that can accommodate changes in a period longer than the control period, based on the trajectory information and the information extracted by the extraction unit 521. For example, the generation unit 522 can generate a learning trajectory according to information indicating a target object, or can generate a learning trajectory according to information indicating an execution skill.

[0094] The above is an example of the configuration of the learning device 500. Next, an example of the operation of the learning device 500 will be described with reference to FIG.

[0095] 17 shows an example of the operation of the learning device 500. Referring to Fig. 17, the extraction unit 521 extracts information that may change at a period longer than the control period from trajectory information including state information for each control period along a time series (step S301).

[0096] The generating unit 522 generates a learning trajectory that can accommodate changes in a period longer than the control period, based on the information extracted by the extracting unit 521 and the trajectory information (step S302).

[0097] As described above, the learning device 500 includes an extraction unit 521 and a generation unit 522. With this configuration, the generation unit 522 can generate a learning trajectory that can respond to changes in periods longer than the control period, based on the information extracted by the extraction unit 521 and the trajectory information. As a result, the diffusion model can be trained using the generated learning trajectory. This allows learning that captures the medium- to long-term structure. The extraction unit 521 can be implemented, for example, using the function of the long-period state extraction unit 252 described in the first embodiment. The generation unit 522 can be implemented using at least a portion of the function of the learning trajectory generation unit 253 for generating a task trajectory or a skill trajectory.

[0098] The learning device 500 described above can be realized by incorporating a predetermined program into an information processing device such as the learning device 500. Specifically, a program according to another aspect of the present disclosure is a program for causing an information processing device to perform processing to extract information that may change at periods longer than the control period from trajectory information including state information for each control period along a time series, and generate a learning trajectory that can accommodate changes at periods longer than the control period based on the extracted information and the trajectory information.

[0099] Furthermore, a generation method executed by an information processing device such as the learning device 500 described above is a method in which the information processing device extracts information that may change at periods longer than the control period from trajectory information that includes state information for each control period along a time series, and generates a learning trajectory that can respond to changes at periods longer than the control period based on the extracted information and the trajectory information.

[0100] Any program having the above-described configuration, or a computer-readable recording medium having the program recorded thereon, or a generation method, etc., can achieve the same functions and effects as the above-described learning device 500, and therefore can achieve the above-described objective of the present disclosure.

[0101] <Supplementary Notes> Part or all of the above embodiments can be described as follows: Below, an overview of the learning device and the like in the present disclosure will be described. However, the present disclosure is not limited to the following configuration.

[0102] (Supplementary Note 1) A learning device comprising: an extraction unit that extracts information that may change in a period longer than a control period from trajectory information including state information for each control period along a time series; and a generation unit that generates a learning trajectory that can accommodate changes in a period longer than the control period based on the information extracted by the extraction unit and the trajectory information. (Supplementary Note 2) The learning device according to Supplementary Note 1, wherein the generation unit generates the learning trajectory including initial state information and final state information among state information in which the object indicated by the information extracted by the extraction unit is the same. (Supplementary Note 3) The learning device according to Supplementary Note 1 or Supplementary Note 2, wherein the extraction unit extracts information indicating a target object included in state information as information that may change in a period longer than the control period, and the generation unit generates the learning trajectory including initial state information and final state information among a series of state information in which the target object is the same among state information included in the trajectory information. (Supplementary Note 4) The learning device according to any one of Supplementary Notes 1 to 3, wherein the extraction unit extracts information indicating an execution skill included in state information as information that may change in a period longer than a control period, and the generation unit generates the learning trajectory including first state information and last state information of a series of state information having the same execution skill from the state information included in the trajectory information. (Supplementary Note 5) The learning device according to any one of Supplementary Notes 1 to 4, wherein the generation unit generates a learning trajectory that can respond to changes in a period longer than the control period, and generates the learning trajectory including state information that is adjacent in time series from the trajectory information. (Supplementary Note 6) The learning device according to any one of Supplementary Notes 1 to 5, comprising a learning unit that learns a diffusion model using the learning trajectory generated by the generation unit. (Supplementary Note 7) The learning device according to Supplementary Note 6, wherein the learning unit learns the diffusion model for each type of the learning trajectory generated by the generation unit.(Supplementary Note 8) The learning device according to Supplementary Note 6 or Supplementary Note 7, further comprising: a trajectory generation unit that generates a trajectory using the diffusion model learned by the learning unit, wherein the generation unit generates the learned trajectory that can accommodate changes in a period longer than a control period, and generates the learned trajectory consisting of state information that is adjacent in time series from the trajectory information, wherein the trajectory generation unit generates a trajectory using an early diffusion model learned based on the learned trajectory that can accommodate changes in a period longer than the control period, and then generates a trajectory using an early diffusion model learned based on the learned trajectory consisting of state information that is adjacent in time series. (Supplementary Note 9) A generation method in which an information processing device extracts information that may change in a period longer than the control period from trajectory information including state information for each control period along the time series, and generates a learned trajectory that can accommodate changes in a period longer than the control period based on the extracted information and the trajectory information. (Supplementary Note 10) A program for executing in an information processing device the process of extracting information that may change at a period longer than the control period from trajectory information including state information for each control period along a time series, and generating a learning trajectory that can respond to changes at a period longer than the control period based on the extracted information and the trajectory information.

[0103] Note that some or all of the configurations described in Supplementary Notes 2 to 8 that are dependent on the learning device described in Supplementary Note 1 may also be dependent in a similar dependent relationship on the generation method described in Supplementary Note 9 and the recording medium described in Process 10. Furthermore, not limited to Supplementary Notes 9 and 10, some or all of the configurations described as Supplements may also be dependent on various hardware, software, various recording means for recording software, or systems within the scope of the above-described embodiments.

[0104] The programs described in the above embodiments and appendices may be stored in a storage device or a computer-readable recording medium, such as a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0105] Although the present disclosure has been described above with reference to the above-described embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0106] 100 Control system 200 Learning device 210 Operation input unit 220 Screen display unit 230 Communication I / F unit 240 Memory unit 241 Trajectory information 242 Program 250 Arithmetic processing unit 251 Trajectory information acquisition unit 252 Long-period state extraction unit 253 Learning trajectory generation unit 254 Task trajectory learning unit 255 Skill trajectory learning unit 256 Motion trajectory learning unit 257 Output unit 300 Control device 310 Operation input unit 320 Screen display unit 330 Communication I / F unit 340 Memory unit 341 Model information 342 Program 350 Arithmetic processing unit 351 Initial information acquisition unit 352 Task trajectory generation unit 353 Skill trajectory generation unit 354 Motion trajectory generation unit 355 Control input calculation unit 356 Robot control unit 400 Robot 500 Learning device 501 CPU 502 ROM 503 RAM 504 Program group 505 Storage device 506 Drive device 507 Communication interface 508 Input / output interface 509 Bus 510 Recording medium 511 Communication network 521 Extraction unit 522 Generation unit

Claims

1. A learning device having: an extraction unit that extracts information that may change at periods longer than a control period from trajectory information including state information for each control period along a time series; and a generation unit that generates a learning trajectory that can respond to changes at periods longer than the control period based on the information extracted by the extraction unit and the trajectory information.

2. A learning device as described in claim 1, wherein the generation unit generates the learning trajectory including initial state information and final state information among state information in which the information extracted by the extraction unit indicates the same target.

3. A learning device as described in claim 1, wherein the extraction unit extracts information indicating a target object included in state information as information that may change at a period longer than a control period, and the generation unit generates the learning trajectory including the first state information and the last state information of a series of state information in which the target object is the same among the state information included in the trajectory information.

4. A learning device as described in claim 1, wherein the extraction unit extracts information indicating an execution skill included in the state information as information that may change at a period longer than a control period, and the generation unit generates the learning trajectory including the first state information and the last state information of a series of state information in which the execution skill is the same among the state information included in the trajectory information.

5. A learning device as described in claim 1, wherein the generation unit generates a learning trajectory that can respond to changes in a period longer than a control period, and generates the learning trajectory consisting of state information that is adjacent in time series among the trajectory information.

6. A learning device according to claim 1, comprising a learning unit that learns a diffusion model using the learning trajectory generated by the generation unit.

7. A learning device according to claim 6, wherein the learning unit learns the diffusion model for each type of the learning trajectory generated by the generation unit.

8. A learning device as described in claim 6, further comprising a trajectory generation unit which generates a trajectory using the diffusion model learned by the learning unit, wherein the generation unit generates the learned trajectory capable of responding to changes in a period longer than a control period, and generates the learned trajectory consisting of state information which is adjacent in time series from the trajectory information, and the trajectory generation unit generates a trajectory using an early diffusion model learned based on the learned trajectory capable of responding to changes in a period longer than a control period, and then generates a trajectory using the early diffusion model learned based on the learned trajectory consisting of state information which is adjacent in time series.

9. A generation method in which an information processing device extracts information that may change at periods longer than the control period from trajectory information including state information for each control period along a time series, and generates a learning trajectory that can respond to changes at periods longer than the control period based on the extracted information and the trajectory information.

10. A program for implementing in an information processing device the process of extracting information that may change at periods longer than the control period from trajectory information including state information for each control period along a time series, and generating a learning trajectory that can respond to changes at periods longer than the control period based on the extracted information and the trajectory information.

Images