Information processing device, information processing method, and information processing program

Abstract

An information processing device according to the present disclosure comprises: a data acquisition unit that acquires, for each teaching means, a plurality of pieces of teaching data for teaching an operation to a robot; and a teaching operation processing unit that, on the basis of the acquired plurality of pieces of teaching data, generates teaching operation data for teaching the robot a series of operations in which some or all of the operations taught for each teaching means are combined.

Description

Information processing device, information processing method, and information processing program

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program.

[0002] To achieve advanced tasks using robots, such as remote control, there is a need for a technology for teaching robots how to perform certain actions. Known examples of such technology include a technology for teaching a robot a series of actions using one of a number of different teaching methods (e.g., a teaching method suitable for loading and unloading items, a teaching method suitable for welding, etc.) (see, for example, Patent Document 1). Also known is a technology for outputting an evaluation of a series of actions performed by a robot to a user (see, for example, Patent Document 2).

[0003] JP 5-143151 A JP 2022-175570 A

[0004] Conventional technology can change the teaching method for each series of actions, include in the output content the results obtained through a machine learning process from information about the series of actions, or prompt the user to redo the teaching depending on the evaluation value for the series of actions.

[0005] However, because the conventional technology teaches a series of movements using a single teaching method, some of the movements that make up the series of movements may not be suitable for that teaching method, so there is room for improvement in the quality of teaching to robots using the conventional technology.

[0006] Therefore, an object of the present disclosure is to propose an information processing device, an information processing method, and an information processing program that can improve the quality of teaching a robot.

[0007] The information processing device according to the present disclosure includes a data acquisition unit that acquires multiple pieces of teaching data for each teaching means to teach a robot an action, and a teaching action processing unit that generates teaching action data that teaches the robot a series of actions that combine some or all of the actions taught by each teaching means, based on the multiple pieces of teaching data acquired.

[0008] 1 is a diagram for explaining the purpose of an information processing system according to an embodiment. FIG. 2 is a diagram for explaining an overview of processing of an information processing system according to an embodiment. FIG. 3 is a block diagram showing the configuration of an information processing system according to a first embodiment. FIG. 4 is a diagram for explaining an example of generation of taught motion data. FIG. 5 is a diagram for explaining an example of piecing together of taught motion data by means of using a weighting function. FIG. 6 is a diagram for explaining an example of generation of a weighting function based on a basis function. FIG. 7 is a diagram for explaining an example of piecing together of taught motion data by means of using a physical model. FIG. 8 is a diagram for explaining an example of using coefficients of springs and dampers stored for each transition pattern of a teaching means. FIG. 9 is a diagram for explaining an example of piecing together of taught motion data by means of using a playback function of a robot. A flowchart showing the flow of information processing according to the first embodiment. A block diagram showing the configuration of an information processing system according to a second embodiment. FIG. 10 is a diagram for explaining an example of generation of taught motion data based on the evaluated quality of each motion. A flowchart showing the flow of information processing according to the second embodiment. FIG. 11 is a diagram for explaining an example of generation of taught motion data in which motions by a teaching means with the highest evaluation value for the same motion are combined among each motion. A flowchart showing the flow of information processing according to a modified example of the second embodiment. A block diagram (1) showing the configuration of an information processing system according to a third embodiment. FIG. 10 is a block diagram (2) showing the configuration of an information processing system according to a third embodiment. FIG. 11 is a diagram for explaining an example of prediction of an optimal teaching means. FIG. 12 is a diagram for explaining presentation of an optimal teaching means, etc.. FIG. 13 is a flowchart showing the flow of information processing when acquiring data according to the third embodiment. FIG. 14 is a flowchart showing the flow of information processing when predicting an optimal teaching means according to the third embodiment. FIG. 15 is a block diagram showing the configuration of an information processing system according to a fourth embodiment. FIG. 16 is a hardware configuration diagram showing an example of a computer that realizes the functions of an information processing device according to the present disclosure.

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same components are denoted by the same reference numerals, and redundant description will be omitted.

[0010] Hereinafter, embodiments of the present disclosure will be described in the following order: 1. First embodiment 1-1. Overview of information processing system according to first embodiment 1-2. Configuration of information processing system according to first embodiment 1-3. Flow of information processing according to first embodiment 2. Second embodiment 2-1. Configuration of information processing system according to second embodiment 2-2. Flow of information processing according to second embodiment 2-3. Modified example 3. Third embodiment 3-1. Configuration of information processing system according to third embodiment 3-2. Flow of information processing according to third embodiment 4. Fourth embodiment 5. Other embodiments 6. Effects of information processing device according to the present disclosure 7. Hardware configuration 8. Supplementary information

[0011] (1. First embodiment) (1-1. Overview of information processing system according to first embodiment) The purpose of an information processing system 1 according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram for explaining the purpose of an information processing system according to an embodiment.

[0012] The information processing system 1 includes an information processing device 10 (not shown in Fig. 1). The information processing device 10 is a device such as a computer. The information processing device 10 generates teaching action data to teach the robot 100 a series of actions constituting a task or the like required of the robot 100 by a teaching means that teaches the robot 100 the actions.

[0013] The robot 100 is a machine with physicality similar to that of a living organism or an autonomously controlled machine. In FIG. 1 , the robot 100 is equipped with a manipulator such as an arm. However, there are no particular limitations on the form of the robot 100 or whether the robot 100 is a machine in the real world or a virtual robot in a simulation environment. For example, the robot 100 may be a legged robot that moves using multiple mechanical legs, or may be a drone.

[0014] A task is an action required of the robot 100. In FIG. 1 , the task is composed of a series of actions, including an action of moving to a predetermined point in step S1, an action of grasping the object 200 in step S2, and an action of lifting the object 200 and spilling the liquid inside in step S3. However, the task is not particularly limited as long as it is an action required of the robot 100. A task may be a series of actions made up of multiple actions, or may be composed of a single action.

[0015] The predetermined point may be any point designated by the user, etc. In Fig. 1, the predetermined point is the point where the object 200 is placed.

[0016] The teaching means is a means for teaching the robot 100 to perform an action. For example, the teaching means may involve hardware or be written in software. In FIG. 1 , the teaching means is a VR (Virtual Reality) system and an exoskeleton-wearable device 400. The VR system 300 is a teaching means that allows the instructor to virtually experience a VR space using a VR device such as goggles attached to the instructor's eyes. The exoskeleton-wearable device 400 is an exoskeleton-type device that is attached to a part of the instructor's body, such as the instructor's fingers.

[0017] When the information processing device 10 generates data for teaching a series of actions to the robot 100 using a single teaching means, some of the series of actions may not be suitable for that teaching means. On the other hand, if the information processing device 10 can teach the robot 100 actions using teaching means suitable for each of the multiple actions that make up the series of actions, the quality of teaching to the robot 100 can be improved compared to when the series of actions is taught using a single teaching means. However, because it is difficult to change the teaching means for some or all of the multiple actions that make up a task, this has not been possible in the past.

[0018] Furthermore, the robot 100 is required to be able to handle a variety of tasks. However, in order for the robot 100 to be able to handle a variety of tasks, it is necessary to build a system that teaches the robot 100 using all of the teaching means required for the tasks. This makes the system large-scale, which may increase costs. Furthermore, since the teaching operations using the teaching means tend to be complicated, there is a risk that the quality of the teaching to the robot 100 may decline.

[0019] Therefore, the information processing device 10 performs a process of changing the teaching means for some or all of the multiple actions that make up a task in order to improve the quality of teaching to the robot 100 while avoiding increases in costs and complication of teaching operations. Below, as a premise for explaining this process, teaching means suitable for each of steps S1 to S3 in Fig. 1 will be explained.

[0020] A teaching means suitable for the operation of step S1 is the VR system 300. The VR system 300 allows the instructor to intuitively operate the robot 100 with realistic movements. Furthermore, when combined with a distance measurement sensor, the VR system 300 can accurately move the instructor or the robot 100 to a predetermined point where the object 200 is located. Therefore, the VR system 300 is suitable for the operation of the robot 100 in step S1 to move to a predetermined point.

[0021] The teaching means suitable for the operation of step S2 is the exoskeleton-wearable device 400. The exoskeleton-wearable device 400 can reflect the movement of the instructor's body, such as the instructor's fingers, in the operation of the robot 100. Furthermore, when combined with a torque / force sensor, the exoskeleton-wearable device 400 can grasp the object 200 with an appropriate force without crushing it. Therefore, the exoskeleton-wearable device 400 is suitable for the operation of the robot 100 in step S2 to grasp the object 200.

[0022] Furthermore, a teaching means suitable for the operation of step S3 is a combination of the VR system 300 and the exoskeleton-mounted device 400. The VR system 300 allows intuitive operation, such as operating the end effector of the arm of the robot 100. Furthermore, when combined with a distance measurement sensor, the VR system 300 can lift the object 200 to an appropriate point.

[0023] The exoskeleton-wearable device 400 is suitable for complex and highly flexible operations, such as controlling a multi-fingered hand. Furthermore, when combined with a torque / force sensor, the exoskeleton-wearable device 400 can tilt the object 200 so that the liquid inside spills without crushing the object 200. Therefore, the combination of the VR system 300 and the exoskeleton-wearable device 400 can utilize the advantages of both systems, making it suitable for the action of lifting the object 200 in step S3 and spilling the liquid inside.

[0024] 1 is also suitable for the operation of step S3. Program 500 is a program implemented in robot 100. Program 500 describes the operation to be taught to robot 100. Program 500 can describe operations expressed as numerical values, such as the distance to the point where robot 100 lifts object 200 and the grip strength with which robot 100 grasps object 200. Therefore, program 500 is suitable for the operation of lifting object 200 and spilling the liquid inside in step S3, which requires accurate position control.

[0025] Next, an overview of the processing of the information processing system according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram for explaining the overview of the processing of the information processing system according to the embodiment. The information processing device 10, not shown in Fig. 2, acquires multiple pieces of teaching data for each teaching means to teach the robot 100 how to perform an action. For example, the information processing device 10 acquires first teaching data 600 for teaching the robot 100 a series of actions from step S1 to step S3 using the VR system 300.

[0026] The information processing device 10 acquires second teaching data 700 for teaching the robot 100 the action of step S2, which is part of the above-mentioned series of actions, by the exoskeleton-mounted device 400, which is teaching means separate from the VR system 300, which is the first teaching means. The information processing device 10 also acquires third teaching data 800 for teaching the robot 100 the action of step S3, which is part of the above-mentioned series of actions, by the program 500, which is teaching means separate from the VR system 300, which is the first teaching means.

[0027] Next, based on the acquired multiple pieces of teaching data, the information processing device 10 generates teaching motion data that teaches the robot 100 a series of motions that combine some or all of the motions taught by each teaching means.

[0028] 2, the series of actions from step S1 to step S3 are actions that have already been taught to the robot 100 by the VR system 300 based on the first teaching data 600. In this case, the information processing device 10 generates teaching action data based on the second teaching data 700 and the third teaching data 800, which teaches part of the series of actions by another teaching means.

[0029] For example, the information processing device 10 overwrites the action of step S2, which is part of the actions of steps S1 to S3 that have already been taught by the VR system 300, with the action taught by the exoskeleton-wearable device 400. Furthermore, the information processing device 10 overwrites the action of step S3 that has already been taught by the VR system 300 with the action taught by the program 500.

[0030] This allows the robot 100 to be taught a series of actions that combine the action of step S1 taught by the VR system 300, the action of step S2 taught by the exoskeleton-wearable device 400, and the action of step S3 taught by the program 500.

[0031] In this way, the information processing device 10 can easily combine actions taught by various teaching means, and can easily teach actions to the robot 100 using teaching means suitable for each of a plurality of actions that make up a series of actions. Therefore, the information processing device 10 can improve the quality of teaching to the robot 100 while avoiding increases in costs, etc.

[0032] (1-2. Configuration of Information Processing System According to First Embodiment) Next, the configuration of the information processing system 1 according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the information processing system according to the first embodiment.

[0033] (Information Processing Device) The information processing device 10 includes an action teaching unit 1*1, a data acquiring unit 1*2, a data storage unit 201, an action processing designating unit 301, an instruction action processing unit 302, an action command value generating unit 401, and an actuation unit 402.

[0034] The motion teaching unit 1*1 is made up of a first motion teaching unit 111 to an Nth motion teaching unit 1N1. The data acquisition unit 1*2 is made up of a first data acquisition unit 112 to an Nth data acquisition unit 1N2. The number N is not particularly limited as long as it is a natural number of 2 or more.

[0035] Each of the above-described blocks included in the information processing device 10 also functions as a control unit. The control unit is a central processing unit (CPU) or a micro processing unit (MPU) that executes a program stored in a storage device such as a random access memory (RAM). The control unit may be an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0036] (Movement Teaching Unit) The movement teaching unit 1*1 is a block that generates teaching data for teaching a movement to the robot by a predetermined teaching means based on the content taught by the instructor using a predetermined teaching means. For example, the first movement teaching unit 111 generates first teaching data for teaching a movement to the robot by the first teaching means based on the content taught by the first instructor 11 using the first teaching means.

[0037] The second movement teaching unit 121 generates second teaching data for teaching the robot a movement by the second teaching means based on the content taught by the second instructor 12 by the second teaching means. The Nth movement teaching unit 1N1 generates Nth teaching data for teaching the robot a movement by the Nth teaching means based on the content taught by the Nth instructor 1N by the Nth teaching means.

[0038] The first teaching data and the second teaching data are not limited to the above-described first teaching data 600 and second teaching data 700. The first instructor 11 to the Nth instructor 1N may have the same or different levels of proficiency. For example, the first instructor 11 may have a beginner level of proficiency, while the second instructor 12 and the Nth instructor 1N may have an expert level of proficiency.

[0039] Furthermore, the first teaching means to the Nth teaching means are not particularly limited. For example, the first teaching means to the Nth teaching means may be arbitrarily selected from among those involving hardware, those written in software, etc. Those involving hardware include a remote controller, a leader / follower type device, a VR system, an exoskeleton-mounted device, a mouse, direct teaching, motion capture data, etc.

[0040] A remote controller is a teaching tool that controls the position and posture of the end effector of a robot's arm. A teacher can easily operate a remote controller. However, a remote controller is not suitable for teaching the posture of a robot or the movement of each joint.

[0041] A leader-follower type device is a teaching method in which a follower type device follows the movement of a leader type device as the leader type device is operated by an instructor. While a leader-follower type device can easily adjust the position and force of a robot, it requires complex hardware.

[0042] A VR system is a teaching method that allows a teacher to virtually experience a VR space using a VR device. A VR system allows a teacher to intuitively operate a robot with realistic movements, enabling intuitive operations such as operating the end effector of a robot arm. However, a VR system is not suitable for teaching a robot complex movements with a high degree of freedom, such as controlling a multi-fingered hand.

[0043] An exoskeleton-wearable device is a teaching tool that is attached to a part of the instructor's body, such as the instructor's fingers. The exoskeleton-wearable device can reflect the instructor's body movements in the robot's movements. However, when an exoskeleton-wearable device is used as a teaching tool, a conversion process is required to absorb the physical differences between humans and robots.

[0044] A mouse is a teaching tool used as a type of computer input device. A mouse allows a teacher to control a robot through simple operation of buttons and a wheel. However, because mouse operation is limited, the degree of freedom in teaching the robot to move is low. For this reason, the movements to be taught to the robot must be created in advance.

[0045] Direct teaching is a teaching method in which a teacher directly operates a robot in a gravity-compensated state. Direct teaching allows a teacher to intuitively teach a robot. However, direct teaching limits the actions that can be taught to a robot.

[0046] Motion capture data is a teaching tool based on recordings of the movements of a teacher wearing tracking markers.

[0047] The software etc. that can be selected as the first teaching means to the Nth teaching means include programs, natural language, visual information such as images, time-series data regarding the state of the robot and the environment around the robot, etc.

[0048] A program is a teaching means in which the robot's operations are written in advance. The program may be written using a programming language, or may be a program that defines the relationship between input and output, such as a function or neural network, or may be a language written in a form for controlling a robot or machine.

[0049] The natural language is a teaching means in which the natural language for the actions to be taught to the robot is expressed by voice or text. The natural language may be, for example, a natural language spoken by the instructor, a natural language written on paper or on a computer, or a natural language output from a large-scale language model that describes the robot's actions in detail.

[0050] Visual information such as images is a teaching means for teaching a robot. For example, the visual information such as images may be video or still images of the robot or a living creature or animal with a similar physicality to the robot.

[0051] The time-series data relating to the robot and its surrounding environment is a teaching means that represents teaching data relating to the robot and its surrounding environment over time. For example, the time-series data is teaching data recorded in the past that teaches the robot a series of actions. Furthermore, if the robot has previously been taught an action using the time-series data, the teaching data can be played back to teach the robot the same action as the previously taught action.

[0052] The teaching means can also be combined with sensors, such as a sensor that measures the state of the robot, a sensor that measures the environment around the robot, and a sensor that measures objects around the robot.

[0053] Specifically, the sensors that measure the state of the robot are encoders, torque / force sensors, tactile sensors, sensors that measure current values, sensors that measure position / velocity / acceleration / force, and IMUs (Inertial Measurement Units).Specifically, the sensors that measure the environment are RGB (Red-Green-Blue color model) sensors, RGB-D (Depth) sensors, depth sensors, and LiDAR (Light Detection and Ranging).Specifically, the sensors that measure objects are the above sensors attached to the objects.

[0054] (Data Acquisition Unit) The data acquisition unit 1*2 is a block that acquires multiple pieces of teaching data generated by the operation teaching unit 1*1 for each teaching means. The data acquisition unit 1*2 can also acquire data related to the teaching data.

[0055] For example, the data acquisition unit 1*2 acquires various sensor data as data related to the teaching data, teaching data processed by functions or networks, subgoals, final goals, and timestamps of the actions to be taught to the robot, which are written as character strings in the program. Specifically, the data acquisition unit 1*2 acquires position, speed, acceleration, and force as various sensor data.

[0056] The data acquisition unit 1*2 can also process teaching data, etc. As an example, the data acquisition unit 1*2 processes teaching data, etc. by acquiring signals from various sensors, preprocessing such as filtering and data processing, and adjusting the timing to start teaching an operation when multiple sensors are used. The teaching start timing is the time when teaching of an operation is started for each teaching means. The number of teaching start timings is not particularly limited.

[0057] As another example, the data acquisition unit 1*2 performs a process of analyzing the natural language teaching data and linking the natural language analysis result to the robot's skills. As another example, the data acquisition unit 1*2 performs a process of converting the teaching data of visual information such as videos and images into robot movements.

[0058] (Data Storage Unit) The data storage unit 201 stores multiple pieces of teaching data, etc. acquired by the data acquisition unit 1*2. For example, the data storage unit 201 is a block that stores multiple pieces of teaching data, etc. output from the data acquisition unit 1*2, by linking them with the numbers of the teaching means and the teaching data. Specifically, if the number of the first teaching means or the first teaching data is 1, the data storage unit 201 stores a table in which the number 1 is linked to the first teaching means and the first teaching data.

[0059] (Movement Processing Designation Unit) The movement processing designation unit 301 is a block that designates, to the teaching movement processing unit 302, a processing method (overwriting, selection, mixing, etc.) for a plurality of pieces of teaching data and a teaching start timing.

[0060] For example, the operation processing designation unit 301 designates some or all of the following as the instruction start timing: the time designated by the user, the time when the similarity between multiple instruction data is highest, and the time when a change in the sensor value is detected.

[0061] As one example, the timing designated by the user is a timing designated by the user among the timestamps attached to the teaching data. As another example, the timing designated by the user is a timing at which the user stops the robot from playing back the motion of the teaching data at a timing of the user's choosing while the robot is actually moving the robot. These timings designated by the user are also timings at which the teaching motion processing unit 302 receives a motion stop signal from the motion processing designation unit 301.

[0062] The timing when the degree of similarity between the plurality of teaching data is highest is, for example, the timing when the degree of similarity between the final state of each of the plurality of motions constituting the series of motions taught by the first teaching means and the initial state of the motion taught by the second teaching means is highest. For example, an index such as DTW (Dynamic Time Warping) can be used to measure the degree of similarity.

[0063] The time when a change in the sensor value is detected is, for example, the time when the robot's behavior changes, which is detected when a sensor event occurs, where the sensor value changes by a predetermined value or more. The predetermined value is, for example, the threshold value of a sensor such as a torque / force sensor or a tactile sensor, when it is generally recognized that the robot's behavior changes, such as when the robot moves to a predetermined point and then grasps an object.

[0064] (Teaching motion processing unit) The teaching motion processing unit 302 is a block that generates teaching motion data that teaches the robot a series of motions that combine some or all of the motions taught for each teaching means, based on the multiple teaching data acquired by the data acquisition unit 1 * 2. For example, the teaching motion processing unit 302 overwrites, selects, mixes, etc. the multiple teaching data stored in the data storage unit 201 to generate teaching motion data in the same format as the teaching data processed by the data acquisition unit 1 * 2.

[0065] An example of generation of teaching motion data will be described below with reference to Fig. 4. Fig. 4 is a diagram for explaining an example of generation of teaching motion data.

[0066] 4, a data acquisition unit 1*2 acquires first teaching data 600 and second teaching data 700. A teaching operation processing unit 302 splices together the first teaching data 600 and the second teaching data 700 that are taught to the robot 100 by each teaching means. In FIG. 4, the teaching operation processing unit 302 splices together these pieces of teaching data at the teaching start timing 900, etc., based on the teaching start timing 900 and the splicing means.

[0067] In this way, when the teaching start timing 900 is set in advance, the teaching operation processing unit 302 generates teaching operation data so that teaching of some or all of the multiple operations that make up a series of operations begins at the preset teaching start timing 900.

[0068] For example, the teaching operation processing unit 302 generates teaching operation data so that some or all of the timing specified by the user, the timing when the similarity between multiple teaching data is highest, and the timing when a change in the sensor value is detected, correspond to the teaching start timing 900.

[0069] Furthermore, when the teaching operation processing unit 302 connects these pieces of teaching data using the connecting means, it generates teaching operation data based on the continuity of the multiple pieces of teaching data. If unprocessed teaching data is connected, discontinuous operations will occur. Therefore, if discontinuous operations are taught to the robot 100, safety will be threatened and the robot 100 will be damaged.

[0070] Therefore, the teaching operation processing unit 302 connects the teaching data while ensuring the continuity of the operation of the robot 100. To this end, the teaching operation processing unit 302 selects, as a connecting means, some or all of a means using a weighting function, a means using a physical model, and a means using the playback function of the robot 100.

[0071] As a result, the teaching movement processing unit 302 generates teaching movement data in which some or all of each movement is combined so that the weighting functions associated with each movement are approximately continuous, or each movement is approximately continuous in physical space, or the actual movement of each movement is approximately continuous.

[0072] (Means Using Weighting Functions) An example of piecing together teaching data using means using a weighting function will be described below with reference to FIG. 5. This means is a means that places importance on the continuity of the motion command values ​​that are values ​​that instruct the robot to move. FIG. 5 is a diagram for explaining an example of piecing together teaching data using means using a weighting function. In FIG. 5, when the weighting function wn(t) of FIG. 5 is used, the teaching motion data is expressed as a function of the following formula (1). The shape of the weighting function in FIG. 5 is merely an example. The shape of the weighting function is arbitrary.

[0073]

[0074] In formula (1), n ​​is the number of the teaching means or teaching data. y' is the position, speed, acceleration, force information, etc. of the robot detected by the sensor from the series of movements taught by the teaching movement data. yn is the position, speed, acceleration, force information, etc. of the robot detected by the sensor from the movements taught by the teaching data. wn(t) is a weighting function, which is the time series data of wn, which is a weight expressed as a scalar value.

[0075] The teaching operation processing unit 302 calculates a first weighting function w associated with the operation based on the first teaching data 600. １ (t) and a second weighting function w associated with the operation based on the second teaching data 700. ２ These weighting functions are added together so that (t) approximately matches (t).

[0076] For example, in FIG. 5, at time t １ and time t ２ In this case, the first weight function w １ (t) and the second weighting function w ２ Therefore, the teaching operation processing unit 302 determines whether the time t １ At time t, the first teaching means is switched to the second teaching means. ２ In the above, the teaching operation processing unit 302 generates teaching operation data for switching the second teaching means to the first teaching means. The teaching operation processing unit 302 can also obtain wn(t) by learning.

[0077] The teaching motion processing unit 302 uses at least one of a basis function and a function representing a neural network as the weighting function wn(t). First, an example will be described in which the teaching motion processing unit 302 uses a basis function such as a radial basis function as the weighting function wn(t). For example, the teaching motion processing unit 302 uses a plurality of basis functions ψ where l, which represents the number of the basis function, ranges from 1 to L, as shown in the following formula (2): ｌ The weight function is generated by adding (t).

[0078]

[0079] L indicates that the basis function number is L, and is therefore essentially the number of basis functions. θ is a parameter (policy parameter) described as the weight of the basis function.

[0080] An example of generating a weighting function based on a basis function will be described below with reference to Fig. 6. Fig. 6 is a diagram for explaining an example of generating a weighting function based on a basis function.

[0081] The teaching operation processing unit 302 calculates the basis function ψ ｌ Based on (t), the basis function ψ ｌ The teaching operation processing unit 302 acquires a parameter θ to be multiplied by (t). For example, the teaching operation processing unit 302 acquires the parameter θ through learning. Because the teaching operation processing unit 302 can arbitrarily determine the value of the parameter θ through learning, it can arbitrarily change the shape of the weighting function wn(t), which is proportional to the value of the parameter θ.

[0082] When the teaching operation processing unit 302 acquires the parameter θ corresponding to each teaching means by learning, it calculates the first weighting function w corresponding to the first teaching means. １ (t), a second weighting function w corresponding to the second teaching means ２ For example, the teaching operation processing unit 302 acquires the parameter θ of equation (2) constituting the weighting function wn(t) by performing reinforcement learning to minimize the acceleration value of the robot.

[0083] Next, an example will be described in which the teaching operation processing unit 302 uses a neural network as the weighting function w(t). For example, when the input of the neural network is a function fρ(t) proportional to time (t) and a parameter ρ, and the output is a weight wn, the teaching operation processing unit 302 obtains the parameter ρ by training the neural network to satisfy the following formula (3):

[0084]

[0085] Furthermore, when the input of the neural network is a function fρ(s) proportional to the state (s) and the parameter ρ, and the output is a weight wn, the teaching operation processing unit 302 obtains the parameter ρ by training the neural network to satisfy the following equation (4): The state (s) is, for example, a sensor value or a state observable (RGB data that is the observation result of the environment or objects around the robot, the state of the robot, etc.).

[0086] The weight wn, which is the output of the neural network, may be the weight of a specific teaching means, or may be the weights of all the first teaching means to the Nth teaching means output simultaneously.

[0087] (Means Using a Physical Model) Next, an example of piecing together teaching data using a means using a physical model will be described with reference to Fig. 7. This means places importance on the continuity in the physical space of the movements of the robot 100. In other words, the teaching movement processing unit 302 uses a physical model when generating teaching movement data based on the continuity in the physical space.

[0088] 7 is a diagram illustrating an example of piecing together teaching data by means of a physical model. In FIG. 7, the teaching operation processing unit 302 smoothly connects a final state 601 of a first teaching data 600 with an initial state 701 of a second teaching data 700, which is the next teaching data, using virtual springs and dampers.

[0089] For example, the teaching operation processing unit 302 generates an operation command value for the robot 100 to move slowly from the final state 601 to the initial state 701, assuming that there is a spring and a damper, based on the following equation (5) which represents a state 650 between the final state 601 and the initial state 701. As a result, the robot 100 smoothly transitions between the operations taught by the first teaching means and the second teaching means while reliably executing the operations taught by these teaching means. "Slowly" refers to, for example, a speed at which the acceleration on the left side of the following equation (5) is minimized.

[0090]

[0091] In equation (5), α is a parameter, k is a virtual spring coefficient, and c is a virtual damper coefficient. １ is the position y of the robot 100 whose movement is taught based on the first teaching data 600. １ is the velocity of the robot 100, which corresponds to the differential value of y ２ is the position y of the robot 100 whose movement is taught based on the second teaching data 700. ２ is the velocity of the robot 100 corresponding to the differential value of

[0092] The coefficients of the springs and dampers may be set by the user or may be obtained by learning by the teaching operation processing unit 302 .

[0093] The teaching operation processing unit 302 can also use spring and damper coefficients stored for each transition pattern of the teaching means. An example of using the spring and damper coefficients stored for each transition pattern of the teaching means will be described below with reference to FIG. 8 . FIG. 8 is a diagram for explaining an example of using the spring and damper coefficients stored for each transition pattern of the teaching means. For example, when transitioning from the second teaching means to the first teaching means, the teaching operation processing unit 302 determines the spring coefficient k to be 1.0 and the damper coefficient c to be 3.0 based on the table 651 in FIG. 8 .

[0094] (Means for Using the Robot's Playback Function) An example of piecing together teaching data using a means for using the robot's playback function will be described below with reference to Fig. 9. This means places emphasis on the continuity of the actual movement, including errors of the robot. Fig. 9 is a diagram for explaining an example of piecing together teaching data using a means for using the robot's playback function.

[0095] 9 , the teaching operation processing unit 302 plays back the robot's motion taught by the first teaching means based on the first teaching data 600. The teaching operation processing unit 302 stops the playback midway at a timing corresponding to a teaching start timing 900 set by the user. The teaching operation processing unit 302 also starts teaching the robot 100 a motion taught by the second teaching means based on the second teaching data 700, continuing from the teaching start timing 900.

[0096] In this way, when generating teaching motion data based on the continuity of actual motion, the teaching motion processing unit 302 generates the teaching motion data so that the timing specified by the user becomes the teaching start timing.

[0097] (Combination) The teaching operation processing unit 302 can also combine means using a weighting function, means using a physical model, and means using the playback function of the robot.

[0098] For example, the teaching motion processing unit 302 reproduces the motion of the robot based on the first teaching data 600 by means of the robot's playback function, while acquiring teaching data that is used to teach the robot a motion by each of the multiple teaching means from the data storage unit 201. Thereafter, the teaching motion processing unit 302 optimizes the weight by means of a weighting function, and then generates smooth teaching motion data by means of a physical model.

[0099] (Movement command value generation unit) The movement command value generation unit 401 generates movement command values ​​for the robot based on the teaching movement data generated by the teaching movement processing unit 302. For example, the movement command value generation unit 401 generates, as movement command values, the position, speed, acceleration, and force of the robot, as well as control gains that are gains for controlling the movement of the robot.

[0100] (Actuation Unit) The actuation unit 402 operates the robot based on the operation command values ​​generated by the operation command value generation unit 401. For example, the actuation unit 402 operates the actuators of the robot based on the position, speed, acceleration, force, and control gain of the robot as the operation command values.

[0101] (1-3. Flow of Information Processing According to First Embodiment) Next, the flow of information processing according to the first embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the flow of information processing according to the first embodiment.

[0102] In step S11, the first motion teaching unit 111 generates first teaching data for teaching the robot a motion using a first teaching means. The first data acquisition unit 112 acquires the first teaching data generated by the first motion teaching unit 111.

[0103] In step S12, if the data storage unit 201 determines that teaching to the robot by the first teaching means has ended (step S12; Yes), it stores the first teaching data in step S13. In step S12, if the data storage unit 201 determines that teaching to the robot by the first teaching means has not ended (step S12; No), it returns to step S11.

[0104] In step S14, the teaching operation processing unit 302 reproduces the robot operation taught to the robot by the first teaching means based on the first teaching data. The user can then confirm the teaching result from the reproduced robot operation.

[0105] In step S15, if the user determines that additional instruction to the robot is necessary (step S15; Yes), in step S16, the Nth motion teaching unit 1N1 generates Nth teaching data for teaching the robot an operation by the Nth teaching means. For example, the second motion teaching unit 121 generates second teaching data for teaching the robot an operation by the second teaching means.

[0106] Further, the Nth data acquisition unit 1N2 acquires the Nth teaching data generated by the Nth motion teaching unit 1N1. For example, the second data acquisition unit 122 acquires the second teaching data generated by the second motion teaching unit 121. In this way, the data acquisition unit 1*2 acquires multiple pieces of teaching data for each teaching means.

[0107] In step S15, if the user determines that additional instruction to the robot is not necessary (step S15; No), the information processing device 10 ends the information processing.

[0108] In step S17, if the data storage unit 201 determines that the teaching of the robot by the Nth teaching means has ended (step S17; Yes), in step S18, the data storage unit 201 stores the Nth teaching data.

[0109] In step S17, if the data storage unit 201 determines that the teaching of the robot by the Nth teaching means has not ended (step S17; No), the process returns to step S16. In this case, in step S16, for example, the third movement teaching unit 131 generates third teaching data for teaching the robot a movement by the third teaching means. In addition, the third data acquisition unit 132 acquires the third teaching data generated by the third movement teaching unit 131.

[0110] In step S19, the operation processing designation unit 301 designates a teaching start timing. In step S20, the teaching operation processing unit 302 generates teaching operation data for teaching the robot a series of operations that combines some or all of the operations taught for each teaching means by processing that integrates the multiple teaching data. For example, the teaching operation processing unit 302 generates teaching operation data that switches from teaching the robot by a first teaching means to teaching the robot by an Nth teaching means at the teaching start timing designated by the operation processing designation unit 301.

[0111] In step S21, if the motion command value generating unit 401 determines that the process of generating the teaching motion data has ended (step S21; Yes), it generates a motion command value based on the teaching motion data generated by the teaching motion processing unit 302. Thereafter, the information processing device 10 ends the information processing. In step S21, if the motion command value generating unit 401 determines that the process of generating the teaching motion data has not ended (step S21; No), it returns to step S16.

[0112] (2. Second embodiment) (2-1. Configuration of information processing system according to second embodiment) The information processing device 10 can also evaluate the quality of each action taught to the robot for each teaching means. The configuration of an information processing system 1A according to the second embodiment will be described using FIG. 11. FIG. 11 is a block diagram showing the configuration of the information processing system according to the second embodiment. The information processing system 1A includes an information processing device 10A instead of the information processing device 10.

[0113] (Information Processing Device) The information processing device 10A further includes a motion evaluation unit 303. The information processing system 1A also includes a data storage unit 201A, a teaching motion processing unit 302A, and a motion processing designation unit 301A, instead of the data storage unit 201, the teaching motion processing unit 302, and the motion processing designation unit 301. The motion evaluation unit 303 also functions as a control unit.

[0114] (Data Storage Unit) The data storage unit 201A further stores an evaluation value of the quality of each action taught to the robot for each teaching means, which is output from the action evaluation unit 303. For example, the data storage unit 201A stores a table in which teaching data expressed as time-series data is linked to evaluation values ​​of the quality of actions based on the teaching data.

[0115] (Movement Evaluation Unit) The movement evaluation unit 303 evaluates the quality of each movement taught to the robot for each teaching means. For example, the movement evaluation unit 303 generates an evaluation value that evaluates the quality of each movement. The movement evaluation unit 303 also outputs the evaluation value of the quality of each movement to the user 21, the data storage unit 201A, the movement processing designation unit 301A, and the teaching movement processing unit 302A.

[0116] The action evaluation unit 303 performs the above-mentioned evaluation based on evaluation criteria such as whether the task was successful, the time it took to complete the task, the difference from an intermediate point, the magnitude of the force applied to the object, the amount of deformation of the object, the speed, the magnitude of acceleration, the gripping stability, evaluation by the user 21, and sensor values.

[0117] When the evaluation criterion is whether or not the task is successful, for example, if the task is to grasp and place an object around the robot, the action evaluation unit 303 evaluates the task based on whether the object was placed at the target position and whether the object was crushed or slipped during the task. The action evaluation unit 303 may automatically determine whether or not the object was placed at the target position based on, for example, an image of the robot's action, or the like, or the determination may be made by a human. When the determination is made by a human, the action evaluation unit 303 generates an evaluation value based on the human's determination result.

[0118] When the operation evaluation unit 303 uses the task completion time (the time it takes to complete the task) as the evaluation criterion, for example, if the robot's movements are required to be quick, the shorter the task completion time, the higher the evaluation value generated.

[0119] When the evaluation criterion is the difference from the intermediate point, the action evaluation unit 303 performs the evaluation based on, for example, the magnitude of the difference between a predetermined intermediate point that the robot should pass and a point that the robot actually passed. The smaller the difference, the higher the evaluation value generated by the action evaluation unit 303. The intermediate point may be, for example, position information, a value that a sensor should take, or image information, as long as it represents information equivalent to the intermediate point.

[0120] When the magnitude of the force applied to an object or the amount of deformation of the object is used as the evaluation criterion, for example, if the object needs to be held gently with the minimum necessary force, the smaller the force applied to the object or the amount of deformation of the object, the higher the evaluation value generated by the action evaluation unit 303. The action evaluation unit 303 can also obtain force information related to the force applied to the object directly from a sensor attached to the object, or estimate it from sensor information such as an image.

[0121] When the speed or acceleration of the robot is used as the evaluation criterion, the motion evaluation unit 303 performs evaluation based on, for example, the speed or acceleration of the end effector or each joint of the robot. For example, when smooth motion of the robot is required, the motion evaluation unit 303 generates a higher evaluation value the smaller the acceleration of the robot.

[0122] When the action evaluation unit 303 uses grasping stability as the evaluation criterion, it evaluates the degree of grasping stability (stability index) based on, for example, the size of the area of ​​the object that the robot comes into contact with (contact area) or the amount of slippage of the object relative to the robot.

[0123] Generally, the larger the contact area, the more stably the robot can hold an object. Therefore, when the contact area is used as the stability index, the larger the contact area, the higher the evaluation value generated by the action evaluation unit 303. Furthermore, when a tactile sensor is used as the sensor for teaching data, the smaller the amount of slippage of the object measured as the stability index, the higher the evaluation value generated by the action evaluation unit 303.

[0124] When the action evaluation unit 303 uses the evaluation by the user 21 as the evaluation criterion, it generates an evaluation value based on the evaluation result in which the user 21 directly observes and evaluates the robot's actions. The action evaluation unit 303 may output the evaluation value input by the user 21 as the evaluation value as is, or may generate an evaluation value based on the evaluation result input by the user 21 using characters or the like. Furthermore, when the evaluation value is input by the user 21 to the action evaluation unit 303, the evaluation value may be a continuous value or a discrete value.

[0125] When the action evaluation unit 303 uses the sensor value as the evaluation standard, it may generate an evaluation value based on the sensor value itself, or based on a value obtained by processing the sensor value. For example, the action evaluation unit 303 generates an evaluation value based on a value obtained by processing the sensor value with a function or the like by the data acquisition unit 1*2. Specifically, the action evaluation unit 303 outputs a value output from a neural network in response to the input of the sensor value as the evaluation value.

[0126] The action evaluation unit 303 can also extract low-evaluation and high-evaluation parts from each evaluated action. For example, the action evaluation unit 303 extracts low-evaluation parts based on a threshold, an average value, a median value, a variance, and teaching data from an expert.

[0127] The action evaluation unit 303 extracts, as low-rated parts, parts below a threshold value, for example, a threshold value set in advance by the user 21 or a threshold value extracted by the action evaluation unit 303 itself based on information about actions with low ratings specified by the user 21.

[0128] When extracting low-evaluation parts based on the average and median, the action evaluation unit 303 extracts, for example, parts below the average or median of the evaluation value of the entire action taught for each teaching means as low-evaluation parts.

[0129] When extracting low-evaluation portions based on variance, for example, if the same motion has been taught to the robot multiple times, the action evaluation unit 303 extracts portions of the variance of all teaching data whose variance is greater than or equal to a predetermined value as low-evaluation portions. The predetermined value is not particularly limited as long as it is a value that is generally recognized as large, such as greater than or equal to the average or median of the variance of all teaching data.

[0130] When extracting low-evaluation portions based on the expert's teaching data, the action evaluation unit 303 extracts, for example, portions that deviate from the correct answer data by a predetermined degree or more as low-evaluation portions, assuming that the expert's teaching data is correct. The predetermined degree is not particularly limited as long as it is a value that is generally recognized as large, such as a value equal to or greater than the average or median of the deviations of all teaching data.

[0131] The action evaluation unit 303 extracts highly evaluated parts based on the threshold, average, median, variance, and expert teaching data. For example, the action evaluation unit 303 extracts, as highly evaluated parts, parts that are higher than the threshold, average, or median, parts where the variance of all teaching data is less than a predetermined value, or parts where the deviation from the expert teaching data is less than a predetermined degree.

[0132] (Movement Processing Designation Unit) The movement processing designation unit 301A designates a processing means for a plurality of pieces of teaching data and a teaching start timing, further based on the quality of each movement evaluated by the movement evaluation unit 303. For example, the movement processing designation unit 301A designates the teaching start timing based on a timestamp designated by the user 21 based on the evaluation value, an instruction of a movement stop signal, or a sensor event in which the sensor value changes by more than a predetermined value.

[0133] The operation processing designation unit 301A can also automatically designate the processing means and the teaching start timing by itself based on the evaluation value, without relying on an instruction from the user 21.

[0134] (Teaching Motion Processing Unit) The teaching motion processing unit 302A generates teaching motion data further based on the quality of each motion evaluated by the motion evaluation unit 303. For example, the teaching motion processing unit 302A generates teaching motion data so that teaching of a motion is started for each teaching means at the teaching start timing specified by the motion processing specification unit 301A based on the evaluated quality of each motion.

[0135] An example of generation of teaching motion data based on the evaluated quality of each motion will be described below with reference to Fig. 12. Fig. 12 is a diagram for explaining an example of generation of teaching motion data based on the evaluated quality of each motion.

[0136] 12 , after the first data acquisition unit 112 acquires first teaching data 600, the action evaluation unit 303 generates a first evaluation value 6000, which is an evaluation value of the quality of the action taught by the first teaching means, and extracts a low-evaluation portion 6001. In FIG. 12 , the first evaluation value 6000 is expressed as time-series data of the evaluation value r(t).

[0137] Next, the second data acquisition unit 122 acquires the second teaching data 700. The quality of the movement in the second teaching data 700 corresponding to the movement in the low-evaluated portion 6001 is evaluated by the movement evaluation unit 303 as being higher than the evaluation value of the movement in the low-evaluated portion 6001.

[0138] In this case, the teaching operation processing unit 302A automatically overwrites the low-evaluated portion 6001 of the operations taught based on the first teaching data 600 with the operations taught based on the second teaching data 700.

[0139] In the above example, the movement processing designation unit 301A overwrites the movement of the low-evaluated part 6001. However, if the user 21 inputs a movement to be overwritten to the teaching movement processing unit 302A based on the evaluation value fed back from the movement evaluation unit 303, the movement processing designation unit 301A can overwrite the movement designated by the user 21 via the teaching movement processing unit 302A.

[0140] (2-2. Flow of information processing according to the second embodiment) Next, the flow of information processing according to the second embodiment will be described with reference to Fig. 13. Fig. 13 is a flowchart showing the flow of information processing according to the second embodiment. Steps S31, S32, S37 to S39, S43, and S44 are similar to steps S11, S12, S15 to S17, S21, and S22.

[0141] In step S33, the action evaluation unit 303 evaluates the quality of the action taught to the robot by the first teaching means. In step S34, the data storage unit 201A stores the first teaching data and an evaluation value of the quality of the action taught by the first teaching means. In step S35, in addition to the processing of step S14, the action evaluation unit 303 processes the evaluation results. In step S36, the action evaluation unit 303 extracts parts with low and high evaluations.

[0142] In step S40, the action evaluation unit 303 evaluates the quality of the action taught to the robot by the Nth teaching means. In step S41, the data storage unit 201A stores the Nth teaching data and an evaluation value of the quality of the action taught by the Nth teaching means. In step S42, in addition to the processes of steps S19 and S20, the action evaluation unit 303 processes the evaluation result.

[0143] (2-3. Modification) The teaching motion processing unit 302A can also generate teaching motion data in which, among the motions taught for each teaching means, motions by the teaching means with the highest evaluation value for the same motion are combined as the teaching motion data. An example of generating teaching motion data in which, among the motions taught for each teaching means, motions by the teaching means with the highest evaluation value for the same motion are combined will be described below with reference to FIG. 14. FIG. 14 is a diagram for describing an example of generating teaching motion data in which, among the motions, motions by the teaching means with the highest evaluation value for the same motion are combined.

[0144] The movement evaluation unit 303 generates a first evaluation value 6000 regarding the quality of the movement taught by the first teaching means, a second evaluation value 7000 regarding the quality of the movement taught by the second teaching means, and a third evaluation value 8000 regarding the quality of the movement taught by the third teaching means.

[0145] In this case, the teaching motion processing unit 302A generates teaching motion data that combines the following motions: The motion of the portion 6002 of the same motion with the highest first evaluation value The motion of the portion 6003 of the same motion with the highest first evaluation value The motion of the portion 7001 of the same motion with the highest second evaluation value The motion of the portion 8001 of the same motion with the highest third evaluation value

[0146] Next, the flow of information processing according to the modified example of the second embodiment will be described with reference to Fig. 15. Fig. 15 is a flowchart showing the flow of information processing according to the modified example of the second embodiment. Steps S51 to S54 are similar to steps S38 to S41.

[0147] In step S55, if the motion teaching unit 1*1 uses another teaching means (step S55; Yes), the motion teaching unit 1*1 returns to step S51. For example, if the motion teaching unit 1*1 has generated up to the second teaching data for teaching the robot a motion using the second teaching means, in step S51, the motion teaching unit 1*1 generates third teaching data for teaching the robot a motion using the third teaching means. In step S55, if the motion teaching unit 1*1 uses another teaching means (step S55; No), the motion teaching unit 1*1 proceeds to step S56.

[0148] In step S56, the teaching operation processing unit 302A compares the evaluation values ​​of the same operation among the operations taught by each teaching means. For example, when N is 3, the teaching operation processing unit 302A compares a first evaluation value for the operation taught to the robot by the first teaching means, a second evaluation value for the operation taught to the robot by the second teaching means, and a third evaluation value for the operation taught to the robot by the third teaching means.

[0149] In step S57, the teaching operation processing unit 302A determines the teaching means. For example, the teaching operation processing unit 302A determines the teaching means with the highest evaluation for the same operation among the operations taught by each teaching means as the teaching means for each of the multiple operations constituting the series of operations to be taught to the robot.

[0150] Next, the teaching motion processing unit 302A generates teaching motion data that combines, as the teaching motion data, motions by the teaching means having the highest evaluation values ​​for the same motion among the motions taught for each teaching means. For example, the teaching motion processing unit 302A generates teaching motion data that combines the motion with the highest evaluation value for the same motion among the first evaluation values, the motion with the highest evaluation value for the same motion among the second evaluation values, and the motion with the highest evaluation value for the same motion among the third evaluation values.

[0151] (3. Third Embodiment) (3-1. Configuration of Information Processing System According to Third Embodiment) The information processing device 10A can also predict the optimal teaching means as the teaching means for each of a plurality of movements that make up a series of movements to be taught to the robot. The configuration of an information processing system 1B according to the third embodiment will be described below with reference to FIGS. 16 and 17. FIG. 16 is a block diagram (1) showing the configuration of an information processing system according to the third embodiment. FIG. 17 is a block diagram (2) showing the configuration of an information processing system according to the third embodiment.

[0152] 16 and 17, information processing system 1B includes information processing device 10B instead of information processing device 10A in information processing system 1A. Fig. 16 shows blocks of information processing device 10B used when acquiring data. Fig. 17 shows blocks of information processing device 10B used when predicting the optimal teaching means.

[0153] (Information Processing Device) Compared to the information processing device 10A, the information processing device 10B further includes a task information processing unit 304, an action data input unit 101, an action data comparison unit 202, and an optimal teaching means prediction unit 305. The information processing device 10B also includes a data storage unit 201B instead of the data storage unit 201A. The optimal teaching means prediction unit 305 also functions as a control unit.

[0154] First, the task information processing unit 304 and the data storage unit 201B, which are blocks used when acquiring data, will be described with reference to Fig. 16. Next, the operation data input unit 101, the operation data comparison unit 202, and the optimum teaching means prediction unit 305, which are blocks used for the optimum teaching means prediction value, will be described with reference to Fig. 17.

[0155] 16 is a block that generates task information related to a task. For example, the task information processing unit 304 generates, as task information, a language string that represents a task linked to time-series data that represents the trajectory of a series of actions that make up the task or the state of the robot, and some or all of information related to the goal of the task.

[0156] The task information processing unit 304 acquires, as a language string representing a task, for example, "move, pick and place" or "pick up a block, push a block, etc.", which are character strings representing the actions of the instruction data expressed as time-series data. The task information processing unit 304 can also acquire time-series data itself, such as instruction data, or information obtained by converting time-series data using some function or means. The task information processing unit 304 also acquires, as information regarding the goal of the task, for example, information regarding the final goal of the task and information regarding the progress of the task (such as an image that is the target of the task).

[0157] The data storage unit 201B stores task information in addition to the plurality of pieces of instruction data and the evaluation values. For example, the data storage unit 201B stores a table in which the instruction data represented as time-series data, the evaluation values ​​of the quality of actions based on the instruction data, and character strings representing the actions of the instruction data are linked to each other.

[0158] 17 is a block that receives input of operation data related to operation. The operation data input unit 101 may receive input of time-series data of a series of operations that constitute a robot task, or may receive input of information in which the time-series data of the series of operations is processed by some kind of function or the like.

[0159] The action data input unit 101 may receive, as action data, the instruction data itself generated by the action instruction unit 1*1 or information obtained by processing the instruction data by the data acquisition unit 1*2.

[0160] In addition, the operation data input unit 101 may accept input of, as operation data, videos or still images of a living being with a physicality similar to that of the robot, such as a human or an animal, performing the operation required of the robot, or data that has been processed from these.

[0161] (Movement Data Comparison Unit) The movement data comparison unit 202 compares the movement data output from the movement data input unit 101 with the teaching data stored in the data storage unit 201B. The movement data comparison unit 202 also calculates the similarity between these pieces of data. For example, the movement data comparison unit 202 calculates the similarity using a general pattern matching algorithm such as DTW (Dynamic Time Warping).

[0162] (Optimal Teaching Means Prediction Unit) The optimal teaching means prediction unit 305 predicts the optimal teaching means as a teaching means for each of a plurality of movements that constitute a series of movements of the robot.

[0163] For example, the optimal teaching means prediction unit 305 predicts the optimal teaching means based on the similarity between the operation data output from the operation data input unit 101 and the teaching data itself or data that has been processed from the teaching data stored in the data storage unit 201B.

[0164] The optimal teaching means prediction unit 305 can also use a neural network for class classification when comparing similarities and determining whether each of the multiple actions that make up a series of actions matches each action taught by each teaching means.

[0165] An example of prediction of an optimal teaching means will be described below with reference to Fig. 18. Fig. 18 is a diagram for explaining an example of prediction of an optimal teaching means. In Fig. 18, data 2000, which is processing of teaching data, is stored in a table format in a data storage unit 201B.

[0166] The optimum teaching means prediction unit 305 compares the similarity between the data 2000 obtained by processing the teaching data stored in the data storage unit 201B and the action data 2100 output from the action data input unit 101.

[0167] In other words, the optimal teaching means prediction unit 305 compares the similarity between each of the actions of steps S61 to S63 taught for each teaching means and each of the multiple actions of steps S71 to S73 that constitute the series of actions output from the action data input unit 101.

[0168] For example, the optimal teaching means prediction unit 305 predicts that the action most similar to the action of step S71 in the action data 2100 is the action of grasping an object in step S62 in the data 2000 obtained by processing the teaching data.

[0169] The optimum teaching means prediction unit 305 predicts that the action having the highest similarity to the action of step S72 in the action data 2100 is the action of placing the object in step S63 in the data 2000 obtained by processing the teaching data.

[0170] In addition, the optimal teaching means prediction unit 305 predicts that the action having the highest similarity to the action of step S73 in the action data 2100 is the action of moving to a specified point in step S61 in the data 2000 in which the teaching data has been processed.

[0171] Next, the optimal teaching means prediction unit 305 predicts the optimal teaching means as the teaching means for the motion with the highest similarity. In this case, the optimal teaching means prediction unit 305 compares the evaluation values ​​between the teaching means, and predicts that the teaching means with the highest evaluation value for the quality of the motion, among the teaching means for the motion with the highest similarity, is the optimal teaching means for each motion in steps S71 to S73.

[0172] For example, the optimal teaching means prediction unit 305 predicts that the program 500 with the highest evaluation value for the quality of the action among the teaching means for the action of grasping an object in step S62 is the optimal teaching means for the action in step S71.

[0173] The optimal teaching means prediction unit 305 predicts that the exoskeleton wearable device 400 with the highest evaluation value for the quality of the action among the teaching means for the action of placing an object in step S63 is the optimal teaching means for the action in step S72.

[0174] The optimal teaching means prediction unit 305 predicts that the VR system 300 with the highest evaluation value for the quality of the movement among the teaching means for the movement to the specified point in step S61 is the optimal teaching means for the movement in step S73.

[0175] Returning to the description of FIG. 17 , the optimal teaching means prediction unit 305 can also predict the optimal instructor for each of a plurality of movements constituting a series of movements, based on the proficiency of the instructors for each movement. For example, if the instructors for movement are a first instructor 11 who is a beginner, a second instructor 12 who is an expert, and an Nth instructor 1N who is an expert, the optimal teaching means prediction unit 305 predicts that the second instructor 12 or the Nth instructor 1N is the optimal instructor for movement.

[0176] The optimal teaching means prediction unit 305 can also present to the user 21 the optimal teaching means, the divisions of each of the multiple actions that make up the series of actions taught by the optimal teaching means, and some or all of the names of each of the multiple actions (estimated task names).

[0177] Hereinafter, the presentation of the optimum teaching means, the division of each of the plurality of motions, and the name of each of the plurality of motions will be described with reference to Fig. 19. Fig. 19 is a diagram for explaining the presentation of the optimum teaching means, etc.

[0178] In step S81, the optimal teaching means prediction unit 305 presents data 3000 including the teaching means optimal for the already-taught operation in step S81 to the user 21. In this case, the optimal teaching means prediction unit 305 presents the data 3000 to the user 21, for example, by transmitting the data 3000 to a terminal of the user 21 and then displaying the data 3000 on the terminal.

[0179] In step S82, the optimal teaching means prediction unit 305 presents data 3100 including the optimal teaching means for each of the movements that have been taught up to step S82 to the user 21. In step S83, the optimal teaching means prediction unit 305 presents data 3200 including the optimal teaching means for each of the movements that have been taught up to step S83 to the user 21.

[0180] After the instruction is completed, the optimal instruction means prediction unit 305 presents the user 21 with data 3300 including the optimal instruction means for each of the multiple movements that make up the series of movements shown in the instruction movement data, the division of each of the multiple movements, and the name of each of the multiple movements.

[0181] For example, the optimal teaching means prediction unit 305 presents the name of the action in step S81, "pick," and the optimal program for that action to the user 21. The optimal teaching means prediction unit 305 presents the name of the action in step S82, "place," and the optimal exoskeleton-wearable device for that action to the user 21. Furthermore, the optimal teaching means prediction unit 305 presents the name of the action in step S83, "move," and the optimal VR system for that action to the user 21.

[0182] Furthermore, the optimal teaching means prediction unit 305 presents the divisions between the actions of step S81, step S82, and step S83 to the user 21 by color-coding them in a distinguishable manner, as in data 3300. The optimal teaching means prediction unit 305 can also present the user 21 with data as data 3300 that does not include the names of the respective actions described above.

[0183] (3-2. Information processing flow according to the third embodiment) Next, the information processing flow according to the third embodiment will be described using Fig. 20 and Fig. 21. First, the information processing flow when acquiring data according to the third embodiment will be described using Fig. 20. Fig. 20 is a flowchart showing the information processing flow when acquiring data according to the third embodiment. Steps S91 to S93 and S96 are similar to steps S51 to S53 and S56.

[0184] In step S94, the task information processing unit 304 generates task information. In step S95, the data storage unit 201B stores the task information in addition to the plurality of pieces of teaching data and the evaluation values.

[0185] Next, the flow of information processing when predicting the optimal teaching means according to the third embodiment will be described with reference to Fig. 21. Fig. 21 is a flowchart showing the flow of information processing when predicting the optimal teaching means according to the third embodiment.

[0186] In step S101, the operation teaching unit 1*1 generates teaching data. The data acquisition unit 1*2 acquires multiple pieces of teaching data generated by the operation teaching unit 1*1 for each teaching means. The data storage unit 201B stores the multiple pieces of teaching data acquired by the data acquisition unit 1*2.

[0187] In step S102, the optimal teaching means prediction unit 305 calculates the degree of matching (similarity) between the plurality of teaching data stored in the data storage unit 201B and the data set that is the action data output from the action data input unit 101. In this way, the optimal teaching means prediction unit 305 compares the similarity between each action taught for each teaching means and each of the plurality of actions that make up the series of actions output from the action data input unit 101.

[0188] In step S103, the optimal teaching means prediction unit 305 predicts the optimal teaching means for the motion that has the highest similarity to each of the multiple motions that make up the sequence of motions. In this case, the optimal teaching means prediction unit 305 determines the teaching means that maximizes the evaluation of the quality of the motion from among the teaching means for the motion with the highest similarity. In step S104, the optimal teaching means prediction unit 305 presents the teaching means determined in step S103 to the user 21.

[0189] (4. Fourth Embodiment) The information processing device 10A can also present feedback information, which is information according to the evaluated quality of each action, to the user 21. Hereinafter, the configuration of an information processing system 1C according to the fourth embodiment will be described with reference to FIG. 22. FIG. 22 is a block diagram showing the configuration of the information processing system according to the fourth embodiment. The information processing system 1C includes an information processing device 10C instead of the information processing device 10A.

[0190] (Information Processing Apparatus) The information processing apparatus 10C further includes, in addition to the information processing apparatus 10A, a feedback information control unit 306. The feedback information control unit 306 also functions as a control unit.

[0191] (Feedback Information Control Unit) The feedback information control unit 306 is a block that changes, switches, or selects the method of providing feedback to the user 21 based on the evaluation results of each motion taught by each teaching means. The feedback information control unit 306 also presents feedback information, which is information according to the evaluated quality of each motion, to the user 21.

[0192] The feedback information may be, for example, feedback in the form of sound, light, or vibration according to the distance measured by a distance sensor, tactile feedback measured by a tactile sensor, force feedback measured by a force sensor, or verbal feedback.

[0193] Also, changing, switching or selecting the feedback method means changing the control gain of the teaching means, such as the control gain of force feedback, the control gain of impedance, or the control gain of admittance.

[0194] The rules for changing, switching, and selecting feedback information may be set by a human or may be learned by means of reinforcement learning or evolutionary computation. For example, one method for learning the rules is to initially switch feedback information according to random rules and give a high reward to switching means that are highly evaluated for the quality of their actions.

[0195] The following describes an example of presentation of feedback information to the user 21 by the feedback information control unit 306. For example, when the robot's task is to grasp an object, if the gripping force is low and the object tends to slip, the feedback information control unit 306 lowers the control gain of the force feedback, which is one of the control gains of the teaching means, and then urges the user 21 to teach a large force.

[0196] If the robot's task is to control a general manipulator, and if the robot's position or accuracy is poor and it is likely to collide with an object in the surrounding environment, the feedback information control unit 306 plays a sound corresponding to the distance measured by the distance sensor to the user 21. In this case, the feedback information control unit 306 transmits feedback information related to the sound to the terminal of the user 21, and then causes the terminal to output the sound.

[0197] When the robot is performing a task involving contact with an object or the like in the surrounding environment, the feedback information control unit 306 provides feedback to the user 21 by vibration according to the distance measured by the distance measuring sensor in order to provide clearer feedback of the contact. In this case, the feedback information control unit 306 transmits feedback information related to the vibration to the terminal of the user 21 and then vibrates the terminal.

[0198] Furthermore, the feedback information control unit 306 displays information according to the evaluation value of each action taught for each teaching means, and plays a sound according to the evaluation value to the user 21. In this case, the feedback information control unit 306 transmits feedback information related to the sound to the terminal of the user 21, and then causes the terminal to display the feedback information or output the sound.

[0199] (5. Other Embodiments) The processes according to the embodiments can be implemented in various different forms other than the above-described embodiments.

[0200] Of the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. The various information shown in each drawing is not limited to the information shown in the drawings.

[0201] The components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of the configuration can be functionally or physically distributed or integrated in any unit depending on various loads and usage conditions.

[0202] The above-described embodiments and modifications can be combined as appropriate within the scope of not causing any contradiction in the processing content.

[0203] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0204] (6. Effects of the information processing device according to the present disclosure) As described above, the information processing device according to the present disclosure (information processing device 10 in the embodiment) includes a data acquisition unit (data acquisition unit 1*2 in the embodiment) and a teaching operation processing unit (teaching operation processing unit 302 in the embodiment).

[0205] The data acquisition unit acquires a plurality of pieces of teaching data for each teaching means to teach the robot an action. The teaching action processing unit generates teaching action data for teaching the robot a series of actions that combine some or all of the actions taught by each teaching means based on the plurality of pieces of teaching data acquired.

[0206] In this way, the information processing device can easily combine actions taught by various teaching means, and can easily teach a robot actions using teaching means suitable for each of the multiple actions that make up a series of actions. This allows the information processing device to improve the quality of teaching to the robot, enabling the robot to perform more accurate and dexterous actions.

[0207] Furthermore, since the information processing device can teach some of the operations using different teaching means, it is possible to simplify the information processing device, reduce costs, and select teaching means according to the required quality of teaching.

[0208] The teaching motion processing unit generates teaching motion data that combines the motion of the robot moving to a predetermined point, which is taught by the VR device, and the motion of the robot grasping an object, which is taught by an exoskeleton-type device that is attached to part of the instructor's body.

[0209] The VR device allows the instructor to intuitively control the robot with realistic movements, making it suitable for robots that do not require complex operations to move to a specific location. The exoskeleton-type device allows the instructor's body movements, such as the instructor's fingers, to be reflected in the robot's movements, making it suitable for robots that require complex operations with their multi-fingered hands to grasp an object.

[0210] Therefore, the information processing device can further improve the quality of teaching to the robot by generating teaching motion data that combines the motion of the robot being taught by the VR device to move to a specified location and the motion of the robot being taught by the exoskeleton-wearable device to grasp an object.

[0211] If some or all of the series of movements have already been taught to the robot, the teaching movement processing unit generates teaching movement data for teaching some or all of the series of movements by another teaching means as teaching movement data.

[0212] The information processing device can overwrite part or all of a series of movements that have already been taught by generating teaching movement data that teaches part or all of the series of movements using a different teaching means. This reduces the risk that a task will fail due to a movement that has only been taught partway, requiring the entire task to be redone. Furthermore, the information processing device can generate teaching movement data that partially includes new teaching data by reusing teaching data previously acquired.

[0213] Furthermore, the information processing device can use a more accurate teaching means to teach a robot an action that was previously taught to it using a simple teaching means with low accuracy. Therefore, the information processing device can improve the quality of teaching as much as necessary by gradually improving the accuracy of teaching.

[0214] When a teaching start timing, which is the time when teaching of an action starts for each teaching means, is set in advance, the teaching action processing unit generates teaching action data so that teaching of some or all of the multiple actions that make up a series of actions starts at the preset teaching start timing.

[0215] This allows the information processing device to start teaching some or all of the multiple movements that make up a series of movements at the timing when teaching starts, so that teaching the robot movements can be started at a time desired by the user.

[0216] The teaching operation processing unit generates teaching operation data so that the timing to start teaching is some or all of the following: the timing specified by the user, the timing when the similarity between multiple teaching data is highest, and the timing when a change in the sensor value is detected.

[0217] This allows the information processing device to start teaching the robot an action using a different teaching means at an appropriate time, such as when the user desires, when the actions taught by each teaching means are approximately continuous, or when the robot's action changes.

[0218] The teaching motion processing unit generates teaching motion data in which some or all of the motions are combined so that some or all of the motions are substantially continuous.

[0219] This allows the information processing device to continuously connect teaching data that teaches the robot actions using different teaching means. Therefore, the information processing device can introduce teaching data from a new teaching means into the teaching action data without making detailed adjustments to the robot's initial state, etc., to ensure continuity with other teaching data. Furthermore, the quality of the teaching action data improves, reducing the cost of the robot's subsequent skill acquisition, allowing the information processing device to improve the robot's performance.

[0220] The teaching motion processing unit generates teaching motion data in which some or all of the motions are combined so that the weighting functions associated with each motion are approximately continuous, or the motions are approximately continuous in physical space, or the actual motions of each motion are approximately continuous.

[0221] This allows the information processing device to continuously connect teaching data taught by different teaching means in a functional, spatial, or actual motion manner, thereby improving the quality of teaching to the robot and generating teaching motion data with guaranteed continuity by a means desired by the user.

[0222] The teaching operation processing unit uses at least one of a basis function and a function representing a neural network as the weighting function, which enables the information processing device to control the robot's operation more precisely, thereby further improving the quality of teaching the robot's operation.

[0223] The teaching motion processing unit uses a physical model when generating the teaching motion data based on continuity in physical space, which allows the information processing device to smoothly connect, for example, the final state of a given teaching data with the initial state of the next teaching data using virtual springs and dampers, thereby further improving the quality of teaching the robot's motion.

[0224] When generating the teaching motion data based on the continuity of actual motion, the teaching motion processing unit generates the teaching motion data so that the timing specified by the user becomes the teaching start timing. This allows the information processing device to start teaching the robot a motion using the next teaching means at a timing desired by the user and which ensures the continuity of actual motion, thereby further improving the quality of teaching the robot's motion.

[0225] The information processing device further includes a motion evaluation unit (motion evaluation unit 303 in this embodiment) that evaluates the quality of each motion, and the teaching motion processing unit generates teaching motion data further based on the evaluated quality of each motion.

[0226] This allows the information processing device to evaluate the quality of each motion taught based on the teaching data, and therefore, by informing the user of the motions with high and low teaching quality, the information processing device can encourage the user to select the most appropriate teaching means. Furthermore, since the information processing device can quantitatively compare the evaluation of the quality of each motion taught based on the teaching data, the information processing device can determine the most appropriate teaching means.

[0227] The teaching motion processing unit generates teaching motion data that combines the motions of the same motions, among the respective motions, performed by the teaching means with the highest evaluation value. This allows the information processing device to automatically determine the optimal teaching means based on the evaluation value of the quality of each motion, thereby improving the quality of the teaching and reducing the time and effort required by the user to select the teaching means.

[0228] The information processing device further includes a motion evaluation unit that evaluates the quality of each motion, and the teaching motion processing unit generates teaching motion data so that teaching of the motion for each teaching means is started at a teaching start timing designated based on the evaluated quality of each motion. This allows the information processing device to start teaching of a highly evaluated motion at an appropriate teaching start timing, thereby further improving the quality of teaching.

[0229] The information processing device further includes an optimal teaching means prediction unit (optimal teaching means prediction unit 305 in this embodiment) that predicts the optimal teaching means as a teaching means for a movement that has the highest similarity to each of the multiple movements that make up a series of movements, among the movements. This allows the information processing device to prompt the user to select the optimal teaching means as the movement that is most similar to the movements that make up the task required of the robot, thereby further improving the quality of teaching.

[0230] The optimal teaching means prediction unit predicts the teaching means with the highest evaluation value for the quality of the motion as the optimal teaching means, among the teaching means for the motion with the highest similarity. As a result, when there are multiple teaching means for the motion with the highest similarity, the information processing device can prompt the user to select the teaching means with the highest evaluation value for the quality of the motion, thereby further improving the quality of the teaching.

[0231] The optimal teaching means prediction unit further predicts an optimal instructor for each of the multiple movements that make up the series of movements, based on the instructor's proficiency level for each movement. This allows the information processing device to prompt the user to select the optimal teaching means based on the instructor's proficiency level evaluation, thereby further improving the quality of teaching.

[0232] The optimal teaching means prediction unit presents to the user the optimal teaching means, the divisions of each of a plurality of movements constituting a series of movements to be taught by the optimal teaching means, and some or all of the names of each of the plurality of movements. This allows the information processing device to present to the user the optimal teaching means and the optimal teaching start timing for each of the plurality of movements, thereby urging the user to select the optimal teaching means, etc.

[0233] The information processing device further includes a feedback information control unit that presents feedback information, which is information according to the evaluated quality of each action, to the user.

[0234] This allows the information processing device to provide feedback on the quality of instruction to the user or instructor. Furthermore, the information processing device can efficiently provide feedback to the user or instructor with the minimum amount of information necessary by selecting or switching the information to be fed back to the user or instructor according to the evaluated quality of each action. This allows the information processing device to improve the quality of instruction and enhance the instructor's teaching skills.

[0235] (7. Hardware Configuration) Information devices such as the information processing device 10 according to the above-described embodiments are realized by a computer 1000 configured as shown in FIG. 23 . FIG. 23 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device according to the present disclosure. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. The components of the computer 1000 are connected by a bus 1050.

[0236] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. The CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to the various programs.

[0237] The ROM 1300 stores boot programs such as a Basic Input Output System (BIOS) that is executed by the CPU 1100 when the computer 1000 starts up, and programs that depend on the hardware of the computer 1000 .

[0238] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records an information processing program according to the present disclosure, which is an example of program data 1450.

[0239] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (such as the Internet). The CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0240] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. The CPU 1100 receives data from input devices such as a keyboard and a mouse via the input / output interface 1600. The CPU 1100 transmits data to output devices such as a display and a speaker via the input / output interface 1600. The input / output interface 1600 can also function as a media interface for reading a program recorded on a predetermined recording medium.

[0241] The media may be optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase Change Rewritable Discs), magneto-optical recording media such as MOs (Magneto-Optical disks), tape media, magnetic recording media, or semiconductor memories.

[0242] When the computer 1000 functions as the information processing device 10 according to the embodiment, the CPU 1100 of the computer 1000 executes an information processing program loaded onto the RAM 1200 to realize the functions of control units such as the motion teaching unit 1*1, data acquisition unit 1*2, data storage unit 201, motion processing designation unit 301, teaching motion processing unit 302, motion command value generation unit 401, and actuation unit 402 shown in Fig. 3. The information processing program according to the present disclosure and data in the storage device are stored in the HDD 1400.

[0243] The CPU 1100 reads and executes the program data 1450 from the HDD 1400. However, as another example, the CPU 1100 can also obtain these programs from other devices via an external network 1550.

[0244] (8. Supplementary Information) The present technology can also be configured as follows. (1) An information processing device comprising: a data acquisition unit that acquires, for each teaching means, a plurality of pieces of teaching data for teaching a robot an action; and a teaching action processing unit that generates, based on the plurality of acquired teaching data, teaching action data for teaching the robot a series of actions that combine some or all of the actions taught for each teaching means. (2) The information processing device described in (1), wherein the teaching action processing unit generates, as the teaching action data, teaching action data that combines an action of the robot moving to a predetermined point that is taught by a VR device and an action of the robot grasping an object that is taught by an exoskeleton-mounted device that is an exoskeleton-type device worn on a part of the body of an instructor. (3) The information processing device described in (1) or (2), wherein, when some or all of the series of actions are actions that have already been taught to the robot, the teaching action processing unit generates, as the teaching action data, teaching action data for teaching some or all of the series of actions by another teaching means. (4) The information processing device according to any one of (1) to (3), wherein, when a teaching start timing, which is a time when teaching of an action is started for each of the teaching means, is set in advance, the teaching action processing unit generates the teaching action data so that teaching of some or all of the multiple actions constituting the series of actions is started at the preset teaching start timing. (5) The information processing device according to (4), wherein the teaching action processing unit generates the teaching action data so that the teaching start timing is some or all of a time specified by a user, a time when the multiple pieces of teaching data have the highest similarity, and a time when a change in a sensor value is detected. (6) The information processing device according to (5), wherein the teaching action processing unit generates, as the teaching action data, teaching action data in which some or all of the actions are combined so that some or all of the actions are substantially continuous.(7) The information processing device described in (6), wherein the teaching motion processing unit generates, as the teaching motion data, teaching motion data in which some or all of the motions are combined so that weighting functions associated with the motions are substantially continuous, or so that the motions are substantially continuous in physical space, or so that actual motions of the motions are substantially continuous. (8) The information processing device described in (7), wherein the teaching motion processing unit uses, as the weighting function, at least one of a basis function and a function representing a neural network. (9) The information processing device described in (7), wherein the teaching motion processing unit uses a physical model when generating the teaching motion data based on continuity in the physical space. (10) The information processing device described in (7), wherein the teaching motion processing unit generates the teaching motion data so that the teaching start timing is a time specified by a user when generating the teaching motion data based on continuity in the actual motion. (11) The information processing device according to any one of (1) to (10), further comprising a movement evaluation unit that evaluates the quality of each of the movements, wherein the teaching movement processing unit generates the teaching movement data further based on the evaluated quality of each of the movements. (12) The information processing device according to (11), wherein the teaching movement processing unit generates, as the teaching movement data, a combination of movements by teaching means having the highest evaluation value for the same movement among the movements. (13) The information processing device according to any one of (4) to (10), further comprising a movement evaluation unit that evaluates the quality of each of the movements, wherein the teaching movement processing unit generates the teaching movement data such that teaching of the movement is started for each teaching means at a teaching start timing specified based on the evaluated quality of each of the movements. (14) The information processing device according to any one of (11) to (13), further comprising an optimal teaching means prediction unit that predicts an optimal teaching means as a teaching means for a movement among the movements that has the highest similarity to each of a plurality of movements constituting the sequence of movements. (15) The information processing device according to (14), wherein the optimal teaching means prediction unit predicts, as the optimal teaching means, a teaching means having the highest evaluation value of the quality of the action among the teaching means of the action having the highest similarity.(16) The information processing device according to (14) or (15), wherein the optimal teaching means prediction unit further predicts an optimal instructor for each of the plurality of movements constituting the series of movements, based on the proficiency of the instructor for each of the movements. (17) The information processing device according to any one of (14) to (16), wherein the optimal teaching means prediction unit presents to a user the optimal teaching means, a division of each of the plurality of movements constituting the series of movements taught by the optimal teaching means, and some or all of the names of each of the plurality of movements. (18) The information processing device according to any one of (11) to (17), further comprising a feedback information control unit that presents to a user feedback information that is information according to the evaluated quality of each of the movements. (19) An information processing method, including: a computer acquiring a plurality of pieces of teaching data for teaching a robot a movement, for each teaching means; and generating, based on the acquired plurality of teaching data, teaching movement data for teaching the robot a series of movements that combines some or all of the movements taught for each teaching means. (20) An information processing program for causing a computer to function as an information processing device comprising: a data acquisition unit that acquires, for each teaching means, a plurality of pieces of teaching data for teaching a robot an action; and a teaching action processing unit that generates, based on the plurality of acquired teaching data, teaching action data for teaching the robot a series of actions that combine some or all of the actions taught for each teaching means.

[0245] REFERENCE SIGNS LIST 1 Information processing system 10 Information processing device 11 First instructor 12 Second instructor 1N Nth instructor 1*1 Motion teaching unit 111 First motion teaching unit 121 Second motion teaching unit 1N1 Nth motion teaching unit 1*2 Data acquisition unit 112 First data acquisition unit 122 Second data acquisition unit 1N2 Nth data acquisition unit 201 Data storage unit 301 Motion processing designation unit 302 Teaching motion processing unit 401 Motion command value generation unit 402 Actuation unit

Claims

1. An information processing device comprising: a data acquisition unit that acquires multiple pieces of teaching data for each teaching means to teach a robot an action; and a teaching action processing unit that generates teaching action data that teaches the robot a series of actions that combine some or all of the actions taught by each teaching means based on the multiple pieces of teaching data acquired.

2. The information processing device according to claim 1, wherein the teaching motion processing unit generates teaching motion data that combines a motion of the robot moving to a predetermined location, which is taught by a VR (Virtual Reality) device, and a motion of the robot grasping an object, which is taught by an exoskeleton-type device that is worn on part of the body of an instructor.

3. The information processing device according to claim 1, wherein, when some or all of the series of movements have already been taught to the robot, the teaching movement processing unit generates teaching movement data for teaching some or all of the series of movements by another teaching means as the teaching movement data.

4. The information processing device according to claim 1, wherein, when a teaching start timing, which is the time when teaching of an action is to begin for each teaching means, is preset, the teaching action processing unit generates the teaching action data so that teaching of some or all of the multiple actions that make up the series of actions begins at the preset teaching start timing.

5. The information processing device according to claim 4, wherein the teaching operation processing unit generates the teaching operation data so that the timing to start teaching is some or all of the following: a time specified by the user; a time when the similarity between the plurality of teaching data is highest; and a time when a change in the sensor value is detected.

6. The information processing device according to claim 5, wherein the teaching motion processing unit generates the teaching motion data by combining some or all of the respective motions so that some or all of the respective motions are substantially continuous.

7. The information processing device according to claim 6, wherein the teaching movement processing unit generates teaching movement data in which some or all of the movements are combined so that the weighting functions associated with the movements are approximately continuous, the movements are approximately continuous in physical space, or the actual movements of the movements are approximately continuous.

8. The information processing device according to claim 7, wherein the teaching operation processing unit uses at least one of a basis function and a function representing a neural network as the weighting function.

9. The information processing device according to claim 7, wherein the teaching motion processing unit uses a physical model when generating the teaching motion data based on continuity in the physical space.

10. The information processing device of claim 7, wherein when the teaching operation processing unit generates the teaching operation data based on the continuity of the actual operation, the teaching operation data is generated so that the timing specified by the user becomes the timing to start the teaching.

11. The information processing device according to claim 1, further comprising a movement evaluation unit that evaluates the quality of each movement, wherein the teaching movement processing unit generates the teaching movement data further based on the evaluated quality of each movement.

12. The information processing device according to claim 11, wherein the teaching operation processing unit generates the teaching operation data by combining, among the respective operations, operations performed by the teaching means with the highest evaluation value for the same operation.

13. An information processing device as described in claim 4, further comprising an action evaluation unit that evaluates the quality of each of the actions, and wherein the teaching action processing unit generates the teaching action data so that teaching of the action is started for each of the teaching means at a teaching start timing specified based on the evaluated quality of each of the actions.

14. The information processing device according to claim 11, further comprising an optimal teaching means prediction unit that predicts the optimal teaching means as a teaching means for a movement that has the highest similarity to each of the plurality of movements that make up the sequence of movements, among the movements.

15. The information processing device according to claim 14, wherein the optimal teaching means prediction unit predicts, as the optimal teaching means, the teaching means having the highest evaluation value for the quality of the action among the teaching means for the action with the highest similarity.

16. The information processing device according to claim 14, wherein the optimal teaching means prediction unit further predicts an optimal instructor for each of the plurality of movements constituting the series of movements based on the proficiency of the instructor for each of the movements.

17. The information processing device according to claim 14, wherein the optimal teaching means prediction unit presents to the user the optimal teaching means, the divisions of each of the multiple actions that make up the series of actions taught by the optimal teaching means, and some or all of the names of each of the multiple actions.

18. The information processing device according to claim 11, further comprising a feedback information control unit that presents feedback information, which is information corresponding to the evaluated quality of each action, to the user.

19. An information processing method including the steps of: a computer acquiring a plurality of pieces of teaching data for each teaching means to teach a robot an action; and based on the acquired plurality of pieces of teaching data, generating teaching action data for teaching the robot a series of actions that combine some or all of the actions taught by each teaching means.

20. An information processing program for causing a computer to function as an information processing device comprising: a data acquisition unit that acquires multiple pieces of teaching data for teaching a robot an action for each teaching means; and a teaching action processing unit that generates teaching action data for teaching the robot a series of actions that combine some or all of the actions taught for each teaching means based on the multiple pieces of teaching data acquired.

Images