Information processing device, information processing method, program, and trained model

Abstract

To provide an information processing device, an information processing method, a program, and a learned model that estimate an appropriate alternative tool.SOLUTION: An information processing device includes: an acquisition unit that, in a trial motion using a first tool by a robot, acquires information indicating a force generated between the first tool and an object; and an estimation unit that estimates, on the basis of the information acquired by the acquisition unit and information indicating a force generated between a specific tool and the object in a specific task using the specific tool, information for performing the specific task using the first tool.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, a program, and a trained model. [Background technology]

[0002] Patent Document 1 discloses a technique for suggesting an appropriate alternative tool that can provide the operation of a desired tool. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 4523501 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, for example, a tool that is different in shape from the desired tool may not be suggested as an alternative tool.

[0005] An object of the present disclosure is to provide a technique capable of estimating an appropriate alternative tool. [Means for solving the problem]

[0006] In a first aspect of the present disclosure, an information processing device is provided having an acquisition unit that acquires information indicating a force generated between a first tool and an object during a trial operation by a robot using the first tool, and an estimation unit that estimates information for performing a specific task using the first tool based on the information acquired by the acquisition unit and information indicating a force generated between a specific tool and an object during a specific task using the specific tool.

[0007] In addition, in a second aspect of the present disclosure, an information processing method is provided, which acquires information indicating a force generated between a first tool and an object during a trial operation by a robot using the first tool, and estimates information for performing a specific task using the first tool based on the acquired information and information indicating a force generated between a specific tool and an object during a specific task using the specific tool.

[0008] In addition, in a third aspect of the present disclosure, a program is provided that causes a computer to execute a process of acquiring information indicating a force generated between a first tool and an object during a trial operation by a robot using the first tool, and estimating information for performing a specific task using the first tool based on the acquired information and information indicating a force generated between a specific tool and an object during a specific task using the specific tool.

[0009] In addition, in a fourth aspect of the present disclosure, a trained model is provided for causing a computer to execute a process to estimate information for performing a specific task using a first tool based on information indicating a force generated between a first tool and an object during a trial motion by a robot using the first tool, and information indicating a force generated between a specific tool and an object during a specific task using the specific tool. Effect of the Invention

[0010] According to one aspect, suitable alternative tools can be estimated. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device according to an embodiment. [Diagram 3] FIG. 1 is a diagram illustrating an example of a configuration of an information processing device according to an embodiment. [Figure 4] 10 is a flowchart illustrating an example of processing of the information processing device according to the embodiment. [Diagram 5] FIG. 4 is a diagram illustrating an example of a task DB according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The principles of the present disclosure are described with reference to some exemplary embodiments. It should be understood that these embodiments are set forth for illustrative purposes only, to aid those skilled in the art in understanding and practicing the present disclosure, without implying any limitation on the scope of the present disclosure. The disclosure described herein may be implemented in various ways other than those described below. In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0013] <System configuration> The configuration of an information processing system 1 according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of the information processing system 1 according to an embodiment. In the example of Fig. 1, the information processing system 1 includes an information processing device 10, a robot (an example of an "external device") 20, and a sensor 30.

[0014] The information processing device 10 may be, for example, a device that controls the robot 20 using AI (Artificial Intelligence). The information processing device 10 acquires (measures, calculates) information indicating a force generated between an alternative tool and an object in an action of attempting a specific task using the alternative tool. Then, the information processing device 10 estimates information for performing a specific task using an alternative tool based on the acquired information and information indicating a force generated between a specific tool and an object in a specific task using a specific tool that has been registered in advance. Furthermore, the information processing device 10 may cause the robot 20 to perform an action of performing the specific task using an alternative tool based on the estimated information.

[0015] The robot 20 is a robot that performs tasks using an arm or the like. The robot 20 may be a device capable of performing various tasks, and the external shape is not limited. The robot 20 may be used for various purposes, such as for home use, exploration, and factory use. The sensor 30 is a sensor that measures the surroundings of the robot 20. The sensor 30 may be, for example, a force sensor, a camera, or a LiDAR. The force sensor is, for example, a sensor that measures the magnitude of force and torque acting in multiple directions (for example, three axes). The force sensor may be, for example, a piezoelectric type, an electrical resistance type (strain gauge type), or a capacitance type force sensor.

[0016] The numbers of the information processing devices 10, the robots 20, and the sensors 30 are not limited to those in the example of Fig. 1. The information processing devices 10 and the sensors 30 may be housed inside the housing of the robot 20.

[0017] <Hardware configuration> Fig. 2 is a diagram showing an example of a hardware configuration of an information processing device 10 according to an embodiment. In the example of Fig. 2, the information processing device 10 (computer 100) includes a processor 101, a memory 102, and a communication interface 103. These units may be connected by a bus or the like. The memory 102 stores at least a part of a program 104. The communication interface 103 includes an interface required for communication with other network elements.

[0018] When the program 104 is executed by the processor 101, memory 102, etc. in cooperation, the computer 100 performs at least some processing of the embodiments of the present disclosure. The memory 102 may be of any type suitable for a local technology network. The memory 102 may be, as a non-limiting example, a non-transitory computer-readable storage medium. The memory 102 may also be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory. Although only one memory 102 is shown in the computer 100, several physically different memory modules may be present in the computer 100. The processor 101 may be of any type. The processor 101 may include one or more of a general-purpose computer, a special-purpose computer, a microprocessor, a digital signal processor (DSP), and a processor based on a multi-core processor architecture, as a non-limiting example. The computer 100 may have multiple processors, such as application-specific integrated circuit chips that are time-slaved to a clock that synchronizes the main processor.

[0019] The embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor or other computing device.

[0020] The present disclosure also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, that execute on a target real or virtual processor device to perform the processes or methods of the present disclosure. Program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or split between program modules as desired in various embodiments. The machine-executable instructions of the program modules may be executed in local or distributed devices. In a distributed device, the program modules may be located in both local and remote storage media.

[0021] The program codes for carrying out the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes are provided to a processor or controller of a general purpose computer, a special purpose computer, or other programmable data processing apparatus. When the program codes are executed by the processor or controller, the functions / operations in the flowcharts and / or the block diagrams they implement are performed. The program codes are executed entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine, or entirely on a remote machine or server.

[0022] The program can be stored and provided to the computer using various types of non-transitory computer-readable media. The non-transitory computer-readable media include various types of tangible recording media. Examples of the non-transitory computer-readable media include magnetic recording media, magneto-optical recording media, optical disk media, semiconductor memories, and the like. The magnetic recording media include, for example, flexible disks, magnetic tapes, hard disk drives, and the like. The magneto-optical recording media include, for example, magneto-optical disks, and the like. The optical disk media include, for example, Blu-ray disks, CD (Compact Disc)-ROM (Read Only Memory), CD-R (Recordable), CD-RW (ReWritable), and the like. The semiconductor memory includes, for example, solid-state drives, mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, RAMs (random access memories), and the like. The program may also be provided to the computer by various types of temporary computer-readable media. Examples of the temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire or an optical fiber, or via a wireless communication path.

[0023] <Configuration> Next, a configuration of the information processing device 10 according to the embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the configuration of the information processing device 10 according to the embodiment. In the example of Fig. 3, the information processing device 10 has an acquisition unit 11, an estimation unit 12, and a control unit 13. Each of these units may be realized by cooperation between one or more programs installed in the information processing device 10 and hardware such as a processor 101 and a memory 102 of the information processing device 10.

[0024] The acquisition unit 11 acquires information indicating a force occurring between an alternative tool and an object during a trial motion by the robot 20 using the alternative tool. The estimation unit 12 estimates information for performing a specific task using an alternative tool based on the information acquired by the acquisition unit 11 and information indicating a force occurring between a specific tool and an object during a specific task using the specific tool. The control unit 13 controls the robot 20 to perform the specific task using an alternative tool based on the information estimated by the estimation unit 12.

[0025] <Processing> Next, an example of processing of the information processing device 10 according to the embodiment will be described with reference to Fig. 4 and Fig. 5. Fig. 4 is a flowchart showing an example of processing of the information processing device 10 according to the embodiment. Fig. 5 is a diagram showing an example of a task DB 501 according to the embodiment.

[0026] In step S1, the acquisition unit 11 acquires information indicating a specific task, which is a task to be performed by the robot 20 or the like using a tool. Here, the acquisition unit 11 may determine the specific task based on, for example, an instruction from a user. In this case, the acquisition unit 11 may, for example, recognize a voice from the user using AI or the like and determine the specific task desired by the user. Note that the task to be performed using a tool may include, for example, hammering a nail, hitting a pest, etc.

[0027] Next, the acquisition unit 11 acquires information indicating a force generated between the predetermined tool and an object by a specific task from the task DB 501 (step S2). The task DB 501 may be recorded in a storage device inside the information processing device 10, or may be recorded in a storage device outside the information processing device 10. In the example of Fig. 5, the task DB 501 records, in association with a task ID, a task, information indicating a predetermined tool used in the task, information indicating a force generated between the predetermined tool and an object by the task, operation information of the predetermined tool, and operation information of the robot 20. The task ID is identification information of a task.

[0028] The information indicative of the specific tool used in the task may include, for example, at least one of information indicative of the label (e.g., name) of the specific tool, the mass of the specific tool, and the shape of the specific tool.

[0029] The information indicating the force generated between the specified tool and the object by the task may include at least one of a force distribution map indicating the magnitude of the force generated at each part (each position) of the specified tool when the specified tool collides with the object, a force vector map indicating the magnitude and direction of the force generated at each part of the specified tool, and a peak value of the force. Note that the force distribution map and the force vector map may be, for example, a map in a polar coordinate system or a map in a Cartesian coordinate system. Also, the force distribution map and the force vector map may be, for example, three-dimensional map (point group, point cloud) data in a Cartesian coordinate system.

[0030] The motion information of the predetermined tool is information indicating the motion of the predetermined tool when the task is performed using the predetermined tool. The motion information of the predetermined tool may include, for example, at least one of the maximum speed of the predetermined tool, the speed at which the predetermined tool collides with an object (collision speed), and time series data of the speed of the predetermined tool.

[0031] The operation information of the robot 20 is information indicating the operation of the robot 20 when the robot 20 executes the task by using the predetermined tool. The information in the task DB 501 may be registered in advance, for example, at the time of shipping from a factory. In addition, the information processing device 10 may record a record of a specific task in the task DB 501 when the robot 20 is made to learn the specific task by, for example, imitation learning.

[0032] The processes in steps S3 to S5 below may be executed for each of the one or more substitute tools. In this case, the information processing device 10 may detect (estimate) one or more objects that can be substitute tools from among a plurality of objects present around the robot 20 by analyzing an image captured by the sensor 30, which is a camera, using a convolutional neural network (CNN).

[0033] Next, the control unit 13 causes the robot 20 to perform a trial motion (preliminary motion, preliminary motion) of a motion to execute a specific task using an alternative tool (step S3). The alternative tool is a tool other than the predetermined tool used in the specific task recorded in the task DB 501. The alternative tool and the predetermined tool may be the same type of tool (e.g., a predetermined hammer and another hammer). When the predetermined tool is a predetermined hammer, the alternative tool may be, for example, another hammer, a wooden mallet, a stone, a frying pan, etc. When the predetermined tool is a predetermined slipper, the alternative tool may be, for example, another slipper, a rolled up newspaper, a dustcloth, etc.

[0034] Here, the control unit 13 may cause the robot 20 to execute, as the trial motion, a motion that is smaller than the motion of the robot 20 using a predetermined tool to perform a specific task, by using an alternative tool. As a result, for example, if the specific task is a task of hammering a nail, a motion of lightly tapping the nail or the like with a stone is executed as the trial motion. In this case, the control unit 13 may cause the arm of the robot 20 to grasp the alternative tool, and cause the robot 20 to execute a motion in which at least one of the speed and the moving distance is smaller than the motion in the motion information recorded in the task DB 501.

[0035] Next, the acquisition unit 11 acquires information indicating the force generated between the substitute tool and the object in the trial motion of the robot using the substitute tool (step S4). Here, the acquisition unit 11 may calculate information indicating the substitute tool, the force generated between the substitute tool and the object, and motion information of the substitute tool based on information measured by the sensor 30, which is a force sensor provided on the substitute tool. In this way, for example, information indicating the force that a stone receives from a nail in the motion of tapping the nail with the stone is acquired.

[0036] Furthermore, the acquisition unit 11 may calculate information indicating the substitute tool, the force occurring between the substitute tool and an object, and motion information of the substitute tool based on information measured by, for example, a sensor 30 which is a force sensor provided in the robot 20. In this case, the sensor 30 may be provided in an arm other than the arm holding the substitute tool, among the multiple arms of the robot 20. In this way, for example, information indicating the force received by a stone when the arm is lightly tapped with the stone is acquired.

[0037] Furthermore, the acquisition unit 11 may calculate information indicating the substitute tool, the force generated between the substitute tool and an object, and motion information of the substitute tool based on, for example, information measured by the sensor 30, which is a force sensor provided on the robot 20, and an image (still image or video) captured by the sensor 30, which is a camera. In this case, the force sensor may be provided on an arm holding the substitute tool. Thereby, for example, information indicating the force received by a stone in the action of lightly tapping a nail or the like with a stone is calculated (estimated) based on the shape of the substitute tool recognized from the image and the information measured by the force sensor.

[0038] The acquisition unit 11 may also perform a dynamics simulation based on an image (still image or video) captured by the sensor 30, which is a camera, to calculate information indicating the alternative tool, a force occurring between the alternative tool and an object, and motion information of the alternative tool. In this case, the acquisition unit 11 may calculate the force occurring between the alternative tool and the object based on, for example, the change in the magnitude and direction of the force applied to the alternative tool.

[0039] Next, the estimation unit 12 estimates information for performing a specific task using an alternative tool based on the information acquired by the acquisition unit 11 in the processing of step S4 and information indicating the force generated between the specified tool and an object in a specific task using the specified tool, etc. (step S5).

[0040] Here, the estimation unit 12 may estimate the similarity between the force etc. generated between a predetermined tool and an object in a specific task using the predetermined tool and the force etc. generated between the substitute tool and an object in a trial motion using the substitute tool as information for performing a specific task using a substitute tool. As a result, for example, if the specific task is a task of hammering a nail, a stone whose mechanical properties are relatively closer to those of the predetermined hammer can be selected as the substitute tool rather than a frying pan whose shape is relatively closer to those of the predetermined hammer. Also, for example, if the specific task is a task of hitting a pest, a rolled up newspaper whose mechanical properties are relatively closer to those of the predetermined slippers can be selected as the substitute tool rather than a towel whose shape is relatively closer to those of the predetermined slippers.

[0041] The estimation unit 12 may use a convolutional neural network (CNN) to estimate (infer) a similarity between information indicating a force occurring between a specific tool and an object in a specific task using the specific tool and a force occurring between an alternative tool and an object in a trial motion. In this case, the estimation unit 12 may first reduce the number of dimensions of a map (image), for example, a two-dimensional force distribution map or force vector map indicating the force occurring between the alternative tool and the object, using a convolutional autoencoder. In addition, the estimation unit 12 may input each layer of a three-dimensional force distribution map or force vector map indicating the force occurring between the alternative tool and the object as two-dimensional data into a convolutional autoencoder, and reduce the number of dimensions of the map.

[0042] Then, the estimation unit 12 may use a convolutional neural network to calculate the similarity between the map of forces acting between an alternative tool and an object with a reduced number of dimensions and a map included in the information indicating the forces acting between a specified tool and an object by a task recorded in the task DB 501.

[0043] The estimation unit 12 may also estimate a similarity between the information indicating the predetermined tool used in the task, the force generated between the predetermined tool and the object by the task, and the motion information of the predetermined tool, and the information indicating the alternative tool, the force generated between the alternative tool and the object in the trial motion, and the motion information of the alternative tool. In this case, the estimation unit 12 may input the information indicating the force generated between the predetermined tool and the object in a specific task using the predetermined tool and the similarity between the force generated between the alternative tool and the object in the trial motion, the information indicating the alternative tool, and the motion information of the alternative tool to the trained model. In this case, the trained model may be generated by machine learning such as a neural network (e.g., a multi-layer perceptron), a decision tree, or ensemble learning. The information indicating the alternative tool may include at least one of the label (e.g., name) of the alternative tool, the mass of the alternative tool, and the shape of the alternative tool. The motion information of the alternative tool may also include at least one of the time series data of the maximum speed of the alternative tool in the trial motion, the speed at which the alternative tool collides in the trial motion, and the speed of the alternative tool in the trial motion.

[0044] Next, the control unit 13 controls the robot 20 to perform the specific task using an alternative tool based on the information estimated by the estimation unit 12 (step S6). The information processing device 10 may execute the processes from step S3 to step S5 for each of one or more alternative tools and estimate (calculate) the similarity for each alternative tool. Then, the control unit 13 may cause the robot 20 to perform the specific task using the alternative tool with the highest similarity.

[0045] <Modification> The information processing device 10 may be a device contained in one housing, but the information processing device 10 of the present disclosure is not limited to this. Each unit of the information processing device 10 may be realized by cloud computing configured by one or more computers, for example. Such information processing devices are also included as examples of the "information processing device" of the present disclosure.

[0046] The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit and scope of the present invention. [Explanation of symbols]

[0047] 1. Information Processing Systems 10. Information processing device 11 Acquisition Department 12 Estimation part 13 Control section 20. Robot 30 Sensors

Claims

1. an acquisition unit that acquires information indicating a force generated between a first tool and an object during a trial motion by a robot using the first tool; an estimation unit that estimates, based on the information acquired by the acquisition unit and information indicating a force generated between a specific tool and an object in a specific task using the specific tool, information indicating a similarity between the information indicating a force generated between the specific tool and an object in the specific task using the specific tool and information indicating a similarity between the force generated between the first tool and an object in a trial motion using the first tool; having Information processing device.

2. an acquisition unit that acquires information indicating a two-dimensional or three-dimensional force generated between a first tool and an object during a trial motion by a robot using the first tool; an estimation unit that estimates a similarity between the information indicating a force occurring between a specific tool and an object in a specific task using the specific tool and a force occurring between the first tool and an object in the trial motion using a convolutional neural network based on the information acquired by the acquisition unit and information indicating a force occurring between the specific tool and an object in a specific task using the specific tool; having Information processing device.

3. The acquisition unit is The information processing device according to claim 1 or 2, further comprising: acquiring information indicating a force generated between the first tool and an object in the trial motion, the motion being smaller than the motion when the robot performs the specific task using the specific tool.

4. acquiring information indicating a force generated between a first tool and an object during a trial motion by a robot using the first tool; based on the acquired information and information indicating a force occurring between a specific tool and an object in a specific task using the specific tool, estimating information indicating a similarity between the information indicating the force occurring between the specific tool and an object in the specific task using the specific tool and the force occurring between the first tool and an object in a trial motion using the first tool; Information processing methods.

5. acquiring information indicating a two-dimensional or three-dimensional force occurring between a first tool and an object during a trial motion by a robot using the first tool; based on the acquired information and information indicating a force occurring between a specific tool and an object in a specific task using the specific tool, using a convolutional neural network, estimating a similarity between the information indicating the force occurring between the specific tool and an object in the specific task using the specific tool and the force occurring between the first tool and the object in the trial movement; Information processing methods.

6. acquiring information indicating a force generated between a first tool and an object during a trial motion by a robot using the first tool; based on the acquired information and information indicating a force occurring between a specific tool and an object in a specific task using the specific tool, estimating information indicating a similarity between the information indicating the force occurring between the specific tool and an object in the specific task using the specific tool and the force occurring between the first tool and an object in a trial motion using the first tool; A program that causes a computer to carry out processing.

7. acquiring information indicating a two-dimensional or three-dimensional force occurring between a first tool and an object during a trial motion by a robot using the first tool; based on the acquired information and information indicating a force occurring between a specific tool and an object in a specific task using the specific tool, using a convolutional neural network, estimating a similarity between the information indicating the force occurring between the specific tool and an object in the specific task using the specific tool and the force occurring between the first tool and the object in the trial movement; A program that causes a computer to carry out processing.

8. Information indicating a force occurring between a first tool and an object in a trial motion performed by a robot using the first tool and information indicating a force occurring between a specific tool and the object in a specific task using the specific tool are input; based on each piece of input information, estimate information indicating a force occurring between the specific tool and an object in the specific task using the specific tool, and information indicating a similarity between a force occurring between the first tool and an object in a trial motion using the first tool, and output the estimated result; A trained model that allows a computer to execute a process.

9. Information indicating a two-dimensional or three-dimensional force occurring between a first tool and an object during a trial motion by a robot using the first tool, and information indicating a force occurring between a specific tool and an object during a specific task using the specific tool are input; using a convolutional neural network based on each piece of input information, to estimate a similarity between information indicating a force occurring between the specific tool and an object in the specific task using the specific tool and a force occurring between the first tool and an object in the trial movement, and outputting the estimated result; A trained model that allows a computer to execute a process.

Images