Robot control device, robot control method, and program

Abstract

This robot control device comprises: a first conversion unit that converts inputted information into a command value for controlling a robot; a robot drive unit that has a machine learning model for estimating, on the basis of the command value outputted by the first conversion unit, a parameter for driving the robot, and drives the robot on the basis of the estimation result by the machine learning model; and a second conversion unit that converts information obtained as a result of driving the robot into a response value having the same dimension as that of the command value. The robot control device performs, on the basis of the error between the command value and the response value in a first trial, a second trial by using a value obtained by correcting said command value as the command value to thereby repeatedly perform training of the machine learning model.

Description

Robot control device, robot control method and program

[0001] This application claims priority to Japanese Patent Application No. 2024-099569, filed on June 20, 2024, the contents of which are incorporated herein by reference.

[0002] Conventionally, there are known techniques for controlling the autonomous operation of a robot. In this technical field, for example, a technique such as that described in Non-Patent Document 1 can be exemplified.

[0003] “Do As I Can, Not As I Say: Grounding Language in Robotic Affordances,” arXiv:2204.01691, 2022

[0004] According to the technology described in Non-Patent Document 1, information such as images and language is converted to generate command values ​​for a robot. However, there is a problem in that the conversion to command values ​​alone does not allow for appropriate evaluation of the operation results. In other words, there is a problem in that if the robot's operation based on the generated command values ​​deviates from the desired operation (or if a deviation occurs), it is difficult to accurately correct the operation.

[0005] Therefore, an object of the present invention is to provide a robot control device, a robot control method, and a program that can correct command values ​​to more accurate values ​​through repeated learning even when there is a discrepancy between the command values ​​and the robot's operation.

[0006] (1) One aspect of the present invention is a robot control device that includes: a first conversion unit that converts input information into command values ​​for controlling a robot; a robot driving unit that has a machine learning model that infers parameters for driving the robot based on the command values ​​output by the first conversion unit, and drives the robot based on the inference results of the machine learning model; and a second conversion unit that converts information obtained as a result of driving the robot into a response value of the same dimension as the command value, and that repeatedly learns the machine learning model by performing a second trial using a value that is the corrected command value based on the error between the command value and the response value in a first trial.

[0007] (2) In one aspect of the present invention, in the robot control device described in (1) above, the degree of freedom of the command value output by the first conversion unit is lower than the degree of freedom of the input information.

[0008] (3) In one aspect of the present invention, in the robot control device described in (1) or (2) above, high-dimensional information having values ​​at each coordinate in a two-dimensional coordinate system is input to the first conversion unit, and the first conversion unit converts the high-dimensional information into the command value that is lower in dimension than the high-dimensional information based on the high-dimensional information.

[0009] (4) In one aspect of the present invention, in a robot control device described in any of (1) to (3) above, image information is input to the first conversion unit, and the first conversion unit converts the image information into a command value with a lower dimension than the number of dimensions of the image information based on the image information, and the second conversion unit converts the image obtained as a result of driving the robot into the response value.

[0010] (5) In one aspect of the present invention, in a robot control device described in any one of (1) to (4) above, the first conversion unit converts image information showing at least a portion of the robot and driving parameters of the robot at the time the image information was captured into the command value as input information.

[0011] (6) In one aspect of the present invention, in the robot control device described in any of (1) to (5) above, an optimization information acquisition unit is further provided that acquires information regarding whether to prioritize time, energy, or travel distance when driving the robot, and the machine learning model infers parameters for driving the robot based on whether to prioritize time, energy, or travel distance, based on the information acquired by the optimization information acquisition unit.

[0012] (7) One aspect of the present invention is a robot system comprising a machine learning model learned by a learning device described in any one of (1) to (6) above, and a robot driven based on the command value inferred by the machine learning model.

[0013] (8) One aspect of the present invention is a robot control method including: a first conversion step of converting input information into a command value for controlling a robot; a robot driving step of using a machine learning model to infer parameters for driving the robot based on the command value output by the first conversion step and driving the robot based on the inference result by the machine learning model; and a second conversion step of converting information obtained as a result of driving the robot into a response value of the same dimension as the command value, and repeating learning of the machine learning model by performing a second trial using a value obtained by correcting the command value based on an error between the command value and the response value in a first trial.

[0014] (9) One aspect of the present invention is a robot control device comprising: a first conversion unit that converts input sound information into command values ​​for controlling a robot; a machine learning model that infers parameters for driving the robot based on the command values ​​output by the first conversion unit, the robot driving unit driving the robot based on the inference results of the machine learning model; and a second conversion unit that converts sound information obtained as a result of driving the robot into a response value of the same dimension as the command values, wherein the machine learning model infers parameters for driving the robot so that the sound information obtained as a result of driving the robot falls within a predetermined range based on the sound information input to the first conversion unit.

[0015] (10) One aspect of the present invention is a program that causes a computer to execute each process, the program comprising: a first conversion process that converts input information into a command value for controlling a robot; a robot driving process that uses a machine learning model to infer parameters for driving the robot based on the command value output by the first conversion process and drives the robot based on the inference results from the machine learning model; and a second conversion process that converts information obtained as a result of driving the robot into a response value of the same dimension as the command value, and that repeatedly learns the machine learning model by performing a second trial using a value that is the corrected command value based on the error between the command value and the response value in a first trial.

[0016] (11) One aspect of the present invention is a robot control device comprising: a first conversion unit that converts input information into a command value for controlling a robot; a robot driving unit having a machine learning model that infers parameters for driving the robot based on the command value output by the first conversion unit, and that drives the robot based on the inference results of the machine learning model; and a second conversion unit that converts information obtained as a result of driving the robot into a response value of the same dimension as the command value, and that repeatedly learns the machine learning model by performing an (n+1)th trial using a value obtained by correcting the command value based on the error between the command value and the response value in the nth trial (n is a natural number greater than or equal to 1).

[0017] According to the present invention, even if there is a discrepancy between the command value and the robot's operation, the command value can be corrected to a more accurate command value through repeated learning.

[0018] FIG. 1 is an example for explaining a robot system according to an embodiment. FIG. 2 is a diagram for explaining an example of a functional configuration of a robot control device according to the embodiment. FIG. 3 is a diagram for explaining an example of a functional configuration of a robot device according to the embodiment. FIG. 4 is a diagram for explaining an example of repetitive control performed by the robot system according to the embodiment. FIG. 5 is a diagram for explaining a modified example of repetitive control performed by the robot system according to the embodiment. FIG. 6 is a flowchart showing a series of flows of a robot control method according to the embodiment. FIG. 7 is a diagram for explaining a first specific example to which the robot system according to the embodiment is applied. FIG. 8 is a diagram for explaining a second specific example to which the robot system according to the embodiment is applied. FIG. 9 is a diagram for explaining a third specific example to which the robot system according to the embodiment is applied. FIG. 10 is a diagram for explaining a fourth specific example to which the robot system according to the embodiment is applied. FIG. 11 is a diagram for explaining a fifth specific example to which the robot system according to the embodiment is applied. FIG. 12 is a diagram showing an example of an internal block diagram of hardware provided in the robot control device according to the embodiment.

[0019] [Embodiments] Preferred embodiments of a robot control device, a robot control method, and a program according to the present invention will be described in detail below with reference to the accompanying drawings. Note that the aspects of the present invention are not limited to these embodiments and include various modifications and improvements. In other words, the components described below include those that would be easily conceivable to a person skilled in the art or that are substantially identical, and the components described below can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the present invention. Furthermore, in the drawings, the scale and number of components may differ from the scale and number of the actual structures to make each configuration easier to understand.

[0020] 1 is an example for explaining a robot system according to one embodiment. First, an overview of a robot system 1 according to this embodiment will be described with reference to the same figure. The robot system 1 includes a robot control device 10 and a robot device 20. The scope of application of the robot system 1 according to this embodiment broadly includes systems including a robot that includes one or more movable parts (which can also be referred to as joints, motors, etc.).

[0021] The robot controller 10 controls the robot device 20. Specifically, the robot controller 10 controls the robot device 20 by outputting a predetermined drive signal. The predetermined drive signal may be inferred using a trained model that has been trained in advance.

[0022] Inference using the trained model is performed based on information input to the robot controller 10. The information input to the robot controller 10 can also be considered to be target values ​​that the robot device 20 aims to drive. An example of the information input to the robot controller 10 may be information that is directly used to control the robot device 20, such as angle or torque. Furthermore, the information input to the robot controller 10 is not limited to information that is directly used to control the robot device 20, but may also be information that is indirectly used to control the robot device 20, such as images, sounds, or language.

[0023] The robot control device 10 also acquires a response signal obtained as a result of controlling the robot device 20. The machine learning model included in the robot control device 10 learns the difference between a command value derived from input information (target value) and a response value derived from a response signal obtained as a result of controlling the robot device 20 as the error between the command value and the response value. That is, according to this embodiment, learning is performed based not only on the command value but also on the response value when the robot device 20 is driven based on the command value. By employing such a configuration, it becomes possible to accurately correct errors that occur when the robot device 20 is driven based only on the command value.

[0024] The robot system 1 performs learning to reduce the error between the command value and the response value, thereby correcting any discrepancy between the command value and the robot's operation. In particular, the robot system 1 can correct the command value to a more accurate value by repeatedly performing learning. An example of a specific functional configuration of the robot control device 10 will be described later.

[0025] The robot device 20 has at least one sensor and acquires some information. The robot device 20 performs some function by driving at least one motor in accordance with the acquired information. The robot device 20 may treat information obtained as a result of driving the motor as information obtained from a sensor. An example of a specific functional configuration of the robot device 20 will be described later.

[0026] 2 is a diagram illustrating an example of the functional configuration of a robot control device according to this embodiment. An example of the functional configuration of the robot control device 10 will be described with reference to the diagram. The robot control device 10 includes a control information acquisition unit 11, an optimization information acquisition unit 12, a control unit 13, a storage unit 14, and a robot driving unit 15.

[0027] The control information acquisition unit 11 acquires control information for controlling the robot device 20. The control information may be, for example, information such as an angle or torque that is directly used to control the robot device 20. The control information may also be position information indicating the movement of a part of the robot device 20 (for example, the tip of an arm) as a result of operation based on the angle, torque, or the like. The position information may be expressed, for example, by an image or the like and used as control information. When the control information is expressed by an image, the control information may also be, for example, a trajectory of movement of the robot device 20.

[0028] The control information may also be sound. In this case, sound information generated as a result of sliding of a part of the robot device 20 (for example, the tip of an arm) can be input as control information. For example, when the robot device 20 performs an action of rubbing an object by contacting it, a target value of the sound generated as a result of the rubbing can be used as control information for the sound.

[0029] The control information may be linguistic. In this case, instructions in natural language, such as "move 5 meters ahead" or "get the cup on the shelf," may be input to the robot device 20 as control information. The linguistic control information may be text data or voice information. When the linguistic control information is input as voice information, the control information acquisition unit 11 may be a microphone.

[0030] The optimization information acquisition unit 12 can acquire information regarding what to prioritize in optimization when controlling the robot device 20. The optimization information may be, for example, information regarding which of time, energy, or travel distance should be prioritized when driving. Specific examples of optimization information may include, for example, selecting one of time, energy, or travel distance, or inputting a specific numerical value. Examples of inputting a specific numerical value include "complete the operation within 10 seconds," "operate so that power consumption is 100 W (watts) or less," and "limit the travel distance to 10 m or less."

[0031] Note that acquisition of optimization information by the optimization information acquisition unit 12 is a selectable option, and the optimization information acquisition unit 12 is not necessarily a required component.

[0032] The control unit 13 performs comprehensive control of the robot control device 10. For example, the control unit 13 performs comprehensive control of the robot control device 10 by executing a robot control program stored in the storage unit 14.

[0033] The storage unit 14 is a memory for storing information. The storage unit 14 may include a non-volatile memory for storing a robot control program and a volatile memory for storing temporary information used for control.

[0034] The robot driving unit 15 drives the robot device 20 by outputting a driving signal to the robot device 20. The robot driving unit 15 includes a learning unit 151, an inference unit 152, and a machine learning model 153.

[0035] The learning unit 151 learns the machine learning model 153. The learning of the machine learning model 153 may be performed by, for example, supervised learning. When control is performed by bilateral control in the robot system 1, the learning unit 151 may perform supervised learning using the operation of the leader device (master device) and the operation of the follower device (slave device) as training data.

[0036] The inference unit 152 performs inference using the machine learning model 153, which is a trained model that has been trained in advance by the learning unit 151. The inference unit 152 obtains an inference result by inputting the control information acquired by the control information acquisition unit 11 or information generated by processing the control information (which may also be referred to as converted information) as input information to the machine learning model 153. In this case, the optimization information acquired by the optimization information acquisition unit 12 may also be input to the machine learning model 153 to perform inference.

[0037] The machine learning model 153 is trained to infer parameters for driving the robot. The parameters may be the torque of the motor of the robot device 20. Furthermore, the machine learning model 153 may infer parameters for driving the robot device 20 by further emphasizing time, energy, or travel distance based on the optimization information acquired by the optimization information acquisition unit 12.

[0038] 3 is a diagram illustrating an example of the functional configuration of the robot device according to this embodiment. An example of the functional configuration of the robot device 20 will be described with reference to the diagram. The robot device 20 includes a communication unit 21, a robot control unit 22, a storage unit 23, a motor 24, and a sensor 25.

[0039] The communication unit 21 communicates with the robot control device 10. The communication unit 21 acquires command values ​​from the robot control device 10 and performs an operation based on the acquired command values. The robot device 20 also acquires a response value as a result of performing the operation. The communication unit 21 transmits the acquired response value to the robot control device 10.

[0040] The robot control unit 22 controls the movement of the robot. The robot control unit 22 performs operations based on a control program stored in the storage unit 23 and command values ​​acquired via the communication unit 21. The robot control unit 22 drives the motor 24, for example, by outputting a motor drive signal to the motor 24. The motor 24 drives, for example, a joint. While one motor 24 is illustrated in the figure as an example, the robot device 20 may include multiple motors 24. The multiple motors 24 may be of different types (e.g., an AC motor and a DC motor). The robot control unit 22 acquires sensor values ​​detected by the sensor 25. Although one sensor 25 is illustrated in the figure as an example, the robot device 20 may include multiple sensors 25. The multiple sensors 25 may be of different types (e.g., a photointerrupter and a Hall sensor). The robot control unit 22 may control the driving of the motor 24 based on the acquired sensor values.

[0041] 4 is a diagram illustrating an example of repetitive control performed by the robot system according to this embodiment. An example of repetitive learning performed by the robot control device 10 will be described with reference to the diagram. The functions of the first conversion unit 41, the second conversion unit 42, and the controller 43 shown in the diagram may be realized by the control unit 13 included in the robot control device 10.

[0042] First, the first trial will be described. In the first trial, a target value such as an angle, torque, image, sound, or language is input to the first conversion unit 41. The target value may be a combination of information such as an angle, torque, image, sound, and language. The first conversion unit 41 converts the input information into a command value x for controlling the robot device 20. ｃｍｄ Convert to.

[0043] Here, it is preferable that the degree of freedom of the command value output by the first conversion unit 41 is lower than the degree of freedom of the input information. As a result of the conversion performed by the first conversion unit 41, the dimension of the information (or the degree of input freedom) is reduced.

[0044] Note that when the information input to the first conversion unit 41 is image information such as an RGB image, the RGB image is not treated as three-dimensional information in which pixel values ​​correspond to two-dimensional coordinates, but as high-dimensional information in which each coordinate can have multiple pixel values ​​(e.g., 256 bits). The first conversion unit 41 converts the high-dimensional information into low-dimensional information called command values. In other words, the first conversion unit 41 receives image information, and converts the input image information into command values ​​with a lower dimensionality than the number of dimensions of the image information.

[0045] Here, it may be preferable that the value input to the first conversion unit 41 be a combination of information such as angle, torque, image, sound, and language. For example, image information showing at least a portion of the robot device 20 and drive parameters (such as torque) of the robot device 20 at the time the image information was captured can be used as input information. Control based on such information can improve the accuracy of inference.

[0046] The image is not limited to a visible light image (visible image) such as an RGB image. For example, the image input to the first conversion unit 41 may be a thermal image (thermography) taken by a thermal camera, and may broadly include an image showing a pressure distribution acquired by a tactile sensor. In other words, an image can be described as high-dimensional information having a value at each coordinate in a two-dimensional coordinate system. Based on the input high-dimensional information, the first conversion unit 41 converts a command value x , which is lower in dimension than the high-dimensional information, into a command value x . ｃｍｄ It can also be said to be converted into

[0047] The robot driving unit 15 converts the command value x output by the first conversion unit 41 into ｃｍｄ Based on the inference result by the machine learning model 153, the robot driving unit 15 infers parameters for driving the robot device 20. Based on the inference result by the machine learning model 153, the robot driving unit 15 outputs a driving signal to the robot device 20, thereby driving the robot device 20.

[0048] Here, if the information input to the first conversion unit 41 is sound information, the machine learning model 153 may infer parameters for driving the robot device 20 so that the sound information obtained as a result of driving the robot falls within a predetermined range based on the sound information input to the first conversion unit 41.

[0049] A predetermined response value is obtained as a result of driving the robot device 20. The predetermined response value may be, for example, an angle, a torque, an image, a sound, a language, etc. The predetermined response value may be acquired by a sensor 25 provided in the robot device 20, or a camera, a microphone, etc. (not shown).

[0050] It is preferable that the predetermined response value is the same as the information input to the first conversion unit 41 (for example, if an image is input as a target value, the image is obtained as a response value), but this is not limiting. In other words, the predetermined response value and the information input to the first conversion unit 41 may be in different formats.

[0051] The second conversion unit 42 converts the obtained response value (i.e., information obtained as a result of driving the robot device 20) into the command value x ｃｍｄ Response value x of the same dimension as ｒｅｓ When, for example, an image is obtained as a result of driving the robot device 20, the second conversion unit 42 converts the response value x ｒｅｓ Convert to.

[0052] Next, the second trial will be described. In the second trial, the command value x ｃｍｄ and the response value x ｒｅｓ Specifically, the controller 43 performs iterative learning based on the error e between the command value x ｃｍｄ and the response value x ｒｅｓ The difference between the two is defined as an error e, and the command value x converted by the first conversion unit 41 is ｃｍｄ is corrected based on the error e and is used as the input to the robot driving unit 15 in the second trial.

[0053] Although the diagram shows the first two trials, it is possible to repeat learning from the third trial onwards using the same procedure as for trial 2. In this case, the first trial can be described as the nth trial (n is a natural number equal to or greater than 1), and the second trial as the (n+1)th trial.

[0054] According to this embodiment, the error can be evaluated offline (a model trained retroactively from the future to the past can look ahead) and a control input that reduces future errors can be calculated (the controller corrects the command value based on the error e). Specifically, from the second trial onwards, offline learning (500 samples / second) can be performed, all samples in the operation process can be evaluated, and the machine learning model can be repeatedly trained. In this case, more efficient learning can be achieved than with reinforcement learning, which does not evaluate intermediate samples and only learns the resulting error. Thus, according to this embodiment, an operation process consisting of multiple samples can be considered as one trial, and the errors of all samples can be evaluated in each trial, and the next trial can be corrected using the error, and learning can be repeated.

[0055] Furthermore, according to this embodiment, learning may be performed by comparing an ambiguous target value with the robot operation result, or learning may be performed including a first conversion unit that converts an ambiguous target value into a command value.

[0056] 5 is a diagram for explaining a modified example of the repetitive control performed by the robot system according to this embodiment. A modified example of the repetitive learning described with reference to FIG. 4 will be explained with reference to FIG. 5. In the modified example shown in FIG. 5, an output value of a VLA (Vision Language Action) is input to a first conversion unit 41. That is, in this modified example, a target value can be generated by a VLA model.

[0057] Generally, the VLA model excels at controlling general-purpose (greatest common denominator) operations. However, for operations specialized for a specific application, control using only the VLA model may be insufficient. Therefore, according to this embodiment, by repeatedly learning using the output value of the VLA model as a target value, general-purpose operations (which can also be called rough commands) can be made smoother by converting them into operations specialized for a specific application (which can also be called more precise commands).

[0058] 6 is a flowchart showing a series of steps in the robot control method according to this embodiment. The series of steps in the robot control method according to this embodiment will be described with reference to this figure. The robot control method described with reference to this figure may be performed using the robot control device 10 described above. A computer program that executes the series of steps in the robot control method shown in this figure may also be referred to simply as a program.

[0059] (Step S11 ) First, target values ​​such as angle, torque, image, sound, and language are input to the first conversion unit 41 .

[0060] (Step S12) The first conversion unit 41 converts the input information into a command value x ｃｍｄ Convert to.

[0061] (Step S13) The robot driver 15 calculates the obtained command value x ｃｍｄBased on the result of the inference, the robot driving unit 15 uses the machine learning model 153 to infer driving parameters for driving the robot device 20. Furthermore, based on the inference result, the robot driving unit 15 outputs a driving signal to the robot device 20 to drive the robot.

[0062] (Step S14) Next, information is acquired regarding the results of driving the robot device 20. The information may be, for example, an angle, a torque, an image, a sound, or a language.

[0063] (Step S15) The second conversion unit 42 converts the information obtained in step S14 into a response value x ｒｅｓ Convert to.

[0064] (Step S16) Command value x ｃｍｄ and the response value x ｒｅｓ The difference between the command value x and the command value x is calculated as the error e. ｃｍｄ and the response value x ｒｅｓ Since these are of the same dimension, the error between the command value and the response value can be easily calculated from the difference.

[0065] (Step S17) If the nth trial (next trial) is to be performed by the repetitive learning (i.e., step S17; YES), the process proceeds to step S18. If the nth trial (next trial) is not to be performed by the repetitive learning (i.e., step S17; NO), the process ends.

[0066] (Step S18) When the nth trial (next trial) is performed by repeated learning, the command value x ｃｍｄ A new command value is calculated based on the error e and the calculated new command value. Steps S13 and subsequent steps are repeated based on the calculated new command value.

[0067] Next, specific examples to which the robot system 1 according to this embodiment is applied will be described with reference to Figures 7 to 11. Note that these examples are merely examples for the purpose of explanation and do not limit the present embodiment in any way.

[0068] 7 is a diagram illustrating a first specific example to which the robot system according to this embodiment is applied. In the illustrated example, the robot device 20 is an experimental robot (hereinafter, sometimes referred to as a writing robot) that has a writing unit at the tip of an arm and writes on an object to be written on, such as paper. A specific example in which the robot device 20 is a writing robot will be described with reference to the same figure.

[0069] 7A shows an example of the external appearance of a robot device 20, which is a writing robot. In the example shown, a writing unit (e.g., a pen) provided at the tip of an arm is writing a character on a writing object such as a piece of paper. In the example shown in the figure, the writing unit is specifically writing the character "A."

[0070] In the case of such a writing robot, the information input to the first conversion unit 41 may be image information as shown in Fig. 7(B) . The writing robot writes characters, figures, symbols, etc. shown in the image information.

[0071] In this case, the information input to the first conversion unit 41 is the image information shown in Fig. 7(B). Based on the image information, the first conversion unit 41 converts a command value x ｃｍｄ Convert to.

[0072] In this case, the input information to the second conversion unit 42 is an image captured by an imaging device (not shown) provided in the robot device 20 (an image resulting from actual writing by the robot device 20). Based on the image information, the second conversion unit 42 converts a response value x ｒｅｓ Convert into command value x ｃｍｄ and the response value x ｒｅｓ and have the same dimension, it is possible to calculate the error e by comparing them with each other.

[0073] 8 is a diagram illustrating a second specific example to which the robot system according to this embodiment is applied. In the illustrated example, the robot device 20 is an experimental robot (hereinafter, sometimes referred to as a cleaning robot) that has a brush at the tip of an arm and cleans an object to be cleaned, such as a toilet bowl, by rubbing the brush against the object. A specific example in which the robot device 20 is a cleaning robot will be described with reference to the same figure.

[0074] 8A shows an example of the appearance of a robot device 20, which is a cleaning robot. In the example shown, a brush provided at the tip of an arm is brought into contact with an object to be cleaned, such as a toilet bowl. The cleaning robot cleans the object by sliding the brush while pressing the brush against the object with an appropriate amount of force.

[0075] In the case of such a cleaning robot, the information input to the first conversion unit 41 may be sound information as shown in FIG. 8(B). The horizontal axis of the sound information in the figure represents time, and the vertical axis represents amplitude (sound pressure). The sound information is the sound generated when the brush is rubbed against the object to be cleaned with an appropriate amount of force. The cleaning robot performs cleaning by sliding the brush, targeting the sound information. In this case, the sound information may be only the target sound information, or two pieces of sound information, an upper limit and a lower limit, may also be input. Although not described below, image information when the brush comes into contact with the object to be cleaned, torque information at that time, etc. may also be input.

[0076] In this case, the information input to the first conversion unit 41 is the sound information shown in Fig. 8(B). Based on the sound information, the first conversion unit 41 converts a command value x ｃｍｄ Convert to.

[0077] In this case, the input information to the second conversion unit 42 is a sound picked up by a microphone (not shown) provided in the robot device 20 (the sound resulting from the robot device 20 actually cleaning). Based on the sound information, the second conversion unit 42 converts a response value x ｒｅｓ Convert into command value x ｃｍｄ and the response value x ｒｅｓ and have the same dimension, it is possible to calculate the error e by comparing them with each other.

[0078] FIG. 9 is a diagram illustrating a third specific example to which the robot system according to this embodiment is applied. In the illustrated example, the robot device 20 is an experimental robot (hereinafter, sometimes referred to as a mobile robot) that moves autonomously while avoiding obstacles. The mobile robot may have functions such as grasping and moving luggage and cleaning its own bottom as a result of moving. In the following description, the purpose of the mobile device's movement is not important. With reference to the same figure, a specific example in which the robot device 20 is a cleaning robot will be described.

[0079] 9A shows the appearance of a robot device 20, which is a mobile robot, and an example of a location to which the robot can move. In the example shown, the trajectory of the movement of the mobile robot is indicated by an arrow. By moving along the trajectory indicated by the arrow, the mobile robot can move while avoiding obstacles.

[0080] In the case of such a mobile robot, the information input to the first conversion unit 41 may be image information as shown in FIG. 9(B). In the image information shown in the figure, the start point of the movement is indicated by "S" and the goal point of the movement is indicated by "G". The trajectory of movement from the start point to the movement point is arbitrary. In such a case, the optimization information acquired by the optimization information acquisition unit 12 may be used to determine whether to prioritize time, energy, movement distance, or the like when performing movement.

[0081] In this case, the information input to the first conversion unit 41 is the image information shown in Fig. 9(B). Based on the image information and the optimization information acquired by the optimization information acquisition unit 12, the first conversion unit 41 converts the command value x ｃｍｄ Convert to.

[0082] In this case, the input information to the second conversion unit 42 is an image (the actual trajectory of the movement of the robot device 20) captured by a camera (not shown) provided in the robot device 20 or installed in a location overlooking the movement range of the robot device 20. The trajectory may be shown as a video. Based on the image or video information, the second conversion unit 42 converts a response value x ｒｅｓ Convert into command value x ｃｍｄ and the response value x ｒｅｓ and have the same dimension, it is possible to calculate the error e by comparing them with each other.

[0083] 10 is a diagram illustrating a fourth specific example to which the robot system according to the present embodiment is applied. In the illustrated example, the robot device 20 is an experimental robot (hereinafter, sometimes referred to as a cooking robot) that cooks meat or other food by heating it using a cooking utensil such as a frying pan. A specific example in which the robot device 20 is a cooking robot will be described with reference to the same figure.

[0084] 10A shows the external appearance of a robot device 20, which is a cooking robot. Specifically, in the illustrated example, the cooking robot is grilling meat in a frying pan. The robot device 20, which is a cooking robot, grasps tongs (or drives an arm having the function of tongs), moves the meat to a suitable location, and pulls the meat out of the frying pan at a suitable timing, thereby cooking the meat to a suitable doneness.

[0085] In the case of such a cooking robot, the information input to the first conversion unit 41 may be thermal image information as shown in Fig. 10(B). The thermal image information shown in Fig. 10(B) shows temperature distribution information when meat is cooked to a suitable degree. The thermal image information may also be the temperature distribution when meat is placed in a frying pan.

[0086] In this case, the information input to the first conversion unit 41 is the thermal image information shown in Fig. 10(B). Furthermore, sound information indicating that the meat is cooked to a suitable doneness (for example, sound information as shown in Fig. 8(B)) may also be input to the first conversion unit 41. Based on the thermal image information and sound information, the first conversion unit 41 converts a command value x ｃｍｄ Convert to.

[0087] In this case, the input information to the second conversion unit 42 is a thermal image taken by a thermal camera (not shown) provided in the robot device 20, or sound information collected by a microphone (not shown) provided in the robot device 20. Based on at least one of the thermal image and the sound information, the second conversion unit 42 converts a response value x ｒｅｓ Convert into command value x ｃｍｄ and the response value x ｒｅｓ and have the same dimension, it is possible to calculate the error e by comparing them with each other.

[0088] 11 is a diagram illustrating a fifth specific example to which the robot system according to this embodiment is applied. In the illustrated example, the robot device 20 is an experimental robot (hereinafter, sometimes referred to as a grasping robot) that grasps a grasping object using a grasping unit having a tactile sensor. A specific example in which the robot device 20 is a grasping robot will be described with reference to the same figure.

[0089] 11A shows the appearance of a robot device 20, which is a gripping robot. Specifically, in the illustrated example, the gripping robot uses two gripping units (arms) to grip an object to be gripped. Note that the surfaces where the gripping units and the object to be gripped come into contact are provided with tactile sensors capable of measuring planar pressure distribution. The robot device 20, which is a gripping robot, may grip the object to be gripped and transport it to a predetermined location.

[0090] In the case of such a gripping robot, the information input to the first conversion unit 41 may be pressure distribution information as shown in Fig. 11(B) . The pressure distribution information shown in the figure indicates the pressure distribution when the object to be gripped is appropriately gripped.

[0091] In this case, the information input to the first conversion unit 41 is the pressure distribution information shown in Fig. 11(B). Based on the pressure distribution information, the first conversion unit 41 converts a command value x ｃｍｄ Convert to.

[0092] In this case, the input information to the second conversion unit 42 is pressure distribution information acquired by a tactile sensor provided in the robot device 20. Based on the pressure distribution information, the second conversion unit 42 converts a response value x ｒｅｓ Convert into command value x ｃｍｄ and the response value x ｒｅｓand have the same dimension, it is possible to calculate the error e by comparing them with each other.

[0093] FIG. 12 is a diagram showing an example of an internal block diagram of hardware included in a robot control device according to this embodiment. At least some of the functions of the robot control device 10 can be implemented using a computer. As shown in the figure, the computer includes a central processing unit 901, a RAM 902, an input / output port 903, input / output devices 904 and 905, and a bus 906. The computer itself can be implemented using existing technology. The central processing unit 901 executes instructions contained in a program read from the RAM 902 or the like. In accordance with each instruction, the central processing unit 901 writes data to the RAM 902, reads data from the RAM 902, and performs arithmetic and logical operations. The RAM 902 stores data and programs. Each element included in the RAM 902 has an address and can be accessed using the address. RAM stands for "random access memory." The input / output port 903 is a port through which the central processing unit 901 exchanges data with external input / output devices. The input / output devices 904 and 905 are also input / output devices. The input / output devices 904 and 905 exchange data with the central processing unit 901 via the input / output port 903. The bus 906 is a common communication path used within the computer. For example, the central processing unit 901 reads and writes data from and to the RAM 902 via the bus 906. Also, for example, the central processing unit 901 accesses the input / output port via the bus 906. Furthermore, all or part of the functional units provided in the robot control device 10 may be realized using hardware such as an ASIC, a PLD, or an FPGA. Furthermore, all or part of the functional units may be realized by a combination of software and hardware.

[0094] Summary of the Embodiment According to the above-described embodiment, the robot control device 10 includes the first conversion unit 41, which converts input information into command values ​​for controlling the robot device 20. The robot driving unit 15 uses the machine learning model 153 to infer parameters for driving the robot device 20 based on the command values ​​output by the first conversion unit 41, and drives the robot device 20 based on the inference results from the machine learning model 153. The second conversion unit 42 converts information obtained as a result of driving the robot device 20 into a response value of the same dimension as the command value. Furthermore, the robot control device 10 corrects the command value based on the error between the command value and the response value in the first trial, and uses the corrected value as a new command value in a second trial, thereby repeatedly learning the machine learning model. This configuration makes it possible to correct the command value to a more accurate value even when the command value does not correspond to the robot's operation.

[0095] Furthermore, according to the above-described embodiment, the degree of freedom of the command value output by the first conversion unit 41 is lower than the degree of freedom of the input information. In other words, according to the present embodiment, the first conversion unit 41 converts the input information into information with a lower dimension (input degree of freedom) than the input information. By adopting such a configuration, it becomes possible to control the robot device 20 with information that is easy for humans to understand by inputting high-dimensional information.

[0096] Furthermore, according to the above-described embodiment, the first conversion unit 41 converts into a command value input information, which is image information showing at least a part of the robot device 20 and the driving parameters of the robot device 20 at the time the image information was captured. In this way, by using input information with a larger amount of information, it becomes possible to correct the command value with higher accuracy.

[0097] Furthermore, according to the above-described embodiment, the robot control device 10 includes the optimization information acquisition unit 12, thereby acquiring information regarding whether to prioritize time, energy, or travel distance when performing driving. Furthermore, the machine learning model 153 infers parameters for driving the robot device 20 based on whether to prioritize time, energy, or travel distance, based on the information acquired by the optimization information acquisition unit 12. By employing such a configuration, driving can be performed in accordance with the user's requests. Furthermore, by employing such a configuration, inference can be performed better (e.g., faster) than when learning is performed by a human instructor.

[0098] Furthermore, according to the above-described embodiment, the robot control device 10 includes the first conversion unit 41, which converts input sound information into command values ​​for controlling the robot device 20. The robot control device 10 includes the robot driving unit 15, which uses the machine learning model 153 to infer parameters for driving the robot device 20 based on the command values ​​output by the first conversion unit 41 and drives the robot device 20 based on the inference results of the machine learning model 153. The robot control device 10 includes the second conversion unit 42, which converts sound information obtained as a result of driving the robot device 20 into a response value of the same dimension as the command value. The machine learning model 153 also infers parameters for driving the robot device 20 so that the sound information obtained as a result of driving the robot device 20 falls within a predetermined range based on the sound information input to the first conversion unit 41. By adopting such a configuration, it is possible to correct the command value to a more accurate value based on the sound information obtained as a result of driving the robot.

[0099] Note that all or part of the functions of each unit of the robot control device 10 in the above-described embodiment may be realized by recording a program for realizing these functions on a computer-readable recording medium, and reading and executing the program recorded on the recording medium into a computer system. Note that the term "computer system" here includes hardware such as an OS and peripheral devices.

[0100] Furthermore, "computer-readable recording media" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage units such as hard disks built into computer systems. Furthermore, "computer-readable recording media" may also include devices that dynamically store programs for a short period of time, such as communication lines used when transmitting programs over networks like the Internet or communication lines like telephone lines, or devices that store programs for a fixed period of time, such as volatile memory within computer systems that serve as servers or clients in such cases. Furthermore, the above-mentioned programs may be programs that realize some of the aforementioned functions, or may be programs that can realize the aforementioned functions in combination with programs already stored in the computer system.

[0101] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the spirit of the present invention.

[0102] According to the present invention, even if there is a discrepancy between the command value and the robot's operation, the command value can be corrected to a more accurate command value.

[0103] 1...Robot system, 10...Robot control device, 20...Robot device, 11...Control information acquisition unit, 12...Optimization information acquisition unit, 13...Control unit, 14...Memory unit, 15...Robot driving unit, 151...Learning unit, 152...Inference unit, 153...Machine learning model, 21...Communication unit, 22...Robot control unit, 23...Memory unit, 24...Motor, 25...Sensor, 41...First conversion unit, 42...Second conversion unit

Claims

1. A robot control device comprising: a first conversion unit that converts input information into command values ​​for controlling a robot; a robot driving unit having a machine learning model that infers parameters for driving the robot based on the command values ​​output by the first conversion unit, and that drives the robot based on the inference results of the machine learning model; and a second conversion unit that converts information obtained as a result of driving the robot into a response value of the same dimension as the command value, wherein the robot control device repeatedly learns the machine learning model by performing a second trial using a value that is the corrected command value based on the error between the command value and the response value in a first trial.

2. The robot control device according to claim 1, wherein the degree of freedom of the command value output by the first conversion unit is lower than the degree of freedom of the input information.

3. A robot control device as described in claim 1, wherein high-dimensional information having values ​​at each coordinate in a two-dimensional coordinate system is input to the first conversion unit, and the first conversion unit converts the high-dimensional information into a command value that is lower in dimension than the high-dimensional information based on the high-dimensional information.

4. A robot control device as described in claim 1, wherein image information is input to the first conversion unit, the first conversion unit converts the image information into a command value with a lower dimension than the number of dimensions of the image information, and the second conversion unit converts the image obtained as a result of driving the robot into a response value.

5. A robot control device as described in claim 1, wherein the first conversion unit converts into the command value image information that includes at least a portion of the robot and the driving parameters of the robot at the time the image information was captured as input information.

6. A robot control device as described in claim 1, further comprising an optimization information acquisition unit that acquires information regarding whether to prioritize time, energy, or travel distance when driving, and the machine learning model infers parameters for driving the robot based on whether to prioritize time, energy, or travel distance, based on the information acquired by the optimization information acquisition unit.

7. A robot control method comprising: a first conversion step of converting input information into command values ​​for controlling a robot; a robot driving step of using a machine learning model to infer parameters for driving the robot based on the command values ​​output by the first conversion step, and driving the robot based on the inference results from the machine learning model; and a second conversion step of converting information obtained as a result of driving the robot into a response value of the same dimension as the command value, wherein the machine learning model is repeatedly trained by performing a second trial using a value obtained by correcting the command value based on the error between the command value and the response value in a first trial.

8. A robot control device comprising: a first conversion unit that converts input sound information into command values ​​for controlling a robot; a robot driving unit having a machine learning model that infers parameters for driving the robot based on the command values ​​output by the first conversion unit, and that drives the robot based on the inference results of the machine learning model; and a second conversion unit that converts sound information obtained as a result of driving the robot into a response value of the same dimension as the command values, wherein the machine learning model infers parameters for driving the robot so that the sound information obtained as a result of driving the robot falls within a predetermined range based on the sound information input to the first conversion unit.

9. A program that causes a computer to execute each process, comprising: a first conversion process that converts input information into command values ​​for controlling a robot; a robot driving process that uses a machine learning model to infer parameters for driving the robot based on the command values ​​output by the first conversion process and drives the robot based on the inference results from the machine learning model; and a second conversion process that converts information obtained as a result of driving the robot into a response value of the same dimension as the command value, and that repeatedly learns the machine learning model by performing a second trial using a value that is the corrected command value based on the error between the command value and the response value in a first trial.

10. A robot control device comprising: a first conversion unit that converts input information into command values ​​for controlling a robot; a robot driving unit having a machine learning model that infers parameters for driving the robot based on the command values ​​output by the first conversion unit, and that drives the robot based on the inference results of the machine learning model; and a second conversion unit that converts information obtained as a result of driving the robot into a response value of the same dimension as the command value, wherein the device repeatedly learns the machine learning model by performing an (n+1)th trial using a value obtained by correcting the command value based on the error between the command value and the response value in the nth trial (n is a natural number greater than or equal to 1).

Images