Learning data generation device, learning data generation method, and program

Abstract

To provide a learning data generation device, a learning data generation method, and a program, capable of improving the accuracy of movement of a robot even by the learning based on operation of an unskilled operator.SOLUTION: A learning data generation device 40 generates learning data for imitative learning of a robot 30. The learning data generation device 40 comprises an acquisition part 43 which acquires movement information when a first user operates the robot 30, and a reaction when a second user recognizes the movement information. The learning data generation device 40 further comprises a generation part 46 which generates learning data on the basis of the reaction of the second user and the movement information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a training data generation device, a training data generation method, and a program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a device that learns how to operate a robot based on operation information that instructs the state of the robot and non-operation information that does not affect the state of the robot (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2016 / 189924 Summary of the Invention [Problem to be solved by the invention]

[0004] The accuracy of an operation by an unskilled operator is lower than that of an experienced operator. Even if non-operation information is referenced, if the operator is not skilled in an operation, it is difficult to improve the accuracy of the robot's movement by learning based on that operation.

[0005] An object of the present disclosure is to provide a learning data generation device, a learning data generation method, and a program that can improve the accuracy of a robot's movements even through learning based on the operations of an unskilled operator. [Means for solving the problem]

[0006] A training data generation device according to an embodiment of the present disclosure includes: A learning data generation device for generating learning data for imitation learning of a robot, comprising: an acquisition unit that acquires motion information when a first user operates the robot and a reaction of a second user when the second user recognizes the motion information; a generation unit that generates the learning data based on a reaction of the second user and the action information; Equipped with.

[0007] A learning data generating method according to an embodiment of the present disclosure includes: A learning data generation method for generating learning data for imitation learning of a robot, comprising the steps of: Acquiring operation information when a first user operates the robot and a reaction of a second user when the second user recognizes the operation information; generating the learning data based on a reaction of the second user and the motion information; Includes.

[0008] A program according to an embodiment of the present disclosure includes: A program for operating a learning data generation device for learning a robot's movement, comprising: causing the learning data generation device to acquire motion information when a first user operates the robot and a reaction of a second user when the second user recognizes the motion information; The learning data is generated based on the reaction of the second user and the motion information. Effect of the Invention

[0009] According to the learning data generation device, learning data generation method, and program of an embodiment of the present disclosure, the accuracy of a robot's movements can be improved even by learning based on the operations of an unskilled operator. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing an example of the configuration of a robot control system according to an embodiment. [Figure 2A] FIG. 11 is a diagram showing an example of operation information of a robot. [Figure 2B] 2B is a diagram showing an example in which the operation information in FIG. 2A is corrected to the planned operation information by deleting the operation error portion. FIG. [Figure 2C]2B is a diagram showing an example in which a part of the motion information in FIG. 2A that does not match the motion instructed by an expert is deleted and replaced with reference data. FIG. [Figure 2D] 2B is a diagram showing an example in which a portion of the motion information in FIG. 2A that does not match the motion instructed by an expert is deleted and interpolated. FIG. [Diagram 3] FIG. 1 is a diagram showing an example of a difference between the gaze of an expert and that of an unexpert. [Figure 4] 1 is a flowchart illustrating an example of a procedure of a training data generating method according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] (Example of the configuration of robot control system 1) As shown in FIG. 1, a robot control system 1 according to an embodiment includes a robot 30, a robot controller 20, a teaching data generating device 10, and a learning data generating device 40. At least one component of the robot control system 1 may be communicatively connected via a network, or may be communicatively connected without a network. At least one component of the robot control system 1 may be communicatively connected by wire or wirelessly. At least one component of the robot control system 1 may be communicatively connected via a dedicated line. At least one component of the robot control system 1 may be communicatively connected to each other in various other forms, not limited to these examples. Each component of the robot control system 1 will be specifically described below. A configuration including the robot 30 and the robot controller 20 is also referred to as a robot system.

[0012] <Teaching data generating device 10> The teaching data generation device 10 includes a control unit 11, a storage unit 12, and a communication unit 13. The control unit 11 includes a learning unit 14. The teaching data generation device 10 acquires learning data from the learning data generation device 40 via the communication unit 13. The teaching data generation device 10 generates teaching data by performing learning using the learning data via the learning unit 14. The teaching data is information that causes the robot controller 20 to automatically control the robot 30 to perform a task.

[0013] The control unit 11 or the learning unit 14 may be configured to include at least one processor or dedicated circuit. The processor may execute a program that realizes various functions of the teaching data generation device 10. The processor may be realized as a single integrated circuit. The integrated circuit is also called an IC (Integrated Circuit). The processor may be realized as a plurality of communicatively connected integrated circuits and discrete circuits. The processor may be configured to include a CPU (Central Processing Unit). The processor may be configured to include a DSP (Digital Signal Processor) or a GPU (Graphics Processing Unit). The processor may be realized based on various other known technologies. The dedicated circuit may include, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit), etc.

[0014] The storage unit 12 may be configured to include an electromagnetic storage medium such as a magnetic disk, or may be configured to include a memory such as a semiconductor memory or a magnetic memory. The storage unit 12 may be configured as a hard disk drive (HDD) or a solid state drive (SSD). The storage unit 12 stores various information and programs executed by the processor. The storage unit 12 may function as a work memory for the processor or a dedicated circuit. At least a part of the storage unit 12 may be included in the processor or the dedicated circuit. At least a part of the storage unit 12 may be configured as a storage device separate from the teaching data generation device 10.

[0015] The communication unit 13 may be configured to include a communication device configured to be able to communicate with other devices in a wired or wireless manner. The communication device may be configured to be able to communicate using a communication method based on various communication standards. The teaching data generation device 10 is communicatively connected to at least one component of the robot control system 1 by a line, whether wired or wireless. The teaching data generation device 10 and at least one component of the robot control system 1 are provided with an interface using a mutual standard protocol, and can communicate bidirectionally.

[0016] The teaching data generation device 10 may be configured to include one or more servers. The teaching data generation device 10 may be configured to cause multiple servers to execute parallel processing. The teaching data generation device 10 does not need to be configured to include a physical housing, and may be configured based on virtualization technology such as a virtual machine or a container orchestration system. The teaching data generation device 10 may be configured using a cloud service. When the teaching data generation device 10 is configured using a cloud service, it may be configured by combining managed services. In other words, the functions of the teaching data generation device 10 may be realized as a cloud service.

[0017] The teaching data generating device 10 may include at least one server and at least one database. The server functions as the control unit 11 or the learning unit 14. The database functions as the storage unit 12. The number of servers may be one or may be two or more. When the number of servers is one, the function realized by one server includes the function realized by multiple servers. The servers are connected to each other so as to be able to communicate with each other by wire or wirelessly. The number of databases may be one or may be two or more. The number of databases may be appropriately increased or decreased based on the capacity of data managed by the teaching data generating device 10 and the availability requirements required for the teaching data generating device 10. The database is connected to each server so as to be able to communicate with each other by wire or wirelessly. The teaching data generating device 10 may be connected to an external database. A teaching data generating system including the teaching data generating device 10 and an external database may be configured.

[0018] Although the teaching data generation device 10 is illustrated as one configuration in FIG. 1, multiple configurations can be operated as one system as necessary. In other words, the teaching data generation device 10 is configured as a platform with a variable capacity. By using multiple configurations as the teaching data generation device 10, even if one configuration becomes inoperable due to an unexpected event such as a natural disaster, the operation of the system can be continued using the other configurations. In this case, each of the multiple configurations is connected by a line, whether wired or wireless, and configured to be able to communicate with each other. The multiple configurations may be constructed across a cloud service and an on-premise environment.

[0019] <Robot Controller 20> The robot controller 20 acquires teaching data from the teaching data generation device 10 and controls the robot 30 based on the teaching data, thereby causing the robot 30 to perform a task. When the teaching data generation device 10 generates only scenario data as teaching data, the robot controller 20 may generate source code capable of controlling the robot 30 based on the scenario data.

[0020] The robot controller 20 may, for example, cause the robot 30 to perform an operation of reading a meter installed in a facility such as a factory. The robot controller 20 may cause the robot 30 to perform various operations, including but not limited to this. The robot controller 20 may or may not be connected to a cloud computing environment. When the robot controller 20 is not connected to a cloud computing environment, the operation of the robot controller 20 is completed in an on-premise environment. When the operation of the robot controller 20 is completed in an on-premise environment, the operation of the teaching data generation device 10 may be executed by the robot controller 20.

[0021] The robot controller 20 has a control unit 21, a storage unit 22, and an input unit 23. The control unit 21 can store teaching data and the like acquired from the teaching data generation device 10 in the storage unit 22, and control the robot 30 based on the acquired teaching data. The control unit 21 can also control each function of the robot controller 20. The storage unit 22 can store information necessary for controlling the robot 30 or the robot controller 20 itself.

[0022] The input unit 23 accepts input of a command to manually operate the robot 30. When a command to manually operate the robot 30 is input to the input unit 23, the control unit 21 controls the robot 30 based on the input command. As a result of controlling the robot 30, the control unit 21 generates information indicating how the robot 30 moved, and outputs the information to the learning data generation device 40. The information indicating how the robot 30 moved is also referred to as operation information.

[0023] The control unit 21 may be configured to include at least one processor or dedicated circuit. The processor may execute a program that realizes various functions of the teaching data generation device 10. The processor may be realized as a single integrated circuit. The integrated circuit is also called an IC (Integrated Circuit). The processor may be realized as a plurality of communicatively connected integrated circuits and discrete circuits. The processor may be configured to include a CPU (Central Processing Unit). The processor may be configured to include a DSP (Digital Signal Processor) or a GPU (Graphics Processing Unit). The processor may be realized based on various other known technologies. The dedicated circuit may include, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit), etc.

[0024] The storage unit 22 may be configured to include an electromagnetic storage medium such as a magnetic disk, or may be configured to include a memory such as a semiconductor memory or a magnetic memory. The storage unit 22 may be configured as a hard disk drive (HDD) or a solid state drive (SSD). The storage unit 22 stores various information and programs executed by the processor. The storage unit 22 may function as a work memory for the processor or a dedicated circuit. At least a part of the storage unit 22 may be included in the processor or the dedicated circuit. At least a part of the storage unit 22 may be configured as a storage device separate from the robot controller 20.

[0025] The input unit 23 may include an input device. The input device may include, for example, a joystick or the like. The input device may include, for example, a touch panel or a touch sensor, or a pointing device such as a mouse. The input device may include a physical key. The input device may include a voice input device such as a microphone. The input device is not limited to these examples and may include various other devices.

[0026] The input unit 23 may be realized as an input terminal separate from the robot controller 20. The number of input terminals is not limited to one, and may be two or more. The input terminal may be configured as a tablet terminal, or as a PC terminal such as a desktop PC (Personal Computer) or a notebook PC. The input terminal is not limited to these examples, and may be configured as various devices.

[0027] The robot controller 20 may include a communication device that acquires the teaching data from the teaching data generation device 10. The communication device of the robot controller 20 may be configured in the same manner as or similarly to the communication device of the teaching data generation device 10.

[0028] 1, one robot controller 20 is connected to one robot 30. One robot controller 20 may be connected to two or more robots 30. One robot controller 20 may control only one robot 30, or may control two or more robots 30. The number of robot controllers 20 and robots 30 is not limited to one, and may be two or more.

[0029] <Robot 30> The robot 30 may be configured as a robot arm having an arm. The arm may be configured as, for example, a 6-axis or 7-axis vertical articulated robot. The arm may be configured as a 3-axis or 4-axis horizontal articulated robot or a SCARA robot. The arm may be configured as a 2-axis or 3-axis Cartesian robot. The arm may be configured as a parallel link robot or the like. The number of axes constituting the arm is not limited to those exemplified.

[0030] The robot 30 may include an end effector attached to the arm. The end effector may include, for example, a gripping hand configured to grip a work object. The gripping hand may have a plurality of fingers. The number of fingers of the gripping hand may be two or more. The fingers of the gripping hand may have one or more joints. The end effector may include a suction hand configured to suck the work object. The end effector may include a scooping hand configured to scoop the work object. The end effector may include a tool such as a drill and be configured to perform various processes such as drilling a hole in the work object. The end effector is not limited to these examples and may be configured to perform various other operations.

[0031] The robot 30 can control the position of the end effector by operating the arm. The end effector may have an axis that is a reference for the direction in which it acts on a work object. If the end effector has an axis, the robot 30 can control the direction of the axis of the end effector by operating the arm. The robot 30 controls the start and end of the operation of the end effector acting on the work object. The robot 30 can move or process the work object by controlling the operation of the end effector while controlling the position of the end effector or the direction of the axis of the end effector.

[0032] The robot 30 may include, for example, an automated guided vehicle (AGV), an autonomous mobile robot (AMR), a drone, etc. The robot 30 is not limited to these, and may be configured in various other forms such as a vehicle, an electronic device, or a controlled machine.

[0033] The robot 30 may further include a sensor that detects a state of at least one component of the robot 30. The sensor may detect information about an actual position or posture of at least one component of the robot 30, or about a velocity or acceleration of at least one component of the robot 30. The sensor may detect a force acting on at least one component of the robot 30. The sensor may detect a current flowing through a motor that drives at least one component of the robot 30, or a torque of the motor. The sensor can detect information obtained as a result of an actual operation of the robot 30. The robot controller 20 can grasp the result of the actual operation of the robot 30 by acquiring the detection result of the sensor.

[0034] <Learning Data Generating Device 40> The learning data generating device 40 includes a control unit 41, a storage unit 42, an acquisition unit 43, and an output unit 44. The control unit 41 includes an analysis unit 45 and a generation unit 46. The learning data generating device 40 acquires, via the acquisition unit 43, from the robot controller 20, operation information of the robot 30 when an operator manually operates the robot 30. The learning data generating device 40 outputs, via the output unit 44, the operation information of the robot 30, and allows a user other than the operator to view a video representing the operation information. The operator is also referred to as a first user. The user viewing the video representing the operation information is also referred to as a second user.

[0035] The learning data generation device 40 acquires, by the acquisition unit 43, the reaction of the second user when the second user watches the video representing the motion information. The learning data generation device 40 analyzes the reaction of the second user by the analysis unit 45. The learning data generation device 40 generates learning data by the generation unit 46 based on the analysis result of the reaction of the second user and the motion information. The learning data generation device 40 outputs the generated learning data from the output unit 44 to the teaching data generation device 10.

[0036] The control unit 41 may be configured to include at least one processor or dedicated circuit. The processor may execute a program that realizes various functions of the training data generation device 40. The control unit 41 may be configured in the same manner as or similarly to the control unit 11 of the teaching data generation device 10.

[0037] The storage unit 42 stores various information used in the training data generation device 40, programs executed by the processor, etc. The storage unit 42 may be configured in the same manner as or similarly to the storage unit 12 of the teaching data generation device 10.

[0038] The acquisition unit 43 and the output unit 44 are configured to be able to communicate with other devices via wired or wireless communication. The acquisition unit 43 and the output unit 44 may be configured in the same manner as or similar to the communication unit 13 of the teaching data generation device 10.

[0039] The acquisition unit 43 may include a device that acquires a state when the first user is operating the robot 30. The acquisition unit 43 may include a camera that captures the environment in which the robot 30 is operating or the first user. The acquisition unit 43 may include a microphone that captures the sound of the environment in which the robot 30 is operating or the contents of the speech of the first user. The acquisition unit 43 may include a 360-degree camera that is worn on the head of the first user. The 360-degree camera may capture an image for reproducing the state of the first user when operating the robot 30. The acquisition unit 43 may include a sensor that detects the line of sight of the first user.

[0040] The acquisition unit 43 may include a device that acquires a reaction of the second user when the second user is viewing the operation information of the first user operating the robot 30. The acquisition unit 43 may include a microphone that acquires the contents of the speech of the second user. The acquisition unit 43 may include a sensor that detects the line of sight of the second user.

[0041] The acquisition unit 43 may include an input device that accepts input of an instruction or advice from the second user in response to an operation by the first user. The input device may include, for example, a touch panel or a touch sensor, or a pointing device such as a mouse. The input device may include a physical key. The input device may include a voice input device such as a microphone. The input device is not limited to these examples and may include various other devices.

[0042] The input device may be realized as an input terminal separate from the training data generation device 40. The number of input terminals is not limited to one, and may be two or more. The input terminal may be configured as a tablet terminal, or as a PC terminal such as a desktop PC (Personal Computer) or a notebook PC. The input terminal is not limited to these examples, and may be configured as various devices.

[0043] The output unit 44 may include a display device. The display device may include, for example, a liquid crystal display (LCD), an organic electroluminescence (EL) display or an inorganic electroluminescence (EL) display, or a plasma display panel (PDP). The display device is not limited to these displays and may include displays of various other types. The display device may include a light-emitting device such as an LED (Light Emitting Diode). The display device may include various other devices.

[0044] The output unit 44 may be configured to include an audio output device such as a speaker that outputs auditory information such as voice. The output unit 44 may be configured to include a device that provides a VR (Virtual Reality) environment. The output unit 44 is not limited to these examples and may be configured to include various other devices.

[0045] The training data generation device 40 may be configured to include one or more servers. The training data generation device 40 may be configured to cause a plurality of servers to execute parallel processing. The training data generation device 40 does not need to be configured to include a physical housing, and may be configured based on a virtualization technology such as a virtual machine or a container orchestration system. The training data generation device 40 may be configured using a cloud service. When the training data generation device 40 is configured using a cloud service, it may be configured by combining managed services. That is, the functions of the training data generation device 40 may be realized as a cloud service.

[0046] The training data generating device 40 may include at least one server and at least one database. The server functions as a control unit 41. The database functions as a storage unit 42. The number of servers may be one or more than one. When the number of servers is one, the function realized by one server includes the function realized by multiple servers. The servers are connected to each other so as to be able to communicate with each other by wire or wirelessly. The number of databases may be one or more than two. The number of databases may be appropriately increased or decreased based on the capacity of data managed by the training data generating device 40 and the availability requirements required for the training data generating device 40. The database is connected to each server so as to be able to communicate with each other by wire or wirelessly. The training data generating device 40 may be connected to an external database. A teaching data generating system including the training data generating device 40 and an external database may be configured.

[0047] Although the training data generation device 40 is illustrated as one configuration in FIG. 1, multiple configurations may be operated as one system as necessary. In other words, the training data generation device 40 is configured as a platform with a variable capacity. By using multiple configurations as the training data generation device 40, even if one configuration becomes inoperable due to an unexpected event such as a natural disaster, the operation of the system can be continued using the other configurations. In this case, each of the multiple configurations is connected by a line, whether wired or wireless, and configured to be able to communicate with each other. The multiple configurations may be constructed across a cloud service and an on-premise environment.

[0048] (Example of operation of robot control system 1) In the robot control system 1, the robot controller 20 accepts manual operation of the robot 30 by a first user through the input unit 23. The robot controller 20 controls the robot 30 based on the manual operation of the first user, and generates operation information when the robot 30 is manually operated by the first user. The robot controller 20 outputs the operation information of the robot 30 to the learning data generation device 40. The input unit 23 may be remotely operated.

[0049] The learning data generation device 40 acquires the motion information of the robot 30. The learning data generation device 40 causes the second user to watch a video representing the motion information of the robot 30, and acquires the reaction of the second user. The learning data generation device 40 generates learning data based on the result of analyzing the reaction of the second user and the motion information, and outputs the learning data to the teaching data generation device 10. Specifically, the learning data can be generated by performing an operation of extracting a part of the motion information of the robot 30 as learning data based on the result of analyzing the reaction of the second user, or an operation of correcting the motion information so that it can be used as learning data, etc.

[0050] The teaching data generating device 10 uses the learning data to perform learning so as to be able to imitate the behavior of the robot 30 when the first user manually operates the robot 30, and generates teaching data for the robot 30. Specifically, the teaching data generating device 10 can perform machine learning of the robot 30 by labeling information on the operating environment of the robot 30 and the like with the acquired learning data. Note that the information on the operating environment of the robot 30 and the like may be acquired as appropriate. Learning that imitates the behavior of the robot 30 when the first user manually operates the robot 30 is also referred to as imitation learning. The robot controller 20 generates teaching data using a learned model generated by learning, and automatically controls the robot 30 so as to imitate the behavior of the robot 30 operated by the first user based on the teaching data. Specifically, when the robot 30 is to be operated autonomously, for example, information on the operating environment of the robot 30 and the like may be input to the learned model, and the teaching data may be output. Note that the learned model may be, for example, an AI such as a neural network.

[0051] In the robot control system 1 according to the present disclosure, even if the first user is an unskilled operator, learning data is generated based on the reaction of the second user who is a skilled operator. A skilled operator is also called an expert. An unskilled operator is also called an unskilled person. By generating learning data based on the reaction of an expert, learning that imitates the operation of the robot 30 by an expert is performed by simply watching a video showing the operation information of the robot 30 when operated by an unskilled person, without the expert operating the robot 30. As a result, the accuracy of the teaching data generated by learning is improved. A specific operation example of the learning data generating device 40 will be described below.

[0052] <Information acquired by the learning data generation device 40> As described above, the learning data generation device 40 acquires, by the acquisition unit 43, the operation information of the robot 30 when the first user operates the robot 30. The acquisition unit 43 may acquire not only the operation information of the robot 30 but also various other information.

[0053] The acquisition unit 43 may acquire information on, for example, humans or objects present in the place where the robot 30 is operating. That is, the acquisition unit 43 may acquire information representing the environment in which the robot 30 is operating. The information representing the environment in which the robot 30 is operating is also referred to as environmental information. The environmental information may include video such as an overhead video of the environment in which the robot 30 is operating. The environmental information may include sounds that are heard in the environment in which the robot 30 is operating, such as the operating sounds of the robot 30, the operating sounds of the input unit 23 of the robot controller 20, the sounds of other objects, or the speech of humans other than the first user.

[0054] The acquisition unit 43 may acquire, for example, information regarding the state of the first user while operating the robot 30. The information regarding the state of the first user is also referred to as state information. The state information may include the content of the first user's utterance. The state information may include the first user's line of sight. The state information may include bio-information representing the first user's mental and physical state, such as the first user's heart rate. The state information may include a video of the first user.

[0055] <Getting the second user's reaction> The learning data generation device 40 outputs the information acquired by the acquisition unit 43 through the output unit 44 to allow the second user to recognize the information and experience the operation of the first user. For example, the output unit 44 may display a video showing the operation information of the robot 30 to allow the second user to view it. The output unit 44 may output at least a part of the environmental information or state information to allow the second user to recognize the information. The output unit 44 may output the speech content or the line of sight detection result of the first user as the state information.

[0056] When the first user is an unskilled person, the motion information of the robot 30 operated by the first user may include motions different from those of the robot 30 operated by the second user. Therefore, while watching the video of the motion information of the robot 30 operated by the first user, the second user points out by speech or input to an input device points out the differences from the motion of the robot 30 operated by the second user himself. In addition, the second user may spontaneously speak or move his body or eyes due to a sense of immersion when recognizing the video of the motion information of the robot 30 or the environmental information or state information. The motions performed by the second user while watching the video of the motion information of the robot 30 or recognizing the environmental information or state information are also collectively referred to as the reaction of the second user. The learning data generation device 40 acquires the reaction of the second user by the acquisition unit 43.

[0057] The following three modes may be prepared as an environment in which an operation by a first user who is an unskilled person is evaluated by a second user who is an expert. The environment in which the second user performs evaluation is not limited to the exemplified modes, and various other modes may be prepared.

[0058] <<VR experience of 30 robots' movements>> In the first mode, the output unit 44 may output the operation information of the robot 30 to a VR device so that a second user in a remote location can experience in VR the environment in which the first user operates the robot 30. The output unit 44 may output the operation information so that the second user can view the video of the operation information of the robot 30 from a viewpoint of his / her choice regardless of the line of sight of the first user when the first user operated the robot 30.

[0059] The acquisition unit 43 detects the line of sight or the reaction of the second user, such as speech, of the second user when the second user is experiencing the operation of the first user in VR. The reaction of the second user at this time is used to evaluate the operation information of the robot 30 by the operation of the first user.

[0060] <<Recognition of differences in gaze or speech between a first user and a second user>> In a second mode, the output unit 44 may output the operation information and the environmental information or state information of the robot 30 to a VR device so that the second user can further experience the environmental information or state information in addition to the operation information of the robot 30 in VR.

[0061] The acquisition unit 43 detects the line of sight or the reaction of the second user, such as speech, of the second user when the second user is experiencing the motion information of the robot 30 as a result of the operation by the first user and the environmental information or state information in VR.

[0062] The output unit 44 may play back the gaze or speech of the second user acquired when the second user watches the video in the first mode so that it can be compared with the gaze or speech of the first user included in the state information. In other words, the output unit 44 may output information so that the second user can recognize the difference in gaze or speech between the first user and the second user.

[0063] The acquisition unit 43 detects the reaction of the second user to the difference in gaze or speech between the first user and the second user. The output unit 44 may prompt the second user to explain the difference between the first user and the second user by consciously speaking or inputting to an input device, rather than by a natural reaction. The explanation may be given in natural language. The acquisition unit 43 may acquire the explanation spoken or input by the second user in natural language. The acquisition unit 43 may perform language analysis of the explanation to acquire the analysis result.

[0064] <<Correction of the action by a second user>> The output unit 44 may output the overhead video to a display or the like other than a VR device so that the second user can view the overhead video of the environment in which the robot 30 operates. The output unit 44 may display, in the overhead video, the movement path of the robot 30 when operated by the first user.

[0065] The output unit 44 may prompt the second user to input a motion path of the robot 30 that is assumed when the second user operates the robot 30. The acquisition unit 43 may acquire the motion path that is input by the second user and assumed by the second user. In other words, the acquisition unit 43 may acquire information corrected by the second user on the motion path of the robot 30 when operated by the first user. The correction by the second user is included in the reaction of the second user. The output unit 44 may display the motion path of the robot 30 when operated by the first user and the motion path that is assumed by the second user so that they can be compared.

[0066] <Analysis based on second user responses> The learning data generation device 40 analyzes the reaction of the second user by the analysis unit 45. The analysis unit 45 may, for example, determine, based on the reaction of the second user, a part of the operation information of the robot 30 that can be used as is in the learning data and a part that cannot be used as is in the learning data. The analysis unit 45 may determine, for a part that cannot be used as is in the learning data, whether the part can be used in the learning data if the part is modified. The analysis unit 45 may determine, for a part that cannot be used as is in the learning data, whether the part needs to be deleted and replaced with another operation.

[0067] The analysis unit 45 may analyze the second user's evaluation result of the first user's operation based on the tone or strength of the second user's voice when watching the video of the operation information of the robot 30 operated by the first user.

[0068] The evaluation result may include, for example, rejecting the operation of the first user. The analysis unit 45 determines that the motion information corresponding to the rejected operation is not usable as part of the learning data as it is.

[0069] The evaluation result may include, for example, sending improvement advice to the first user's operation. The analysis unit 45 may respect the first user's operation and determine that the operation information corresponding to the operation for which improvement advice has not been received is a part that can be adopted as it is in the learning data. The analysis unit 45 may determine that the operation information corresponding to the operation for which improvement advice has been received is a part that cannot be adopted as it is in the learning data.

[0070] The evaluation result may include, for example, praising or affirming the operation of the first user. The analysis unit 45 determines that the motion information corresponding to the operation that was praised or affirmed can be used as it is in the learning data.

[0071] The analysis unit 45 may analyze the evaluation result of the second user by applying the utterance of the second user to a predetermined relationship as shown in Table 1, for example.

[0072] [Table 1]

[0073] The analysis unit 45 may classify the second user's utterance into four categories: encouragement, criticism, advice, minor instructions, or major improvement. The analysis unit 45 may compare the utterance of the second user with an example utterance corresponding to each category. The analysis unit 45 may analyze the evaluation result of the second user based on the influence of the utterance corresponding to each category on the operation data of the first user corresponding to the motion information.

[0074] The analysis unit 45 may analyze the evaluation result of the second user based on the ending of the second user's utterance. For example, the analysis unit 45 may analyze that an utterance ending with "you should do ~" or "you should ~" corresponds to advice. The analysis unit 45 may analyze that an utterance ending with "please do ~" or "please do ~" corresponds to instructions. The analysis unit 45 is not limited to these examples and may analyze the second user's utterance based on various utterance patterns.

[0075] <Accumulation of reference data> The analysis unit 45 may analyze the corrections made by the second user and generate reference data of the operation of the robot 30 when operated by an expert as a sample. The analysis unit 45 may generate a library that accumulates the reference data and store it in the storage unit 42. The reference data may be used to replace a part of the operation information of the robot 30 that cannot be used as is in the learning data.

[0076] <Generating learning data> The generation unit 46 of the learning data generation device 40 generates learning data based on the operation information of the robot 30 acquired by the acquisition unit 43 and the analysis result by the analysis unit 45. The generation unit 46 outputs the generated learning data to the teaching data generation device 10.

[0077] <<Changes in behavior to accommodate operational mistakes>> For example, as shown in Fig. 2A, it is assumed that the acquisition unit 43 acquires the motion information of the robot 30 including the operation error of the first user. Meanwhile, the motion path of the robot 30 originally planned by the first user is shown by a dashed line as the planned motion information. The motion of the robot 30 passing a point where the planned path is bent and returning corresponds to the operation error of the first user.

[0078] The analysis unit 45 may determine that the operation information of the robot 30 includes an operation corresponding to an operation error of the first user based on the state information of the first user. The analysis unit 45 may identify an operation corresponding to an operation error of the first user among the operation information of the robot 30 based on the state information of the first user. The analysis unit 45 may identify an operation corresponding to an operation error of the first user based on, for example, the first user uttering that he / she is aware of the error. The analysis unit 45 may identify an operation corresponding to an operation error of the first user based on, for example, an increase in the heart rate of the first user. The analysis unit 45 may determine that the operation information of the robot 30 includes an operation corresponding to an operation error of the first user based on the reaction of the second user. The analysis unit 45 may identify an operation corresponding to an operation error of the first user based on the reaction of the second user.

[0079] The generating unit 46 acquires information on the operation corresponding to the operation error of the first user identified by the analyzing unit 45. As illustrated in Fig. 2B, the generating unit 46 may delete a part of the operation information of the robot 30 corresponding to the operation error of the first user and generate, as learning data, information changed to planned operation information.

[0080] <<Replacement of actions that deviate from those performed by an expert>> In FIG. 2A, the path of the robot 30 when the second user operates the robot 30 is shown by a dashed line as the motion instructed by the expert. The planned path of the first user is significantly different from the path operated by the second user. In other words, the motion of the robot 30 operated by the first user deviates from the motion of the second user who is an expert. The motion of the second user who is an expert is considered to be suitable as learning data.

[0081] The analysis unit 45 may determine, based on the reaction of the second user, that the motion information of the robot 30 includes a motion that deviates from the motion by the operation of the second user. The analysis unit 45 may identify, based on the reaction of the second user, the motion that deviates from the motion by the operation of the second user among the motion information of the robot 30.

[0082] For example, the analysis unit 45 may estimate that the motion intended by the second user is the motion represented by the dashed line based on the second user's utterance that the robot 30 should be moved diagonally in relation to the motion information of the robot 30 shown in Fig. 2A. The analysis unit 45 may identify, among the motion information of the robot 30, motions that deviate from the motion intended by the second user's operation by comparing the motion information of the robot 30 by the operation of the first user with the estimated result of the motion intended by the second user.

[0083] The analysis unit 45 may adopt the path input by the second user as the motion of the robot 30 operated by the second user, based on the second user inputting a path for moving the robot 30 diagonally. The analysis unit 45 may identify, among the motion information of the robot 30, a motion that deviates from the motion operated by the second user by comparing the motion information of the robot 30 operated by the first user with the motion input by the second user.

[0084] The generation unit 46 acquires the movements deviating from the movements resulting from the operation of the second user from the movement information of the robot 30 identified by the analysis unit 45. As illustrated in Fig. 2C or 2D, the generation unit 46 may delete the movements deviating from the movements resulting from the operation of the second user from the movement information of the robot 30, and leave the movements represented by the solid lines as valid portions.

[0085] When there is reference data corresponding to an action that connects the remaining valid actions, the generation unit 46 may generate information in which the deleted action is replaced by the reference data as learning data, as illustrated in FIG. 2C.

[0086] When replacing the deleted action, the generation unit 46 may select or customize the action to be replaced in consideration of the environmental information. For example, the generation unit 46 may select or customize the action to be replaced in consideration of the difference between the environment when the reference data was acquired and the current environment so as to avoid a danger predicted in the current environment. Even if the reference data exists, the generation unit 46 may replace the deleted action with another action without replacing it with the reference data.

[0087] Even if reference data exists, the generation unit 46 may determine that replacement with the reference data is not possible as a result of taking into consideration predicted dangers in the current environment.

[0088] When there is no reference data corresponding to the motion connecting the remaining valid motions, or when it is determined that the motion cannot be replaced by the reference data, the generating unit 46 generates information in which the deleted motion is left blank without replacing it with another motion as learning data, as exemplified in Fig. 2D. The teaching data generated by learning using the learning data including blank portions may be configured to leave the motions in the blank portions to the discretion of the robot 30.

[0089] <<Weighting settings>> The generating unit 46 sets weighting when generating learning data based on the motion information of the robot 30. The weighting of the learning data is set in order to identify characteristic data among a plurality of data included in the learning data. The weighting in the motion information of the robot 30 is set for the motion of the robot 30. The motion reproduced by the teaching data generated by learning using the motion information to which weighting is set as learning data is made to approach the motion to which a heavy weighting is set.

[0090] The analysis unit 45 may set a weighting for the motion information of the robot 30 based on the operation of the first user. The analysis unit 45 may divide the motion information into a plurality of motions, and set a weighting for each of the divided motions.

[0091] The analysis unit 45 may set a weighting for the operation information of the robot 30 based on the content of the utterance of the first user or the biological information of the first user, etc. The analysis unit 45 may set a weighting for the operation information of the robot 30 based on the reaction of the second user.

[0092] For example, as shown in Fig. 3, the analysis unit 45 may set a weighting on the motion information of the robot 30 based on a comparison between the gaze of the second user who is an unskilled person and the gaze of the first user who is an expert. In the example of Fig. 3, the gaze of the unskilled person is at the current position of the robot 30. On the other hand, the gaze of the expert is further along the planned path of the robot 30 than the current position of the robot 30. The analysis unit 45 may focus on the gaze of the expert and set a light weighting on the motion of the part of the motion information of the robot 30 where the expert's gaze is not present, and set a heavy weighting on the part of the motion where the expert's gaze is present.

[0093] <Generating teaching data> The teaching data generation device 10 acquires learning data from the learning data generation device 40 via the communication unit 13. The teaching data generation device 10 executes learning using the acquired learning data via the learning unit 14, and generates teaching data for operating the robot 30 under automatic control. The teaching data generation device 10 outputs the generated teaching data to the robot controller 20 via the communication unit 13. The teaching data generation device 10 may store the generated teaching data in the memory unit 12.

[0094] <Control of Robot 30> The control unit 21 of the robot controller 20 controls the robot 30 based on the teaching data generated by the teaching data generating device 10. The control unit 21 converts the teaching data into, for example, control commands for the motors of the robot 30, and controls the robot 30 by transmitting the control commands to the robot 30, causing the robot 30 to execute actions that imitate actions performed by the first user.

[0095] <Example of procedure for generating training data> The training data generation device 40 may execute a training data generation method including the steps of the flowchart illustrated in Fig. 4. The training data generation method may be realized as a training data generation program executed by a processor constituting the control unit 41 or the like of the training data generation device 40. The training data generation program may be a program that operates the training data generation device 40. The training data generation program may be stored in a non-transitory computer-readable medium.

[0096] The acquisition unit 43 acquires motion information of the robot 30 operated by a first user who is an unskilled person from the robot controller 20 (step S1). The output unit 44 reproduces the motion information of the robot 30 (step S2). Specifically, the output unit 44 may display a video of the motion information of the robot 30 to be viewed by a second user who is an expert. The acquisition unit 43 acquires a reaction of the expert when the second user who is an expert views the motion information of the robot 30 (step S3).

[0097] The control unit 41 determines whether there is a change target in the operation information of the robot 30 based on the reaction of the second user who is an expert (step S4). The change target is a part of the operation information of the robot 30 that cannot be used as it is as learning data. If there is no change target (step S4: NO), the control unit 41 proceeds to the procedure of step S9.

[0098] If there is a change target (step S4: YES), the control unit 41 determines whether there is a replacement target (step S5). The replacement target may be a motion that is significantly different from the motion operated by the second user who is an expert among the change targets. If there is no replacement target (step S5: NO), the control unit 41 proceeds to the procedure of step S8.

[0099] If there is a replacement target (step S5: YES), the control unit 41 determines whether there is reference data to replace the replacement target (step S6). If there is reference data (step S6: YES), the control unit 41 replaces the replacement target with the reference data (step S7). If there is no reference data (step S6: NO), the control unit 41 proceeds to step S8 without replacing the replacement target.

[0100] The control unit 41 changes the change target (step S8). If there is a replacement target that has not been replaced, the control unit 41 may delete the replacement target or may change the replacement target.

[0101] The control unit 41 sets a weighting to the motion information (step S9). When the control unit 41 changes or replaces the motion information, the control unit 41 sets a weighting to the changed or replaced motion information. The control unit 41 generates learning data by executing the procedure up to step S9, and outputs the generated learning data to the teaching data generation device 10. After executing the procedure of step S9, the learning data generation device 40 ends the execution of the procedure of the flowchart in FIG. 4.

[0102] (summary) As described above, in the robot control system 1 according to this embodiment, the learning data generation device 40 can correct the motion information of the robot 30 operated by the first user who is an unskilled person, based on the reaction of the second user who is an expert. As a result, the accuracy of the motion of the robot 30 can be improved even by learning based on the operation of the unskilled person. Even if an expert is not present at the site where the robot 30 is operated, the knowledge of the expert can be applied to the learning of the robot 30.

[0103] (Other embodiments) Hereinafter, configuration examples of a robot control system 1 and a learning data generating device 40 according to another embodiment will be described.

[0104] Information regarding the reaction of the second user who is an expert may be fed back to the first user who is an unskilled person. The first user can improve the operation of the robot 30 by comparing the first user's own operation with the operation pointed out by the second user. In addition, the first user can understand which operation of the robot 30 should be particularly noted among a series of operations of the robot 30 by checking the weighting in the operation information of the robot 30, and can improve the operation of the robot 30.

[0105] The first user may be a worker who actually uses the robot 30 at the sales destination of the robot system. The second user may be an engineer or trainer of the sales source of the robot system. According to the learning data generation device 40 of the present disclosure, even if the worker at the sales destination of the robot system is not yet skilled, a skilled engineer or trainer can correct the motion information when the worker operates the robot 30. As a result, the engineer or trainer of the sales source of the robot system can improve the accuracy of the motion of the robot 30 without visiting the sales destination of the robot system. In addition, training of the worker at the sales destination of the robot system is effectively performed.

[0106] An embodiment of the robot control system 1 has been described above, but the embodiment of the present disclosure can also be implemented as a method or program for implementing the system or device, or as a storage medium on which a program is recorded (for example, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a CD-RW, a magnetic tape, a hard disk, or a memory card). The program may be stored in a non-transitory computer-readable medium.

[0107] The form of the program implementation is not limited to application programs such as object code compiled by a compiler or program code executed by an interpreter, but may be in the form of a program module incorporated into an operating system. Furthermore, the program may or may not be configured so that all processing is performed only by the CPU on the control board. The program may be executed by another processing unit mounted on an expansion board or expansion unit added to the board as necessary. Therefore, it may be configured so that some or all of it is implemented.

[0108] Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can make various modifications or alterations based on the present disclosure. Therefore, it should be noted that these modifications or alterations are included in the scope of the present disclosure. For example, the functions, etc. included in at least one component, etc. can be rearranged so as not to cause logical inconsistency, and multiple components, etc. can be combined into one or divided.

[0109] In the above example, the training data generation device 40, the teaching data generation device 10, and the robot controller 20 are provided separately. As another example, the training data generation device 40 may be provided integrally with the teaching data generation device 10 or the robot controller 20. In this case, the control unit 41 of the training data generation device 40 may be realized by the control unit 11 of the teaching data generation device 10 or the control unit 21 of the robot controller 20.

[0110] In the above example, the second user checks the robot operation of the first user through a VR experience or the like, but the present disclosure is not limited to this. For example, the second user may check the robot operation of the first user through a monitor or the like, or the second user may be present near the first user and directly check the operation of the first user. In this case, the reaction of the second user may be acquired by acquiring the line of sight or voice of the second user through a monitor or a camera or microphone installed around the first user or the second user.

[0111] In the above example, the reaction of the second user is acquired through a camera or a microphone, but the reaction of the second user may be acquired through, for example, a command or a button to which the reaction of the second user is assigned in advance. Specifically, if the robot operation of the first user is good, a command indicating high evaluation may be input by clicking through an input device including a display unit.

[0112] All of the features described in this disclosure and / or all steps of any method or process disclosed herein may be combined in any combination except those combinations in which the features are mutually exclusive. Furthermore, each feature described in this disclosure may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly denied. Thus, unless expressly denied, each disclosed feature is merely one example of a generic series of the same or equivalent features.

[0113] Furthermore, the embodiments of the present disclosure are not limited to any of the specific configurations of the embodiments described above, but rather extend to any novel feature or combination of features described herein, or any novel method or process step or combination of features described herein.

[0114] The robot control system 1 and teaching data generation device 10 according to the present disclosure can be deployed in industrial product production sites, food processing sites that handle ingredients, or sites that manufacture cosmetics or pharmaceuticals.

[0115] In one embodiment, (1) a learning data generation device generates learning data for imitation learning of a robot, the learning data generation device including an acquisition unit that acquires motion information when a first user operates the robot and a reaction of a second user when the second user recognizes the motion information, and a generation unit that generates the learning data based on the reaction of the second user and the motion information.

[0116] (2) In the training data generation device described in (1) above, the acquisition unit may acquire at least one of environmental information of the robot and state information of the first user when the robot is operated by the first user. The generation unit may generate the training data further based on the environmental information or the state information.

[0117] (3) The learning data generation device according to (1) or (2) above may further include an analysis unit that analyzes a reaction of the second user. The analysis unit may determine, based on the reaction of the second user, a portion of the motion information that can be used in the learning data and a portion that cannot be used in the learning data. The generation unit may generate the learning data by modifying, replacing, or deleting the portion that cannot be used in the learning data.

[0118] (4) In the learning data generation device described in (3) above, the generation unit may replace parts that cannot be used in the learning data with reference data that serves as a model of the behavior of the robot 30 when the second user operates the robot.

[0119] (5) The learning data generation device according to any one of (1) to (4) above may further include an output unit that causes the second user to view the motion information in a VR environment.

[0120] (6) In the training data generation device described in (5) above, the acquisition unit may acquire a line of sight of the second user as the reaction of the second user.

[0121] (7) In the training data generation device according to any one of (1) to (6) above, the acquisition unit may acquire an utterance of the second user as a reaction of the second user.

[0122] In one embodiment, (8) a learning data generation method includes generating learning data for imitation learning of a robot, the learning data generation method including acquiring motion information when a first user operates the robot and a reaction of a second user when the second user recognizes the motion information, and generating the learning data based on the reaction of the second user and the motion information.

[0123] In one embodiment, (9) a program operates a learning data generation device that learns a robot's motion. The program causes the learning data generation device to acquire motion information when a first user operates the robot and a reaction of a second user when the second user recognizes the motion information, and to generate the learning data based on the reaction of the second user and the motion information. [Explanation of symbols]

[0124] 1. Robot Control System 10 Teaching data generating device (11: control unit, 12: storage unit, 13: communication unit, 14: learning unit) 20 robot controller (21: control unit, 22: memory unit, 23: input unit) 30. Robot 40 learning data generation device (41: control unit, 42: storage unit, 43: acquisition unit, 44: output unit, 45: analysis unit, 46: generation unit)

Claims

1. A learning data generation device for generating learning data for imitation learning of a robot, comprising: an acquisition unit that acquires motion information when a first user operates the robot and a reaction of a second user when the second user recognizes the motion information; a generation unit that generates the learning data based on a reaction of the second user and the action information; A training data generating device comprising:

2. The acquisition unit acquires at least one of environmental information of the robot and state information of the first user when the robot is operated by the first user, The training data generating device according to claim 1 , wherein the generating unit generates the training data further based on the environmental information or the state information.

3. An analysis unit that analyzes a reaction of the second user is further provided, The analysis unit determines, based on a response of the second user, a portion of the motion information that can be adopted in the learning data and a portion that cannot be adopted in the learning data; The training data generation device according to claim 1 , wherein the generation unit generates the training data by modifying, replacing, or deleting a portion that cannot be used in the training data.

4. The learning data generation device according to claim 3 , wherein the generation unit replaces portions that cannot be used in the learning data with reference data that serves as a model for the behavior of the robot 30 when the second user operates the robot.

5. The training data generation device according to claim 1 , further comprising an output unit that allows the second user to view the motion information in a VR environment.

6. The training data generation device according to claim 5 , wherein the acquisition unit acquires a line of sight of the second user as the reaction of the second user.

7. The training data generation device according to claim 5 , wherein the acquisition unit acquires an utterance of the second user as the reaction of the second user.

8. A learning data generation method for generating learning data for imitation learning of a robot, comprising the steps of: acquiring operation information when a first user operates the robot and a reaction of a second user when the second user recognizes the operation information; generating the learning data based on the reaction of the second user and the action information; A method for generating training data, comprising:

9. A program for operating a learning data generation device for learning a robot's movements, comprising: causing the learning data generation device to acquire motion information when a first user operates the robot and a reaction of a second user when the second user recognizes the motion information; A program that generates the learning data based on the reaction of the second user and the motion information.

Images