Information processing device, learning device, information processing system, robot system, and evaluation method

Abstract

To provide an information processing device configured to motivate a user to operate a robot, a learning device, an information processing system, and a robot system.SOLUTION: In a control apparatus 170, an information processing device includes: a detection processing unit which executes processing for detecting operation-related information of a robot that moves in accordance with an operation command, during a task executed with the robot; an evaluation processing unit which executes processing for comparing the operation-related information with a reference so as to carry out evaluation during the task; and a presentation processing unit which executes processing for causing a presentation device to present an evaluation result during the task. The evaluation processing unit executes the evaluation processing at a timing which is at least one of a timing at which the evaluation target operation of the robot occurs, and a timing at which an evaluation target stage of the task occurs. The presentation processing unit causes the presentation device to present evaluation results pertaining to the evaluation target operations in the order in which the evaluation target operations are executed, on a display of an operation terminal, and causes the presentation device to present evaluation results pertaining to the evaluation target stages in the order in which the evaluation target stages are executed, on the display of the operation terminal.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, a learning device, an information processing system, and a robot system. [Background technology]

[0002] There are technologies for evaluating robot operators. For example, a management system disclosed in Patent Document 1 provides an environment in which the operator can operate a robot located near a remote object while making the operator feel as if the remote object is nearby. A server included in the management system acquires status information indicating the state of the robot or the state of the operator when the operator operates the robot, and determines the operator's suitability for operating the robot by comparing the state indicated by the status information with a reference level. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-135362 A Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, the skill level of robot operation is determined as the aptitude. For example, the operator can only know the overall skill level of the operation for the task through the determined level. The operator cannot find a specific measure to improve the robot operation skill, and there is a possibility that the operator cannot increase the motivation to improve the skill and the will to operate.

[0005] An object of the present disclosure is to provide an information processing device, a learning device, an information processing system, and a robot system that improve a user's motivation to operate a robot. [Means for solving the problem]

[0006] An information processing device according to one aspect of the present disclosure is an information processing device that evaluates an operation of a robot to cause the robot to perform a task, the information processing device including: a detection processing unit that performs, during the execution of the task, a detection process that detects action-related information, which is information related to the action of the robot that operates according to an operation command input by a user to an operation device; an evaluation processing unit that performs, during the execution of the task, an evaluation process that compares the action-related information detected by the detection processing unit with an action-related criterion, which is a criterion related to the action of the robot, and evaluates the information; and a presentation processing that performs, during the execution of the task, a presentation device that presents the evaluation result of the evaluation processing unit to information presented by the user while the user operates the operation device. The evaluation processing unit executes the evaluation process at least one of the timing for each evaluation target action, which is the action of the robot to be evaluated, and the timing for each evaluation target stage, which is the work stage to be evaluated in the work, and the presentation processing unit, when the evaluation process is executed at the timing for each evaluation target action, causes the presentation device to present the evaluation results for the evaluation target action in the same order as the execution order of the evaluation target action, and when the evaluation process is executed at the timing for each evaluation target stage, causes the presentation processing unit to present the evaluation results for the evaluation target stage in the same order as the execution order of the evaluation target stage. Effect of the Invention

[0007] According to the technology disclosed herein, it is possible to increase a user's motivation to operate a robot. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of a configuration of a robot system according to an embodiment; [Diagram 2] FIG. 1 is a perspective view showing an example of a configuration of a robot according to an embodiment; [Diagram 3] FIG. 1 is a block diagram showing an example of a hardware configuration of a control device according to an embodiment. [Figure 4]FIG. 1 is a block diagram showing an example of a functional configuration of a control device according to an embodiment; [Diagram 5] 1 is a flowchart showing an example of an operation of a robot system according to an embodiment. [Figure 6] FIG. 1 is a plan view showing an example of an operation of the robot system according to the embodiment; [Figure 7] FIG. 11 is a block diagram showing an example of a functional configuration of an information processing device and a learning device according to a modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] (Embodiment) Hereinafter, the embodiments of the present disclosure will be described with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. In addition, among the components in the following embodiments, components that are not described in the independent claims showing the highest concept are described as optional components. In addition, each figure in the attached drawings is a schematic diagram and is not necessarily illustrated precisely. Furthermore, in each figure, substantially the same components are given the same reference numerals, and duplicated descriptions may be omitted or simplified. In addition, in this specification and the claims, "apparatus" may mean not only one apparatus but also a system consisting of multiple apparatuses.

[0010] [Robot system configuration] The configuration of a robot system 1 according to an embodiment will be described with reference to FIG. 1. FIG. 1 is a diagram showing an example of the configuration of the robot system 1 according to an embodiment. The robot system 1 includes at least one robot 100, at least one operation terminal 200, and a server 300. Although not limited thereto, in this embodiment, the robot system 1 is configured to provide a service to a user P using the robot 100 operated from a remote location. The robot system 1 can be used in various service industries such as nursing care, medical care, cleaning, security, guidance, rescue, cooking, sales, rental, and provision of goods.

[0011] The robot 100 is a robot that can perform tasks for providing services and the like. In this specification and the claims, an "task" may refer to a series of actions performed by the robot 100 to accomplish, build, and / or create a productive event that is expected to produce some useful result. For example, an "task" may not refer to a series of actions performed by the robot 100 in the course of teaching the robot 100. For example, an "task" may not refer to a series of actions performed by the robot 100 for play, a game, and / or entertainment.

[0012] Although not limited thereto, in this embodiment, a plurality of robots 100 are placed in one service providing area AS, which is a place where a service is provided to a user P. Furthermore, at least one operation terminal 200 is placed in each of a plurality of operation areas AO located away from the service providing area AS.

[0013] Each robot 100 is configured to be connectable to the communication network N via wireless communication so as to be able to communicate data. The communication connecting the robot 100 and the communication network N may be wired communication, or a combination of wired communication and wireless communication, etc. Each operation terminal 200 is configured to be connectable to the communication network N so as to be able to communicate data via wired communication, wireless communication, or a combination of these. One robot 100 and one operation terminal 200 can be connected to each other via the communication network N so as to be able to communicate data. Any wired communication or wireless communication may be used.

[0014] The server 300 manages communication via the communication network N. The server 300 includes a computer device. The server 300 manages authentication, connection, and disconnection of communication between the robot 100 and the operation terminal 200. For example, the server 300 stores identification information and security information of the robot 100 and the operation terminal 200 registered in the robot system 1, and authenticates the qualification of each operation terminal 200 to connect to the robot system 1 using the information. The server 300 manages transmission and reception of data between the robot 100 and the operation terminal 200, and the data may pass through the server 300. The server 300 may be configured to convert data transmitted from a transmission source into a data format usable by the transmission destination. The server 300 may also be configured to store and accumulate information, commands, data, and the like transmitted and received between the robot 100 and the operation terminal 200 during the operation of the robot 100.

[0015] The communication network N is not particularly limited, and may include, for example, a local area network (LAN), a wide area network (WAN), the Internet, or a combination of two or more of these. The communication network N may be configured to use short-distance wireless communication such as Bluetooth (registered trademark) and ZigBee (registered trademark), a network dedicated line, a dedicated line of a communication carrier, a public switched telephone network (PSTN), a mobile communication network, the Internet network, satellite communication, or a combination of two or more of these. The mobile communication network may use a fourth generation mobile communication system and a fifth generation mobile communication system, etc. The communication network N may include one or more networks. In this embodiment, the communication network N is the Internet.

[0016] [Operation terminal configuration] The configuration of an operation terminal 200 according to an embodiment will be described with reference to Fig. 1. The operation terminal 200 is configured to be able to accept input of commands, information, data, etc. by an operator PO and to be able to output the accepted commands, information, data, etc. to another device. The operation terminal 200 includes an operation input device 201, a terminal computer 202, a presentation device 203, and a communication device 204. The operation input device 201, the terminal computer 202, the presentation device 203, and the communication device 204 may have an integrated configuration to form one device, each may be configured to form a device independently and be connected to each other, or at least two of them may be configured to form one device and be connected to another device.

[0017] The operation terminal 200 is an example of an operation device. The configuration of the operation terminal 200 is not particularly limited, and for example, the operation terminal 200 may be a computer such as a personal computer, a smart device such as a smartphone and a tablet, a personal information terminal, a game terminal, a known teaching device used for teaching a robot, a known operation device for a robot, other operation devices, other terminal devices, devices using these, and devices improved from these. The operation terminal 200 may be a dedicated device devised for the robot system 1, or may be a general-purpose device available in the general market. In this embodiment, a known general-purpose device is used for the operation terminal 200. The device may be configured to realize the function of the operation terminal 200 of the present disclosure by installing dedicated software.

[0018] The operation input device 201 is configured to receive an input from an operator PO and to output signals indicating the inputted command, information, data, etc. to the terminal computer 202. The configuration of the operation input device 201 is not particularly limited, and may include, for example, a device to which an input is given through an operation of the operator PO, such as a button, a lever, a dial, a joystick, a mouse, a key, a touch panel, and a motion capture. The operation input device 201 may include an imaging device that captures an image of the operator PO, etc., and an audio input device such as a microphone that receives audio input of the operator PO, etc. The operation input device 201 may be configured to output the captured image data and a signal indicating the inputted audio to the terminal computer 202.

[0019] The terminal computer 202 is configured to process commands, information, data, etc. received via the operation input device 201 and output them to other devices, and to receive input of commands, information, data, etc. from other devices and process those commands, information, data, etc.

[0020] The presentation device 203 is configured to be able to present information to the operator PO. In this embodiment, the presentation device 203 includes a display capable of displaying an image to the operator PO, although this is not limited thereto. The presentation device 203 displays an image of image data received from the terminal computer 202. The presentation device 203 may include an audio output device such as a speaker capable of emitting audio to the operator PO. The presentation device 203 outputs the audio of audio data received from the terminal computer 202.

[0021] The communication device 204 includes a communication interface that is connectable to the communication network N. The communication device 204 is connected to the terminal computer 202, and connects the terminal computer 202 and the communication network N so that data communication can be performed between them. The communication device 204 may include communication devices such as a modem, an ONU (Optical Network Unit), a router, and a mobile data communication device. The communication device 204 may include a computer device having a calculation function or the like.

[0022] [Robot configuration] The configuration of the robot 100 according to the embodiment will be described with reference to FIG. 2. FIG. 2 is a perspective view showing an example of the configuration of the robot 100 according to the embodiment. Each robot 100 includes one transport vehicle 110, at least one robot arm 120, at least one end effector 130, a secondary battery module 141, a power supply circuit 142, a communication device 143, imaging devices 144, 145, and 146, a sound collection device 147, a display device 148, a sound output device 149, and a control device 170. The control device 170 includes an information processing device 180 and a robot controller 190. The information processing device 180 and the robot controller 190 may be configured as separate devices or may be configured as an integrated device. Although not limited thereto, in this embodiment, a robot arm that can also function for industrial use is used for the robot arm 120. The quantity of each of the above components is not limited to the above quantity and can be changed as appropriate.

[0023] The transport vehicle 110 is configured to be self-propelled. In this embodiment, the transport vehicle 110 includes, but is not limited to, two drive wheels 111, four auxiliary wheels 112, and a transport drive device 113 that drives the two drive wheels 111. The transport drive device 113 includes a servo motor (not shown) as an electric actuator that is a drive source. The servo motor is controlled by the robot controller 190. The transport drive device 113 can drive the two drive wheels 111 to rotate in various rotation directions and at various rotation speeds, thereby causing the transport vehicle 110 to travel at various speeds in the forward direction D1A, the backward direction D1B, and the turning direction.

[0024] Although not limited thereto, in this embodiment, two robot arms 120A and 120B are mounted on the transport vehicle 110 via a base 120C as the robot arm 120. Both the robot arms 120A and 120B are configured to be rotatable about an axis S1 in a direction from the transport vehicle 110 toward the base 120C, and constitute a coaxial double-arm robot arm. The robot arm 120A includes links 121A to 124A and arm driving devices M1A to M4A (see FIG. 3) that drive joints that connect the links 121A to 124A to each other. The robot arm 120B includes links 121B to 124B and arm driving devices M1B to M4B (see FIG. 3) that drive joints that connect the links 121B to 124B to each other. The arm driving devices M1A to M4A and M1B to M4B include servo motors as electric actuators that are drive sources. Each servo motor is controlled by the robot controller 190. The links 121A and 121B are connected to the base 120C via joints (not shown). The links 124A and 124B include a mechanical interface connectable to the end effector 130.

[0025] The robot arms 120A and 120B as described above have the configuration of a horizontal articulated arm, but may have any configuration. For example, the robot arms 120A and 120B may be vertical articulated, polar coordinate, cylindrical coordinate, rectangular coordinate, or other types of robot arms. The number of robot arms 120 arranged on the transport vehicle 110 may be one or more.

[0026] The end effector 130 includes two end effectors 130A and 130B that are detachably attached to the links 124A and 124B, respectively. The end effectors 130A and 130B are configured to be able to apply an action to an object handled by the robot 100. The end effectors 130A and 130B are configured to be operable and may include a drive device (not shown). The drive device may be powered by electricity, pneumatic pressure, hydraulic pressure, or the like. The drive device powered by electricity may include a servo motor as an electric actuator. The drive device may be controlled by the robot controller 190.

[0027] The robot 100 further includes an equipment housing 140 on the transport vehicle 110. The secondary battery module 141, the power supply circuit 142, the communication device 143, the information processing device 180, and the robot controller 190 are disposed in the equipment housing 140, but may be disposed at any position on the transport vehicle 110. A table 140a on which an article can be placed is disposed on the equipment housing 140.

[0028] The secondary battery module 141 functions as a power source for the robot 100. The secondary battery module 141 includes at least one secondary battery. A secondary battery is a battery that can charge and discharge power. Examples of secondary batteries include a lead-acid battery, a lithium-ion secondary battery, an all-solid-state battery, a nickel-metal hydride battery, and a nickel-cadmium battery.

[0029] The power supply circuit 142 is a circuit that controls the supply and demand of power to the secondary battery module 141. The power supply circuit 142 is configured to control the supply and demand of power in accordance with commands from the information processing device 180 and the robot controller 190. For example, the power supply circuit 142 may include devices such as a converter, an inverter, a transformer, and an amplifier. The power supply circuit 142 is connected to an external power source such as a commercial power source, and is configured to supply power supplied from the external power source to the secondary battery module 141, store the power, and supply the power stored in the secondary battery module 141 to components in the robot 100 that consume power.

[0030] The communication device 143 is a device for wireless communication, and is configured to be able to connect to the communication network N via wireless communication. The wireless communication used by the communication device 143 is not particularly limited, and may be, for example, wireless communication using mobile data communication, wireless LAN such as wireless Wi-Fi (Wireless Fidelity), short-range wireless communication such as Bluetooth (registered trademark) and ZigBee (registered trademark), or a combination of two or more of these. The communication device 143 has equipment compatible with the wireless communication used.

[0031] The display device 148 includes a display capable of displaying an image. The display may be configured to allow input by the user P, and may be, for example, a touch panel. The display device 148 can display an image of image data sent from the information processing device 180 or the like. The display device 148 can display an image to the user P facing the robot 100.

[0032] The sound collecting device 147 includes a microphone capable of acquiring sound from the surroundings and outputting a sound signal of the sound. The sound collecting device 147 is configured to output the sound signal to the information processing device 180 or the like. The information processing device 180 may be configured to convert the sound signal into sound data and transmit it to the operation terminal 200, or may be configured to perform voice recognition from the sound signal. The sound collecting device 147 can constitute an external sensor of the robot 100 capable of sensing the outside of the robot 100. The internal sensor of the robot 100 can sense the inside of the robot 100, and an example of the internal sensor is a rotation sensor provided on the servo motors of the arm driving devices M1A to M4A and M1B to M4B, etc.

[0033] The audio output device 149 includes a speaker that can convert an audio signal into a sound wave and emit the sound as a voice. The audio output device 149 can output a sound corresponding to an audio signal sent from the information processing device 180 or the like. The audio output device 149 can output a sound to a user P facing the robot 100.

[0034] The imaging devices 144, 145, and 146 each include a camera that captures a digital image, and are configured to send data of the captured image to the information processing device 180. The information processing device 180 may be configured to process image data captured by the imaging devices 144, 145, and 146 into data that can be transmitted over a network, and to send the data to the operation terminal 200 via the communication network N. The information processing device 180 may be configured to perform image processing on the image data and use the image data for other processing. The camera may be a camera that can capture an image for detecting the three-dimensional position of the subject relative to the camera, such as the distance to the subject. For example, the camera may have a configuration such as a stereo camera, a monocular camera, a Time-of-Flight-Camera (TOF camera), a pattern light projection camera such as stripe projection, or a camera using a light section method. The imaging devices 144, 145, and 146 can constitute an external sensor.

[0035] The imaging device 144 is disposed at the tip of at least one of the robot arms 120A and 120B. In this embodiment, the imaging device 144 is disposed in the end effector 130A of the robot arm 120A, but is not limited thereto. The imaging device 145 is disposed in the display device 148. The imaging device 145 can capture an image of the user P to whom a service is to be provided, who faces the robot 100. The imaging device 145 may be configured to be able to change the imaging direction and / or have a wide-angle imaging field of view so that it can capture images of the object on which the end effectors 130A and 130B act, as well as the end effectors 130A and 130B. The imaging device 146 is disposed in the transport vehicle 110 facing the forward direction D1A.

[0036] The robot controller 190 is configured to control the operations of the robot arms 120A and 120B, the end effectors 130A and 130B, and the transport vehicle 110. For example, the robot controller 190 is configured to control the operations of the arm driving devices M1A to M4A and M1B to M4B of the robot arms 120A and 120B, the driving devices (not shown) of the end effectors 130A and 130B, and the transport driving device 113 of the transport vehicle 110. The information processing device 180 is connected to the robot controller 190 via wired communication, wireless communication, or a combination thereof, and is configured to perform arithmetic processing related to the control of the robot arms 120A and 120B, the end effectors 130A and 130B, and the transport vehicle 110.

[0037] The information processing device 180 and the robot controller 190 include computer devices. The configuration of the information processing device 180 is not particularly limited, and may be, for example, an electronic circuit board, an electronic control unit, a microcomputer, a personal computer, a workstation, a smart device such as a smartphone and a tablet, and other electronic devices. The information processing device 180 is connected to the terminal computer 202 of the operation terminal 200 so as to be able to perform data communication through the communication device 143, the communication network N, and the communication device 204. The configuration of the robot controller 190 is not particularly limited, and may be, for example, an electronic circuit board, an electronic control unit, a microcomputer, and other electronic devices. The robot controller 190 may include a drive circuit for controlling power supplied to the arm drive devices M1A to M4A and M1B to M4B, the drive devices (not shown) of the end effectors 130A and 130B, and the transport drive device 113.

[0038] In this embodiment, when the robot 100 is caused to perform a task, the information processing device 180 and the robot controller 190 are configured to control the operations of the control target elements such as the robot arms 120A and 120B, the end effectors 130A and 130B, and the carrier vehicle 110 by manual driving control, but are not limited thereto. The information processing device 180 and the robot controller 190 may be configured to control the operations of at least one of the above control target elements by automatic driving control or a combination of automatic driving control and manual driving control. For example, the combination of automatic driving control and manual driving control may be configured so that automatic driving control and manual driving control are assigned to each control target element, and a part of the task of the robot 100 may be performed by automatic driving control and the other part of the task may be performed by manual driving control.

[0039] For example, manual driving control may be control in which the controlled element is operated in accordance with the operation contents input by the operator PO to the operation input device 201 of the operation terminal 200. For example, in manual driving control, the controlled element may execute an operation in accordance with the operation of the operator PO who operates the operation input device 201. Automatic driving control may be control in which the controlled element is operated autonomously in accordance with a control program.

[0040] [Control device hardware configuration] With reference to FIG. 3, the hardware configuration of the control device 170 according to the embodiment will be described. FIG. 3 is a block diagram showing an example of the hardware configuration of the control device 170 according to the embodiment. The information processing device 180 includes a processor 1801, a memory 1802, a storage 1803, and input / output I / Fs (Interfaces) 1804a to 1804f as components. The components of the information processing device 180 are connected to each other by a bus 1805, but may be connected by any other wired or wireless communication. The robot controller 190 includes a processor 1901, a memory 1902, an input / output I / F 1903, and a drive I / F 1904 as components. The robot controller 190 may include a storage. The components of the robot controller 190 are connected to each other by a bus 1905, but may be connected by any other wired or wireless communication. Not all of the components included in the information processing device 180 and the robot controller 190 are essential.

[0041] A pair of the processor 1801 and the memory 1802, and a pair of the processor 1901 and the memory 1902 each constitute a computing unit. The computing unit transmits and receives commands, information, data, etc. to and from other devices. The computing unit inputs signals from various devices and outputs control signals to each controlled element.

[0042] The memories 1802 and 1902 store programs executed by the processors 1801 and 1901, various fixed data, and the like, respectively. The memories 1802 and 1902 may be configured with storage devices such as semiconductor memories, such as volatile memories and nonvolatile memories. Although not limited thereto, in this embodiment, the memories 1802 and 1902 include RAM (Random Access Memory), which is a volatile memory, and ROM (Read-Only Memory), which is a nonvolatile memory.

[0043] The storage 1803 stores various data. The storage 1803 may be configured with a storage device such as a semiconductor memory, a hard disk drive (HDD: Hard Disk Drive), or a solid state drive (SSD: Solid State Drive).

[0044] Both the processors 1801 and 1901, together with the RAM and ROM, form a computer system. The computer system of the information processing device 180 may realize the functions of the information processing device 180 by the processor 1801 using the RAM as a work area to execute a program recorded in the ROM. The computer system of the robot controller 190 may realize the functions of the robot controller 190 by the processor 1901 using the RAM as a work area to execute a program recorded in the ROM.

[0045] Some or all of the functions of the information processing device 180 and the robot controller 190 may be realized by the above-mentioned computer system, by a dedicated hardware circuit such as an electronic circuit or an integrated circuit, or by a combination of the above-mentioned computer system and hardware circuit. The information processing device 180 and the robot controller 190 may each be configured to execute each process by centralized control of a single device, or may be configured to execute each process by distributed control through cooperation of a plurality of devices. The information processing device 180 and the robot controller 190 may be configured to include at least a portion of each other's functions, or may be integrated.

[0046] For example, but not limited to, the processors 1801 and 1901 include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), a microprocessor, a processor core, a multiprocessor, an ASIC (Application-Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), etc., and each process may be realized by a logic circuit or a dedicated circuit formed in an IC (Integrated Circuit) chip, an LSI (Large Scale Integration), etc. A plurality of processes may be realized by one or more integrated circuits, or may be realized by a single integrated circuit.

[0047] The first input / output I / F 1804a of the information processing device 180 connects the information processing device 180 and the robot controller 190, and enables input / output of information, commands, data, and the like therebetween. The second input / output I / F 1804b connects the information processing device 180 and the communication device 143, and enables input / output of information, commands, data, and the like therebetween. The third input / output I / F 1804c connects the information processing device 180 and the imaging devices 144 to 146, and enables input / output of information, commands, data, and the like therebetween. The fourth input / output I / F 1804d connects the information processing device 180 and the sound collecting device 147, and enables input / output of information, commands, data, and the like therebetween. The fifth input / output I / F 1804e connects the information processing device 180 and the display device 148, and enables input / output of information, commands, data, and the like therebetween. The sixth input / output I / F 1804f connects the information processing device 180 and the audio output device 149, and enables input / output of information, commands, data, and the like therebetween.

[0048] The input / output I / F 1903 of the robot controller 190 connects the robot controller 190 and the first input / output I / F 1804a of the information processing device 180, enabling input / output of information, commands, data, and the like therebetween. The drive I / F 1904 connects the robot controller 190 and the drive circuit 142a, enabling transmission / reception of signals, and the like therebetween. The drive circuit 142a is configured to control the power supplied to the arm drive devices M1A to M4A and M1B to M4B, the drive devices (not shown) of the end effectors 130A and 130B, and the conveyance drive device 113 according to a command value included in a signal received from the robot controller 190. For example, the drive circuit 142a can drive each drive device in cooperation with each other. Although not limited thereto, in this embodiment, the drive circuit 142a is formed as a part of the power supply circuit 142.

[0049] The robot controller 190 may be configured to servo-control the servo motors of the respective drive devices. The robot controller 190 receives, as feedback information, from the drive circuit 142a, the detection value of the rotation sensor provided in each servo motor and a command value of the current from the drive circuit 142a to the servo motor. The robot controller 190 uses the feedback information to determine a command value for driving the servo motor and transmits it to the drive circuit 142a.

[0050] Further, since a general-purpose device can be used for the operation terminal 200, detailed description of the hardware configuration of the operation terminal 200 will be omitted. The terminal computer 202 of the operation terminal 200 includes a processor and a memory (both not shown) similar to the information processing device 180, etc. The terminal computer 202 may also include input / output I / Fs (not shown) for establishing a connection between the terminal computer 202 and the operation input device 201, a connection between the terminal computer 202 and the communication device 204, and a connection between the terminal computer 202 and the presentation device 203.

[0051] [Functional configuration of the control device] With reference to FIG. 4, the functional configuration of the control device 170 according to the embodiment will be described. FIG. 4 is a block diagram showing an example of the functional configuration of the control device 170 according to the embodiment. The information processing device 180 includes an operation command unit 180a, a detection processing unit 180b, an evaluation processing unit 180c, a compensation determination unit 180d, a presentation processing unit 180e, an accumulation processing unit 180f, a criterion determination unit 180g, a timekeeping unit 180h, and memory units 181a and 181b as functional components. The functions of the functional components other than the memory units 181a and 181b are realized by a processor 1801 or the like, and the functions of the memory units 181a and 181b are realized by a storage device such as a memory 1802 and / or a storage 1803. The functions of the storage units 181a and 181b may be realized by separate storage devices or may be realized by one storage device. All of the above functional components are not essential. The robot controller 190 includes drive command units 190a to 190e as functional components. The functions of the drive command units 190a to 190e are realized by a processor 1901, etc. All of the above functional components are not essential.

[0052] The first memory unit 181a stores motion-related reference data (represented as "reference data" in FIG. 4) for various tasks. The motion-related reference data includes information on motion-related references, which are references related to the motion of the robot 100. The motion-related references include references on motion-related information, which is information related to the motion of the robot 100. The motion-related information may include information on motion-related elements such as the motion of the robot 100, the state of the environment surrounding the robot 100, the state of an object on which the robot 100 acts, and the state of the environment surrounding the object.

[0053] For example, the action-related criteria may be criteria representing a minimum required level of the action-related element, a standard level of the action-related element, a good level of the action-related element, an ideal level of the action-related element, etc. The action-related criteria may include criteria for an evaluation target feature, which is a feature for evaluating the action-related element.

[0054] For example, one task is composed of a series of multiple motions of the robot 100. The motion-related criteria for the motions of the robot 100 may include criteria for a series of multiple motions forming the task of each of the robot arms 120A and 120B, the end effectors 130A and 130B, and the transport vehicle 110. For example, the evaluation target features of the motion-related criteria for the motions of the robot 100 may include features such as the time required for the motion, the order of the motion, the position of the robot 100 and each part thereof in the motion, the posture of the robot 100 and each part thereof in the motion, the trajectory of the robot 100 and each part thereof in the motion, the speed of the above positions in the motion, the speed of the above posture in the motion, the acceleration of the above positions in the motion, the acceleration of the above posture in the motion, and the work performance of the robot 100 in the motion.

[0055] The state of the surrounding environment of the robot 100 may include the state of the robot peripheral elements, which are elements such as devices, facilities, and equipment that form the surrounding environment for the robot 100 to perform work. Examples of the robot peripheral elements are devices, facilities, and equipment that are arranged in the work area (service provision area AS, etc.) of the robot 100, devices, facilities, and equipment that cooperate with the robot 100, and sensors such as imaging devices that are arranged in the work area (service provision area AS, etc.). The evaluation target features of the operation-related criteria of the robot peripheral elements may include the force and impact that the robot 100 applies to the robot peripheral elements, changes in position, posture, shape, and state that the robot 100 applies to the robot peripheral elements, and the position, posture, position speed, posture speed, position acceleration, and posture acceleration of the robot 100 and each part thereof relative to the robot peripheral elements. For example, in FIG. 1, the robot peripheral elements of the robot 100 are other robots 100, a storage shelf SR in which an object W is stored, and users other than the user P to whom the robot 100 provides a service.

[0056] The state of the object may include the state of an object handled by the robot 100 using the end effectors 130A and 103B, etc. The evaluation target features of the object's motion-related criteria may include features such as the object's position, posture, position velocity, posture velocity, position acceleration, and posture acceleration, changes in the object's shape, changes in the object's state, and the workmanship of the object. For example, in FIG. 1, the object is an article W.

[0057] The state of the surrounding environment of the object may include the state of the object peripheral elements, which are elements of devices, facilities, and equipment that handle the object other than the robot 100. Examples of the object peripheral elements are devices, facilities, and equipment that support, hold, store, load, mount, transport, etc. the object, devices, facilities, and equipment to which the object is assembled, and devices, facilities, and equipment that process the object. The evaluation target features of the operation-related criterion of the object peripheral elements may include features such as the force and impact that the robot 100 applies to the object peripheral elements, the changes in position, posture, shape, and state that the robot 100 applies to the object peripheral elements, and the position, posture, position speed, posture speed, position acceleration, and posture acceleration of the robot 100 and each part thereof relative to the object peripheral elements. For example, in FIG. 1, the object peripheral elements are the shelves (not shown) of the storage shelf SR, other items in the storage shelf SR, and the user P.

[0058] The second storage unit 181b stores accumulated data, which is data accumulated by the accumulation processing unit 180f. The accumulated data includes information on the evaluation result determined by the evaluation processing unit 180c and information on the action-related criterion used to determine the evaluation result, in association with each other. That is, in the accumulated data, the action-related information, the information on the action-related criterion of the action-related information, and the information on the evaluation result based on the action-related criterion are associated with each other. The second storage unit 181b is an example of a storage unit.

[0059] During manual driving control, the operation command unit 180a generates operation commands for operating the robot arms 120A and 120B, the end effectors 130A and 130B, and the transport vehicle 110 in accordance with operation commands received from the operation terminal 200, and outputs the operation commands to the robot controller 190. During automatic driving control, the operation command unit 180a generates operation commands for operating the robot arms 120A and 120B, the end effectors 130A and 130B, and the transport vehicle 110 in accordance with a control program, and outputs the operation commands to the robot controller 190.

[0060] The operation command is a command indicating the content of the operation input by the operator PO to the operation input device 201. The operation command includes a command for making the robot 100 operate under manual driving control according to the content of the operation. The terminal computer 202 receives a signal indicating the content of the operation from the operation input device 201, converts the signal into an operation command, and transmits the operation command to the information processing device 180.

[0061] The operation command unit 180a generates a first operation command for the robot arm 120A, a second operation command for the robot arm 120B, a third operation command for the end effector 130A, a fourth operation command for the end effector 130B, and a fifth operation command for the transport vehicle 110. For example, the third operation command and the fourth operation command may be generated when the end effectors 130A and 130B are equipped with driving devices. The first operation command and the second operation command may include commands for the target positions of the parts of the robot arms 120A and 120B and commands for the target forces generated by the parts. The third operation command and the fourth operation command may include commands for the target states of the end effectors 130A and 130B. The fifth operation command may include a command for the target position of the transport vehicle 110.

[0062] The target position command may include commands for a target position, a target moving speed of the position, a target attitude, and a target moving speed of the attitude. The target force command may include commands for a magnitude and a direction of a target force. The target force command may include a target acceleration command.

[0063] The timer unit 180h measures the elapsed time and outputs the time measurement result to the detection processing unit 180b, the evaluation processing unit 180c, the consideration determination unit 180d, the presentation processing unit 180e, etc. The time measurement result can be used for processing in the detection processing unit 180b, the evaluation processing unit 180c, the consideration determination unit 180d, the presentation processing unit 180e, etc. For example, the function of the timer unit 180h may be realized by a clock or the like mounted on the computer device of the information processing device 180.

[0064] The detection processing unit 180b executes a detection process for detecting the operation-related information of the robot 100 operating according to an operation command while the robot 100 is performing a task. The detection processing unit 180b outputs the detection result to the evaluation processing unit 180c. In this embodiment, the detection processing unit 180b is configured to output the detection result to the evaluation processing unit 180c without providing a delay time after detecting the operation-related information, but is not limited to this and a delay time may be provided.

[0065] The detection processing unit 180b detects information on a motion-related element as motion-related information during the execution of a task. The detection processing unit 180b may be configured to detect the information on the motion-related element by any method.

[0066] For example, the detection processing unit 180b detects the motion of the robot 100 as the motion-related element, specifically, the motion of the robot arms 120A and 120B, the end effectors 130A and 130B, and the carrier 110. For example, the detection processing unit 180b may be configured to process an operation command and detect a target motion of the robot 100 indicated by the operation command as the motion-related element, or may be configured to process a motion command and detect a target motion of the robot 100 indicated by the motion command as the motion-related element. The detection processing unit 180b may be configured to obtain and process a drive command from the drive command units 190a to 190e of the robot controller 190, and detect a target motion of the robot 100 indicated by the drive command as the motion-related element. The detection processing unit 180b may be configured to obtain and process feedback information from the drive command units 190a to 190e of the robot controller 190, and detect a motion of the robot 100 indicated by the feedback information as the motion-related element. The detection processing unit 180b may be configured to acquire captured image data from the imaging devices 144, 145, and 146 and an external imaging device (not shown) of the robot 100, process the images, and detect the state of the robot 100 indicated by the processed image data as an operation-related element.

[0067] For example, the detection processing unit 180b detects the state of the robot peripheral elements constituting the surrounding environment of the robot 100 as the operation-related elements. For example, the detection processing unit 180b detects the state of the robot peripheral elements using the detection results received from the external sensors of the robot 100 and the information received from the robot peripheral elements. Examples of the external sensors include force sensors (not shown) arranged between the end effectors 130A and 103B and the robot arms 120A and 120B, the imaging devices 144 to 146, the sound collecting device 147, and sensors (not shown) such as an imaging device arranged in the service provision area AS. Examples of the information received from the robot peripheral elements include the state, vibration, impact, position, attitude, speed of position, speed of attitude, etc. of the robot peripheral elements.

[0068] For example, the detection processing unit 180b detects the state of an action target, which is an object on which the robot 100 applies an action, as a motion-related element. For example, the detection processing unit 180b detects the state of the action target using detection results received from external sensors of the robot 100 and information received from the action target. Examples of the external sensors are the force sensors of the robot arms 120A and 120B, the imaging devices 144 to 146, the sound collection device 147, and sensors arranged in the service provision area AS. Examples of information received from the action target are detection results of sensors equipped in the action target, as well as the state, vibration, impact, position, attitude, velocity of position, velocity of attitude, acceleration of position, acceleration of attitude, and the like of the action target.

[0069] For example, the detection processing unit 180b detects the state of the object peripheral elements constituting the surrounding environment of the action object as the operation-related elements. For example, the detection processing unit 180b detects the state of the object peripheral elements using the detection results received from the external sensors of the robot 100 and information received from the object peripheral elements. Examples of the external sensors are the force sensors of the robot arms 120A and 120B, the imaging devices 144 to 146, the sound collecting device 147, and sensors arranged in the service provision area AS. Examples of the information received from the object peripheral elements are the state, vibration, impact, position, attitude, speed of position, and speed of attitude of the object peripheral elements.

[0070] The detection processing unit 180b may be configured to detect the motion-related element at any timing. For example, the detection processing unit 180b may be configured to detect the motion-related element at a predetermined timing, such as a timing at a predetermined time interval, a timing at which a predetermined motion of the robot 100 is performed, and a timing at which a predetermined work stage is performed. Although not limited to this, in the present embodiment, the detection processing unit 180b executes the detection process at a predetermined time interval.

[0071] In the following, the motion of the robot 100 that is the target of processing by the functional components of the information processing device 180 other than the detection processing unit 180b is referred to as the "motion to be processed." For example, a predetermined motion of the robot 100 is an example of the motion to be processed. The unit of the motion to be processed may be set arbitrarily. For example, the division of one unit of the motion to be processed may be set by a time division such as a period during which the motion is executed, a division of a motion change such as a timing at which the motion changes, a specified division, other divisions, or a combination of two or more of these.

[0072] For example, when the robot 100 executes a series of operations to grasp an object placed at point A with the end effector 130A and place it on the table 140a, one unit of the operation to be processed may be set to include all of the series of operations to be executed, or may be set to include a part of the series of operations. For example, one unit of the operation to be processed may be set to include at least one of four operations: a movement operation of the robot arm 120A to move the end effector 130A to the object, a gripping operation of the end effector 130A to grip the object, a movement operation of the robot arm 120A to move the end effector 130A onto the table 140a and place the object, and a grip release operation of the end effector 130A to release the grip of the object. For example, one operation to be processed may be set for the four operations, or two or more operations to be processed may be set.

[0073] A work stage may be set by dividing a series of actions included in a work. The unit of a work stage may be set arbitrarily. For example, the division of one unit of a work stage may be set by a time division such as a period during which the work is performed, a division of an action change such as a timing at which an action included in the work is switched, a specified division, other divisions, or a combination of two or more of these. For example, when the robot 100 receives a command from a user P requesting an object and performs a task of transporting the object placed at point A and handing it over to the user P, one unit of a work stage may be set to include a part of all actions performed during the work.

[0074] For example, one unit of work stage may be set to include at least one of four steps: a step of the robot 100 moving from a position near the user P to a position near the point A, a step of the robot 100 gripping an object at the point A with the end effector 130A and placing it on the robot 100, a step of the robot 100 moving from the position near the point A to a position near the user P, and a step of the robot 100 handing over the object on the robot 100 to the user P with the end effector 130A. For example, one work stage may be set for the four steps, or two or more work stages may be set.

[0075] The evaluation processing unit 180c executes an evaluation process for comparing and evaluating the motion-related information of the robot 100 detected by the detection processing unit 180b with the motion-related criterion of the motion-related information while the robot 100 is performing a task. Specifically, the evaluation processing unit 180c compares information of a motion-related element included in the motion-related information with the motion-related criterion of the motion-related element to determine an evaluation result of the motion-related element, and determines an evaluation result of the motion-related information based on the evaluation result of the motion-related element.

[0076] The motion-related information of the evaluation processing unit 180c is motion-related information corresponding to the motion to be evaluated among the motions to be processed included in the work. Hereinafter, the motion to be evaluated is referred to as the "motion to be evaluated". The motion to be evaluated is at least one motion to be processed among all the motions to be processed included in the work. All of the motions to be processed may be the motion to be evaluated, or a part of all of the motions to be processed may be the motion to be evaluated. The motions to be evaluated may be consecutive or discontinuous.

[0077] For example, the evaluation processing unit 180c may determine the evaluation result of the action-related element and / or the evaluation target feature of the action-related information corresponding to the action to be evaluated based on the deviation amount, deviation direction, deviation frequency, etc. of the action-related element and / or the evaluation target feature of the action-related element from the action-related criterion. Furthermore, the evaluation processing unit 180c may determine the evaluation result of the action-related information based on the evaluation result of the action-related element and / or the evaluation target feature.

[0078] The evaluation processing unit 180c executes the evaluation process at least at one of the timings of each evaluation target action and each evaluation target stage, which is a task stage to be evaluated in the task. All task stages included in the task may be the evaluation target stages, or a part of all the task stages may be the evaluation target stages. Furthermore, action-related information corresponding to all the processed action included in the evaluation target stage may be the evaluation target for the evaluation of the evaluation target stage, or action-related information corresponding to a part of the processed action of all the processed action may be the evaluation target for the evaluation of the evaluation target stage.

[0079] In the evaluation process at each timing of the evaluation target action, the evaluation processing unit 180c reads out the action-related reference data of the evaluation target action from the first storage unit 181a, compares the action-related reference data with the action-related information corresponding to the evaluation target action, and determines the evaluation result of the action-related information. For example, the evaluation processing unit 180c may determine the evaluation level of the action-related information. For example, the evaluation processing unit 180c may compare the criteria of the evaluation target feature included in the action-related reference data with the evaluation target feature of the action-related element of the action-related information, and determine the evaluation level of each evaluation target feature. The evaluation processing unit 180c may determine the evaluation level of the action-related element and / or the evaluation level of the action-related information based on the evaluation level of each evaluation target feature.

[0080] Furthermore, the evaluation processing unit 180c, for example, executes the evaluation process of the action-related information corresponding to the first evaluated action by the timing of the second evaluated action, which is the next evaluated action of the first evaluated action. The evaluation processing unit 180c executes and completes the evaluation process of the action-related information corresponding to the first evaluated action by the timing of the second evaluated action. For example, the timing of the second evaluated action may be a reference timing of the second evaluated action included in the action-related reference data, a detection timing of the action-related information corresponding to the second evaluated action, or a combination of these. The reference timing and the detection timing may be the start timing of the action, the completion timing of the action, or a predetermined timing between these.

[0081] The evaluation processing unit 180c may be configured to execute the evaluation process of the action-related information corresponding to the first action to be evaluated by the timing of the next action to be processed of the first action to be evaluated. For example, the timing of the next action to be processed may be the completion timing of the actual action of the robot 100 corresponding to the action to be processed, or a predetermined timing between the start timing and the completion timing. Although not limited thereto, in this embodiment, the evaluation processing unit 180c starts the evaluation process without a delay time after receiving the detection result of the action-related information corresponding to the first action to be evaluated from the detection processing unit 180b.

[0082] In the evaluation process at each timing of the evaluation target stage, the evaluation processing unit 180c reads out the action-related reference data of the processing target action of the evaluation target among the processing target actions included in the evaluation target stage from the first storage unit 181a, compares the action-related reference data with the action-related information corresponding to the processing target action of the evaluation target, and determines the evaluation result of the action-related information. The evaluation processing unit 180c integrates the evaluation results of the action-related information corresponding to the processing target actions of all the evaluation targets, and determines the evaluation result of the evaluation target stage based on the integration result. For example, the evaluation processing unit 180c may determine the evaluation level of the evaluation target stage. The integration method may be any method. For example, the integration method may be a method of adding the evaluation results of multiple pieces of action-related information, a method of adding the evaluation results of multiple pieces of action-related information with weighting, or a method of integrating the evaluation results of multiple pieces of action-related information by other statistical processing.

[0083] Furthermore, the evaluation processing unit 180c executes, for example, the evaluation process of the first task stage by the timing of the second evaluation target stage, which is the next evaluation target stage of the first evaluation target stage. The evaluation processing unit 180c executes and completes the evaluation process of the action-related information corresponding to the processed actions of all the evaluation targets included in the first evaluation target stage by the timing of the second evaluation target stage. For example, the timing of the second task stage may be the reference timing of the second evaluation target stage included in the action-related reference data, the detection timing of the second evaluation target stage, or a combination of these. The reference timing and the detection timing may be the start timing of the first processed action of the second evaluation target stage, the completion timing of the last processed action, or a predetermined timing between these.

[0084] The evaluation processing unit 180c may be configured to execute the evaluation process of the first evaluation target stage by the timing of the next work stage of the first evaluation target stage. For example, the timing of the next work stage may be the start timing of the first processing target action of the work stage, the completion timing of the last processing target action, or a predetermined timing between them. Although not limited thereto, in this embodiment, the evaluation processing unit 180c starts the evaluation process without a delay time after receiving from the detection processing unit 180b the detection result of the action-related information corresponding to the processing target action of the last evaluation target among all the processing target actions of the evaluation targets included in the first evaluation target stage.

[0085] The compensation determination unit 180d determines compensation to be given to the operator PO for operating the robot 100 based on the evaluation result of the evaluation processing unit 180c. The compensation determination unit 180d determines a compensation corresponding to the evaluation result of the action-related information each time evaluation processing is performed on the action-related information corresponding to the action to be evaluated. The compensation determination unit 180d determines a compensation corresponding to the evaluation result of the stage to be evaluated each time evaluation processing is performed on the stage to be evaluated. For example, the compensation may be a remuneration amount, points, points, score, etc. given to the operator PO. The remuneration amount may be a remuneration amount in real currency or a remuneration amount in virtual currency.

[0086] Furthermore, the compensation determination unit 180d sequentially accumulates each determined compensation during the execution of the work, and calculates an accumulated result that is the result of accumulating the compensation. Every time the compensation determination unit 180d determines a compensation, it outputs information on the determined compensation and information on the accumulated result including the determined compensation to the presentation processing unit 180e. The compensation determination unit 180d may be configured to calculate two accumulated results by separately accumulating the compensation for the action-related information corresponding to the evaluated action and the compensation for the evaluated stage, or may be configured to integrate and accumulate the two compensations to calculate one accumulated result.

[0087] The compensation determination unit 180d may be configured to determine compensation for the action-related information by adding a weight to the evaluation level of the action-related information corresponding to the action to be evaluated. The compensation determination unit 180d may be configured to determine compensation for the action-related information by adding a weight to the evaluation level of the evaluation target feature of the action-related information and integrating the evaluation levels after the weighting, or may be configured to further add a weight to the determined compensation for the action-related information. For example, the difficulty of the operation differs depending on the action to be evaluated. The compensation determination unit 180d may be configured to change the weight depending on the action to be evaluated. The compensation determination unit 180d may be configured to change the weight depending on the evaluation target feature.

[0088] The compensation determination unit 180d may be configured to determine the compensation by adding a weight to the evaluation level of the evaluation target stage. The compensation determination unit 180d may determine the compensation for the evaluation target stage by adding a weight to the evaluation level of the action-related information corresponding to the processing target action of the evaluation target included in the evaluation target stage and integrating the evaluation levels after the weighting, or may be configured to further add a weight to the determined compensation for the evaluation target stage. The compensation determination unit 180d may be configured to change the weight depending on the evaluation target stage and the processing target action. The integration method used by the compensation determination unit 180d in the above may be any method.

[0089] The presentation processing unit 180e executes a presentation process for having the presentation device 203, which receives the presentation of information while the operator PO operates the operation input device 201, present the evaluation result of the evaluation processing unit 180c during the execution of the work. Although not limited to this, in this embodiment, the presentation processing unit 180e causes the presentation device 203 to present the consideration determined by the consideration determination unit 180d and the accumulated result of the consideration during the execution of the work. The presentation processing unit 180e generates data including the consideration received from the consideration determination unit 180d and the accumulated result, and transmits the data and a presentation command for the data to the operation terminal 200. For example, the presentation processing unit 180e may generate image data, audio data, character string data, or data including at least two of these, indicating the consideration and the accumulated result, and transmit it to the operation terminal 200.

[0090] When the evaluation processing unit 180c executes the evaluation process at the timing of each evaluated action, the presentation processing unit 180e transmits the evaluation results of the action-related information corresponding to the evaluated action to the operation terminal 200 in the same order as the execution order of the evaluated action, and causes the presentation device 203 to present the evaluation results. The evaluation results may include an evaluation level and / or a compensation. For example, the presentation processing unit 180e executes the presentation process of the evaluation results of the action-related information corresponding to the first evaluated action by the timing of the next second evaluated action. The timing of the second evaluated action may be any of the timings exemplified for the second evaluated action of the evaluation processing unit 180c, and may be the same as or different from the timing of the second evaluated action of the evaluation processing unit 180c.

[0091] When the evaluation processing unit 180c executes the evaluation process at the timing of each evaluation target stage, the presentation processing unit 180e transmits the evaluation results of the evaluation target stage to the operation terminal 200 in the same order as the execution order of the evaluation target stages, and causes the presentation device 203 to present them. The evaluation results may include an evaluation level and / or a compensation. For example, the presentation processing unit 180e executes the presentation process of the evaluation result of the first evaluation target stage by the timing of the next second evaluation target stage. The timing of the second evaluation target stage may be any of the timings exemplified for the timing of the second evaluation work stage of the evaluation processing unit 180c, and may be the same as or different from the timing of the second evaluation work stage of the evaluation processing unit 180c.

[0092] The accumulation processing unit 180f associates the action-related reference data, information on the evaluation result of the action-related information based on the action-related reference data, and the performance date and time of the action of the robot 100 corresponding to the action-related information, and stores them as accumulated data in the second storage unit 181b. The accumulation processing unit 180f stores the accumulated data on the evaluation target action and the accumulated data on the evaluation target stage in the second storage unit 181b.

[0093] For example, the accumulation processing unit 180f may store, as accumulated data related to the evaluated action, information on the evaluated action, information on the evaluation result of the action-related information corresponding to the evaluated action, and the action-related reference data used in the evaluation process of the evaluation result in the second storage unit 181b in association with each other. The information on the evaluation result may include an evaluation level and / or a compensation.

[0094] For example, the accumulation processing unit 180f may store, as accumulated data related to the evaluation target stage, information on the evaluation target action included in the evaluation target stage, information on the evaluation result of the action-related information corresponding to the processing target action, information on the evaluation result of the evaluation target stage, and at least some of the action-related reference data used in the evaluation process of the evaluation result, in the second storage unit 181b in association with each other. The information on the evaluation result may include an evaluation level and / or a compensation.

[0095] The criterion determining unit 180g determines new action-related reference data using the accumulated data in the second storage unit 181b, and stores the action-related reference data in the first storage unit 181a. The criterion determining unit 180g determines new second action-related reference data using the first action-related reference data included in the accumulated data in the second storage unit 181b and an evaluation result based on the first action-related reference data. The evaluation result based on the first action-related reference data is an evaluation result determined using the first action-related reference data.

[0096] For example, the criterion determining unit 180g can receive a designated condition for determining the second action-related reference data. For example, the criterion determining unit 180g receives the designated condition from the operation terminal 200. For example, the designated condition may be a condition for limiting the first action-related reference data and the evaluation result used for determining the second action-related reference data. The criterion determining unit 180g may extract, from the accumulated data in the second storage unit 181b, information on the evaluation result that satisfies the designated condition and the first action-related reference data that is the action-related reference data used for the evaluation result, and determine the second action-related reference data using the information on the evaluation result and the first action-related reference data.

[0097] For example, the specified condition may be a condition that limits an attribute of the evaluation result for which the first action-related reference data is used. The attribute of the evaluation result may be a user that the evaluation result is a target of, a robot that the evaluation result is a target of, a task that the evaluation result is a target of, a work place that the evaluation result is a target of, a work environment that the evaluation result is a target of, a task execution date that the evaluation result is a target of, and a task execution period that the evaluation result is a target of, etc. The specified condition may be a condition that limits at least one attribute of the evaluation result.

[0098] For example, the criterion determination unit 180g can determine second action-related reference data corresponding to a specific user using the evaluation result of the user and the first action-related reference data. For example, if the evaluation level of the user is low, the criterion determination unit 180g may generate second action-related reference data with a lower standard than the first action-related reference data. When the user operates the robot 100, the user can issue a command to the information processing device 180 via the operation terminal 200 to request an evaluation using the second action-related reference data. This allows the user to receive a higher evaluation from the information processing device 180, and can improve the user's motivation to operate the robot.

[0099] The first drive command unit 190a generates drive commands for the arm drive devices M1A to M4A of the robot arm 120A to drive the joints (not shown) to the state of the target position command and the target force command included in the first operation command, and outputs the drive commands to the drive circuit 142a. The drive commands include command values ​​for the currents of the servo motors. The first drive command unit 190a generates the drive commands using feedback information of the servo motors.

[0100] The second drive command unit 190b generates drive commands for the arm drive devices M1B to M4B of the robot arm 120B to drive the joints (not shown) to the state of the target position command and the target force command included in the second operation command, and outputs the drive commands to the drive circuit 142a. The drive commands include command values ​​for the currents of the servo motors. The second drive command unit 190b generates the drive commands using feedback information of the servo motors.

[0101] The third drive command unit 190c generates a drive command for driving a drive device (not shown) of the end effector 130A to the target state included in the third operation command, and outputs the drive command to the drive circuit 142a.

[0102] The fourth drive command unit 190d generates a drive command for driving a drive device (not shown) of the end effector 130B to the target state included in the fourth operation command, and outputs the drive command to the drive circuit 142a.

[0103] The fifth drive command unit 190e generates a drive command for causing the transportation drive device 113 of the transportation vehicle 110 to drive the drive wheels 111 to the state of the target position command included in the fifth operation command, and outputs the drive command to the drive circuit 142a. The drive command includes a command value for the current of the servo motor. The fifth drive command unit 190e generates the drive command using feedback information of the servo motor.

[0104] [Robot system operation] An example of the operation of the robot system 1 according to the embodiment will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a flowchart showing an example of the operation of the robot system 1 according to the embodiment. Fig. 6 is a plan view showing an example of the operation of the robot system 1 according to the embodiment. An example of executing a service in which the robot 100 searches for an item W requested by a user P from the storage shelf SR and hands it over to the user P will be described below. In the examples of Figs. 5 and 6, the information processing device 180 executes an evaluation process and a presentation process for each evaluation target action and each evaluation target stage, respectively.

[0105] First, the operator PO inputs a request to operate the robot into the operation terminal 200, and the operation terminal 200 transmits the request to the server 300 (step S101). The server 300 searches for a robot 100 that the operator PO can operate, and connects the information processing device 180 of the searched robot 100 to the operation terminal 200 via the communication network N (step S102).

[0106] When the operator PO receives a notification of connection completion from the server 300, he / she inputs a command to start executing the robot operation into the operation terminal 200. The operation terminal 200 transmits the command to the information processing device 180. The information processing device 180 starts the robot 100 so that it can operate according to the operation command received from the operation terminal 200. The information processing device 180 sets the execution flag of the work for servicing the robot 100 to the ON state, that is, starts the work (step S103).

[0107] Next, the operator PO inputs a designation condition for the action-related reference data to be used in the evaluation of the robot operation to the operation terminal 200 (step S104). The operation terminal 200 transmits a command for the designation condition to the information processing device 180. For example, the designation condition may be a condition for designating standard action-related reference data or a condition for designating an attribute of the evaluation result to be used. In this example, the operator PO inputs a designation condition for designating the use of the evaluation result for the operator PO.

[0108] Next, the information processing device 180 generates new action-related reference data using the specified conditions received from the operation terminal 200 and the accumulated data accumulated in the information processing device 180, and determines to use the action-related reference data (step S105). The action-related reference data is action-related reference data adjusted according to the evaluation result of the operator PO.

[0109] Next, the operator PO searches for a user P requesting a service within the service provision area AS while checking the screen of the presentation device 203 that displays the captured images, etc., of the imaging device 145. When the operator PO finds the user P, the operator PO operates the operation terminal 200 while visually checking the screen of the presentation device 203 that displays the captured images, etc., of the imaging devices 145 and / or 146, to make the robot 100 travel to a position near the user P (step S106).

[0110] The information processing device 180 detects the motion-related information of the robot 100 at a predetermined time interval while the robot 100 is running, and executes evaluation processing and presentation processing of the motion-related information each time the information is detected (step S107). In this example, each of the motion-related information at the predetermined time interval corresponds to a motion to be evaluated. The information processing device 180 displays the compensation for the motion-related information and the accumulated compensation result on the presentation device 203. For example, the information processing device 180 evaluates the evaluation target features such as the running speed, running acceleration, running trajectory, and interference with the robot peripheral elements of the robot for each motion-related information. In this example, the information processing device 180 executes the detection processing, evaluation processing, and presentation processing without any delay between them, so that the operator PO can check the compensation for the motion to be evaluated and the accumulated compensation result without any delay from the time when the operation for the motion to be evaluated is input to the operation terminal 200, that is, in real time (real time).

[0111] When the robot 100 arrives at a position near the user P, the information processing device 180 executes an evaluation process and a presentation process of the first work stage, which is the traveling process from the starting position to the nearby position, and causes the presentation device 203 to display the price and estimated cost of the first work stage (step S108).

[0112] When the operator PO confirms that the robot 100 has arrived at a position near the user P via the screen of the presentation device 203, he / she inquires of the user P about the desired item W via input to the operation terminal 200 and output from the display device 148 and / or audio output device 149 of the robot 100 (step S109).

[0113] Next, when the user P inputs information on the requested item W1 to the touch panel of the sound collecting device 147 and / or the display device 148, the operator PO requests information on the position of the requested item W1 in the storage shelf SR from the information processing device 180 via input to the operation terminal 200. The information processing device 180 may store information on the service providing area AS in which the robot 100 is placed and information on the provided services, in addition to information on the robot 100 on which the information processing device 180 is mounted. The operation terminal 200 receives information on the position PW1 of the requested item W1 from the information processing device 180 and presents it to the operator PO (step S110). Note that the server 300 may hold information on the service providing area AS and information on the provided services, and the operation terminal 200 may request information on the position of the requested item W1 from the server 300.

[0114] Next, the operator PO operates the operation terminal 200 while visually checking the screen of the presentation device 203, which displays information about the position PW1, images captured by the imaging devices 145 and / or 146, etc., and causes the robot 100 to travel to a position near the position PW1 (step S111).

[0115] Similarly to the process of step S107, the information processing device 180 detects the motion-related information of the robot 100 at predetermined time intervals while the robot 100 is running, and performs evaluation processing and presentation processing of the motion-related information each time the motion-related information is detected (step S112).

[0116] Similar to the processing of step S108, when the robot 100 arrives at a position near the position PW1, the information processing device 180 executes evaluation processing and presentation processing of the second work stage, which is the traveling process from a position near the user P to a position near the position PW1 (step S113).

[0117] When the operator PO confirms via the screen of the presentation device 203 that the robot 100 has arrived at a position near the position PW1, he / she inputs an operation to the operation terminal 200 for operating the robot arms 120A and 120B and the end effectors 130A and 130B (step S114). This operation is an operation for taking out the requested item W1 stored in the storage shelf SR from the storage shelf SR using the end effectors 130A and 130B, and placing the requested item W1 on the table 140a of the robot 100. The operator PO inputs an operation to the operation terminal 200 while visually checking the screen of the presentation device 203 that displays the captured images of the imaging devices 144 and 145 and the like.

[0118] The information processing device 180 detects motion-related information of the robot 100 during the motion of the robot 100, and executes evaluation processing and presentation processing of the motion-related information every time motion-related information corresponding to the motion to be evaluated is detected (step S115). For example, the motion to be evaluated is at least one of the motions to be processed, such as the movement motion of the robot arms 120A and 120B to move the end effectors 130A and 130B to the requested item W1, the gripping motion of the end effectors 130A and 130B to grip the requested item W1, the movement and placement motion of the robot arms 120A and 120B to move and place the requested item W1 on the table 140a, and the grip release motion of the end effectors 130A and 103B to release the grip of the requested item W1. In this example, all the motions to be processed are the motions to be evaluated.

[0119] The information processing device 180 evaluates the evaluation features of the operation-related information corresponding to each evaluation target operation, such as the time required for the operation of the robot 100, the position, posture, position movement speed, posture movement speed, position acceleration, posture acceleration and movement trajectory of the end effectors 130A and 103B, the position, posture and state of the requested item W1, the force and impact that the robot 100 applies to the requested item W1, the contact, force and impact that the robot 100 applies to the target object peripheral elements (such as the shelf boards of the storage shelf SR and other items), and the interference between the robot 100 and the robot peripheral elements. For example, the acceleration of the end effectors 130A and 103B may indicate the impact that the end effectors 130A and 103B apply to the requested item W1, etc. For example, the information processing device 180 may be configured to process an image of the requested item W1 captured by the imaging device 144 and / or 145, etc., to detect the holding state of the requested item W1 held by the end effectors 130A and 103B, and to evaluate the holding state as an evaluation target feature. The information processing device 180 may be configured to process an image of the requested item W1 on the table 140a captured by the imaging device 144, etc., to detect the placement state of the requested item W1, and to evaluate the placement state as an evaluation target feature.

[0120] When the information processing device 180 detects the release operation of the grip of the requested item W1 by the end effectors 130A and 103B, it executes an evaluation process and a presentation process of the third work stage, which is the retrieval operation process from the arrival of the robot 100 at a position near the position PW1 to the completion of the grip release, and displays the price and the estimated result on the presentation device 203 (step S116).

[0121] Next, the operator P confirms via the screen of the presentation device 203 that the requested item W1 has been placed on the table 140a, and then operates the operation terminal 200 while visually checking the screen of the presentation device 203, which displays the captured images, etc., of the imaging devices 145 and / or 146, to make the robot 100 travel to a position close to the user P (step S117).

[0122] Similarly to the process of step S107, the information processing device 180 detects the motion-related information of the robot 100 at predetermined time intervals while the robot 100 is running, and performs evaluation processing and presentation processing of the motion-related information each time the information is detected (step S118).

[0123] Similar to the processing of step S108, when the robot 100 arrives at a position near the user P, the information processing device 180 executes evaluation processing and presentation processing of the fourth work stage, which is the traveling process from a position near the position PW1 to a position near the user P (step S119).

[0124] Next, when the operator PO confirms the arrival of the robot 100 at a position near the user P via the screen of the presentation device 203, he inputs an operation for operating the robot arms 120A and 120B and the end effectors 130A and 130B into the operation terminal 200 (step S120). This operation is an operation for handing the requested item W1 placed on the table 140a to the user P using the end effectors 130A and 130B. The operator PO inputs an operation into the operation terminal 200 while visually checking the screen of the presentation device 203 that displays the captured images of the imaging devices 144 and 145 and the like.

[0125] The information processing device 180 detects motion-related information of the robot 100 during the motion of the robot 100, and executes evaluation processing and presentation processing of the motion-related information every time motion-related information corresponding to the motion to be evaluated is detected (step S121). For example, the motion to be evaluated is at least one of the motions to be processed, such as the movement motion of the robot arms 120A and 120B to move the end effectors 130A and 130B to the requested item W1, the gripping motion of the end effectors 130A and 130B to grip the requested item W1, the movement motion of the robot arms 120A and 120B to move the requested item W1 in front of the user P and present it to him, and the grip releasing motion of the end effectors 130A and 103B to release the grip of the requested item W1 according to the holding state of the requested item W1 by the user P. In this example, all the motions to be processed are the motions to be evaluated.

[0126] The information processing device 180 evaluates the evaluation target features of the operation-related information corresponding to each evaluation target operation, such as the time required for the operation of the robot 100, the position, posture, position movement speed, posture movement speed, position acceleration, posture acceleration and movement trajectory of the end effectors 130A and 103B, the position, posture and state of the requested item W1, the force and impact that the robot 100 applies to the requested item W1, and the contact, force and impact that the robot 100 applies to the object peripheral element (user P). For example, the information processing device 180 may be configured to process the image of the requested item W1 captured by the imaging devices 144 and / or 145, etc., to detect the holding state of the requested item W1 grasped by the end effectors 130A and 103B, and evaluate the holding state as the evaluation target feature.

[0127] When the information processing device 180 detects the end effectors 130A and 103B releasing the grip of the requested item W1, it executes the evaluation process and presentation process of the fifth work stage, which is the handover operation process from the arrival of the robot 100 at a position near the user P to the completion of releasing the grip, and displays the price and the estimated result on the presentation device 203 (step S122).

[0128] When the operator PO ends the robot operation, he / she inputs a command to end the operation into the operation terminal 200. When the information processing device 180 receives the command to end the operation from the operation terminal 200 (Yes in step S123), it ends the series of processes. The information processing device 180 sets the task execution flag to OFF and stops the robot 100. The server 300 cuts off the connection between the operation terminal 200 and the information processing device 180. When the information processing device 180 does not receive the command to end the operation (No in step S123), it returns to step S106.

[0129] In addition, in the process of steps S106 to S123, the information processing device 180 stores and accumulates the action-related information corresponding to the evaluated action, the evaluation result of the work stage, the compensation, and the accumulated compensation result, in association with the action-related reference data used for the evaluation.

[0130] By the process of steps S101 to S123 as described above, the operator PO can operate the robot 100 while checking the evaluation of the operator PO's operation through the compensation and the accumulated compensation result for each evaluation target action and work step. This allows the operator PO to find a specific measure to improve the robot operation skill. And the operator PO can be motivated to improve the robot operation skill in order to get more compensation.

[0131] (Modification) This modification example differs from the embodiment in that the robot system includes a learning device 400. Below, this modification example will be described focusing on the differences from the embodiment, and descriptions of the same points as in the embodiment will be omitted as appropriate.

[0132] FIG. 7 is a block diagram showing an example of a functional configuration of the information processing device 180A and the learning device 400 according to the modified example. As shown in FIG. 7, the information processing device 180A further includes a data input / output unit 180i as a functional component. The data input / output unit 180i inputs and outputs data between the information processing device 180A and the learning device 400. For example, the data input / output unit 180i outputs accumulated data stored in the second storage unit 181b to the learning device 400. In response to a request from the criterion determination unit 180g, the data input / output unit 180i requests action-related reference data from the learning device 400. The data input / output unit 180i receives a designated condition and information on an evaluation result that satisfies the designated condition from the criterion determination unit 180g and sends the information to the learning device 400. The learning device 400 sends the action-related reference data corresponding to the designated condition and the evaluation result to the data input / output unit 180i.

[0133] The learning device 400 includes a computer device, similar to the information processing device 180A. Although not limited thereto, in this modification, the learning device 400 is a device separate from the information processing device 180A and the robot controller 190, and is connected to the information processing device 180A so as to be able to perform data communication via wired communication, wireless communication, or a combination thereof. Any wired communication or wireless communication may be used. The learning device 400 is mounted on the robot 100 together with the information processing device 180A. The learning device 400 may be incorporated into the information processing device 180A or the robot controller 190. The learning device 400 and the information processing device 180A form an information processing system 500.

[0134] The learning device 400 may be configured to input and output data to and from the information processing device 180A via a storage medium. The storage medium may include a semiconductor-based or other integrated circuit (IC), a hard disk drive (HDD), a hybrid hard disk drive (HHD), an optical disk, an optical disk drive (ODD), a magneto-optical disk, a magneto-optical drive, a floppy disk drive (FDD), a magnetic tape, a solid-state drive (SSD), a RAM drive, a secure digital card or drive, any other suitable storage medium, or a combination of two or more of these.

[0135] In this modification, the learning device 400 is mounted on the robot 100, but may be configured to be arranged outside the robot 100 and connected to the information processing device 180A via a communication network N. The learning device 400 may be configured to be connectable to the information processing devices 180A of a plurality of robots 100. For example, the learning device 400 may be arranged in a service providing area AS, an operation area AO, or other locations. For example, the learning device 400 may be arranged at the location of the server 300 and configured to perform data communication with the information processing device 180A via the server 300. The learning device 400 may be incorporated into the server 300.

[0136] The learning device 400 includes, as functional components, a data input / output unit 401, a learning data storage unit 402, a machine learning unit 403, an input data processing unit 404, and an output data processing unit 405. The function of the learning data storage unit 402 is realized by a memory and / or storage that the learning device 400 may include, and the functions of the functional components other than the learning data storage unit 402 are realized by a processor or the like included in the learning device 400.

[0137] The data input / output unit 401 receives accumulated data from the data input / output unit 180i of the information processing device 180A, and stores the accumulated data in the learning data storage unit 402 as data for machine learning. The data input / output unit 401 receives a request command for action-related reference data, information on specified conditions, and information on evaluation results that satisfy the specified conditions from the data input / output unit 180i, and outputs them to the input data processing unit 404. The data input / output unit 401 receives action-related reference data from the output data processing unit 405, and sends the action-related reference data to the data input / output unit 180i.

[0138] The input data processing unit 404 converts the information on the specified conditions and the information on the evaluation results received from the data input / output unit 401 into data that can be input to the machine learning model of the machine learning unit 403, and outputs it to the machine learning unit 403. The output data processing unit 405 converts the output data of the machine learning unit 403 into action-related reference data, and outputs it to the data input / output unit 401.

[0139] The machine learning unit 403 includes a machine learning model. The machine learning unit 403 uses data for machine learning to train the machine learning model, thereby improving the accuracy of output data for input data to the machine learning model. The machine learning model may be configured as a neural network, random forest, genetic programming, a regression model, a tree model, a Bayesian model, a time series model, a clustering model, an ensemble learning model, or the like, but in this modified example, it is configured as a neural network. The neural network is configured with a plurality of node layers including an input layer and an output layer. The node layer includes one or more nodes. When the neural network is configured with an input layer, an intermediate layer, and an output layer, the neural network sequentially performs output processing from the input layer to the intermediate layer and output processing from the intermediate layer to the output layer for information input to a node of the input layer, and outputs an output result that matches the input information. Each node of one layer is connected to each node of the next layer, and the connections between the nodes are weighted. The information of the node of one layer is output to the node of the next layer with the weighting of the connections between the nodes added.

[0140] The machine learning model receives as input data condition-related information related to a specified condition and evaluation-related information related to an evaluation result, and receives as output data information related to the condition-related information and the action-related criteria corresponding to the evaluation-related information. The condition-related information may be information indicating attributes of the specified condition and / or the evaluation result, and the evaluation-related information may be information indicating the evaluation result.

[0141] Furthermore, the machine learning model uses the condition-related information and evaluation-related information included in the data for machine learning as input data for machine learning, and uses criterion-related information related to the condition-related information and the action-related criterion data corresponding to the evaluation-related information as teacher data. The criterion-related information may be information indicating the action-related criterion data. For example, in machine learning, the machine learning unit 403 adjusts the weighting of the connections between the nodes in the neural network so that the output data of the machine learning model when the input data for machine learning is inputted coincides with or minimizes the error between the output data and the teacher data corresponding to the input data for machine learning. When the condition-related information and evaluation-related information are inputted, the machine learning model after such weighting adjustment can output criterion-related information appropriate to the information.

[0142] The learning device 400 as described above can output action-related reference data that is more appropriate for the specified conditions and evaluation results than the action-related reference data determined by the reference determination unit 180g of the embodiment. Note that in this modification, the machine learning model of the machine learning unit 403 includes the function of the output data processing unit 405, and may be configured to output output data similar to that of the output data processing unit 405.

[0143] (Other embodiments) Although the embodiment and the modified examples of the present disclosure have been described above, the present disclosure is not limited to the above embodiment and modified examples. In other words, various modifications and improvements are possible within the scope of the present disclosure. For example, the embodiment and modified examples to which various modifications have been applied, and forms constructed by combining components in different embodiments and modified examples are also included within the scope of the present disclosure.

[0144] For example, in the embodiment and the modified example, the work of the robot 100 targeted by the robot system 1 is work related to the service industry, but is not limited thereto. The robot system 1 may target any work. For example, the robot system 1 may target work related to industry, agriculture, and other industries.

[0145] In the embodiment and the modified example, the robot system 1 includes the robot 100, but the robot included in the robot system 1 may be any robot capable of performing work. For example, the robot may include a moving device having a configuration different from that of the carrier vehicle 110, and may include, for example, legs or the like. The robot may not include a moving device. For example, the robot may include components other than a robot arm, and may be, for example, a humanoid robot, an animal-type robot, or the like.

[0146] In the embodiment and the modified example, the information processing device 180 is configured to use image data of an external element other than the robot 100, such as a robot peripheral element, an object, and an object peripheral element, in order to detect the state of these elements, but is not limited thereto. For example, the information processing device 180 may be configured to detect the state of the external element using a detection result of an external sensor, which is a sensor arranged separately from the external element, and / or a detection result of an on-board sensor, which is a sensor arranged on the external element. The external sensor may be configured to detect the position and posture of the external element from the outside. For example, the external sensor is configured to perform detection using light waves, lasers, magnetism, radio waves, electromagnetic waves, ultrasonic waves, or a combination thereof, and may be a photoelectric sensor, a laser sensor, a radio wave sensor, an electromagnetic wave sensor, an ultrasonic sensor, various LiDARs, and the like. The on-board sensor may be configured to move together with the external element and detect the position and posture of the external element. For example, the on-board sensor may be an acceleration sensor, an angular velocity sensor, a magnetic sensor, a GPS (Global Positioning System) receiver, and the like.

[0147] The information processing devices 180 and 180A according to the embodiment and the modified example may be configured to use AI (Artificial Intelligence) for processing. For example, the AI ​​can be used for a process for determining an evaluation of a processing target action, a process for determining an evaluation of a task stage, a process for calculating a price and an integrated result, and image processing. For example, the AI ​​may include a learning model that performs machine learning. For example, the learning model may include a neural network.

[0148] In the embodiment and the modified example, the server 300 may be configured to have at least some of the functions of the information processing devices 180 and 180A and the learning device 400. For example, the server 300 may be configured to have at least one function of the functional components of the information processing devices 180 and 180A shown in Figures 4 and 7. The server 300 may be configured to have at least one function of the functional components of the learning device 400 shown in Figure 7.

[0149] Examples of each aspect of the technology of the present disclosure are given as follows: An information processing device according to one aspect of the present disclosure is an information processing device that evaluates the operation of a robot to cause the robot to perform a task, and includes a detection processing unit that performs, during the execution of the task, a detection process that detects action-related information, which is information related to the action of the robot that operates according to an operation command input by a user to an operation device, an evaluation processing unit that performs, during the execution of the task, an evaluation process that compares the action-related information detected by the detection processing unit with an action-related criterion, which is a criterion related to the action of the robot, and evaluates the result of the evaluation process, and a presentation processing that performs, during the execution of the task, a presentation device that receives presentation of information while the user is operating the operation device and presents the evaluation result of the evaluation processing unit. The evaluation processing unit executes the evaluation process at least one of the timing for each evaluation target action, which is the action of the robot to be evaluated, and the timing for each evaluation target stage, which is the work stage to be evaluated in the work, and the presentation processing unit, when the evaluation process is executed at the timing for each evaluation target action, causes the presentation device to present the evaluation results for the evaluation target action in the same order as the execution order of the evaluation target action, and when the evaluation process is executed at the timing for each evaluation target stage, causes the presentation processing unit to present the evaluation results for the evaluation target stage in the same order as the execution order of the evaluation target stage.

[0150] According to the above aspect, the information processing device executes an evaluation process for the evaluation target action and / or the evaluation target stage at at least one of the timing of each evaluation target action and the timing of each evaluation target stage, and presents the evaluation result to the user via the presentation device. Then, the user can check the evaluation of the operation at the above timing. Furthermore, the user can check the evaluation of the operation in the same order as the order of the operations. This allows the user to recognize the evaluated operation and the evaluation result of the operation, and therefore may be motivated to improve the operation accuracy the next time the evaluation target operation is performed. Therefore, the information processing device can increase the user's motivation to operate a robot.

[0151] In an information processing device according to one embodiment of the present disclosure, when the evaluation processing unit executes the evaluation processing at the timing of each of the evaluated actions, the evaluation processing unit may execute the evaluation processing for a first evaluated action by the timing of a second evaluated action which is the next evaluated action after the first evaluated action, and the presentation processing unit may execute the presentation processing of the evaluation result for the first evaluated action by the timing of the second evaluated action.

[0152] According to the above aspect, the information processing device executes the evaluation process and the presentation process for the evaluation target action by the timing of the next evaluation target action, thereby allowing the user to accurately recognize the evaluation target action that has been evaluated and the evaluation result for the evaluation target action.

[0153] In an information processing device according to one embodiment of the present disclosure, when the evaluation processing unit executes the evaluation processing at the timing for each evaluation target stage, the evaluation processing for a first evaluation target stage may be executed by the timing of a second evaluation target stage which is the evaluation target stage next to the first evaluation target stage, and the presentation processing unit may execute the presentation processing of the evaluation result for the first evaluation target stage by the timing of the second evaluation target stage.

[0154] According to the above aspect, the information processing device executes the evaluation process and the presentation process for the evaluation target stage by the timing of the next evaluation target stage. Therefore, the user can accurately recognize the evaluation target stage that has been evaluated and the evaluation result for the evaluation target stage.

[0155] An information processing device according to one embodiment of the present disclosure may further include a compensation determination unit that determines a compensation to be given to the user for operating the robot based on the evaluation result, and the presentation processing unit may cause the presentation device to present the compensation determined by the compensation determination unit and the result of accumulating the compensation during the performance of the work.

[0156] According to the above aspect, the information processing device presents the compensation indicating each evaluation result and the accumulated compensation to the user via the presentation device. This may increase the user's motivation to operate the robot in order to obtain more compensation. For example, the compensation may be a reward amount, points, points, or a score.

[0157] In the information processing device according to one aspect of the present disclosure, the detection processing unit may detect, as the action-related information, at least one of the operation command, an action command generated for the robot to make the robot act according to the operation command, an actual action result of the robot, a state of the surrounding environment of the robot, a state of an object acted upon by the robot, and a state of the surrounding environment of the object. According to the above aspect, the information processing device can perform evaluation processing based on various information.

[0158] In an information processing device according to one aspect of the present disclosure, the evaluation processing unit may compare and evaluate a plurality of evaluation target features, which are features for evaluating the action-related information, with criteria for the plurality of evaluation target features included in the action-related criteria, and evaluate the action-related information based on the evaluation results of the plurality of evaluation target features.

[0159] According to the above aspect, the evaluation method in the information processing device can be clear and simple. Therefore, it is possible to reduce the amount of processing and speed up the evaluation process. For example, the evaluation target features may include features such as the time required for the robot's motion, position, posture, trajectory, position speed, posture speed, position acceleration, and posture acceleration, the force and impact that the robot exerts on the robot's surrounding environment, as well as changes in the position, posture, shape, and state of the surrounding environment, the position, posture, position speed, posture speed, position acceleration, and posture acceleration of the object on which the robot acts, changes in shape, changes in state, and workmanship of the object, the force and impact that the robot exerts on the surrounding environment of the object, as well as changes in the position, posture, shape, and state of the surrounding environment. The information processing device may determine the evaluation result of the motion-related information by weighting the evaluation results of each of the multiple evaluation target features.

[0160] The information processing device according to one aspect of the present disclosure may further include a storage unit and a criterion determination unit, the storage unit stores information on the action-related criterion and information on an evaluation result of the action-related information based on the action-related criterion, and the criterion determination unit may determine information on the second action-related criterion using the information on the first action-related criterion and the information on the evaluation result based on the first action-related criterion stored in the storage unit. According to the above aspect, the information processing device can newly determine the action-related criterion according to the evaluation result stored in the storage unit.

[0161] In an information processing device according to one embodiment of the present disclosure, the criterion determination unit may extract, from the information stored in the storage unit, information on the evaluation result that satisfies a specified condition and information on the first action-related criterion, which is the action-related criterion used in the evaluation result, and determine information on the second action-related criterion using the extracted information on the evaluation result and the information on the first action-related criterion.

[0162] According to the above aspect, the information processing device can determine new action-related criteria using evaluation results that satisfy a specified condition. Thus, the information processing device can determine action-related criteria for various purposes. For example, the specified condition may be a condition that specifies an evaluation result corresponding to an object to be acted on by the robot, a robot, a task, a work location, a work environment, a date and a period during which the task is performed, etc.

[0163] A learning device according to one embodiment of the present disclosure includes a learning data memory unit that stores information on the action-related criteria in an information processing device according to one embodiment of the present disclosure and information on the evaluation result of the action-related information based on the action-related criteria as machine learning data, and a machine learning unit, wherein the machine learning unit performs machine learning using the evaluation result information included in the machine learning data as machine learning input data and the action-related criteria information included in the machine learning data and used in the evaluation result as teacher data, and the machine learning unit uses evaluation information corresponding to the evaluation result information as input data and the action-related criteria information corresponding to the evaluation information as output data.

[0164] According to the above aspect, when the learning device after machine learning receives an input of evaluation information corresponding to an evaluation result, it can output a movement-related criterion appropriate for the evaluation information. For example, when evaluation information corresponding to a high evaluation result is input to the learning device, the learning device outputs a simple and low-level movement-related criterion that is easy to obtain a high evaluation. For example, when evaluation information corresponding to a low evaluation result is input to the learning device, the learning device outputs a high-level movement-related criterion that is difficult to obtain a high evaluation. For example, a user can use the learning device to obtain a movement-related criterion of a level of difficulty desired by the user and use it for robot operation.

[0165] An information processing system according to an aspect of the present disclosure includes an information processing device according to an aspect of the present disclosure and a learning device according to an aspect of the present disclosure, and the learning device is configured to receive input data from the operation device or the information processing device and output output data corresponding to the input data to the information processing device. According to the above aspect, the information processing system can obtain the same effects as the information processing device and the learning device according to an aspect of the present disclosure. The information processing system can execute an evaluation process in the information processing device using the action-related criteria output from the learning device.

[0166] A robot system according to an aspect of the present disclosure includes an information processing device according to an aspect of the present disclosure, the robot, and a robot controller that controls an operation of the robot, the information processing device being communicably connected to the robot controller. According to the above aspect, the robot system can obtain the same effects as the information processing device according to the aspect of the present disclosure.

[0167] A robot system according to an aspect of the present disclosure includes an information processing system according to an aspect of the present disclosure, the robot, and a robot controller that controls an operation of the robot, and the information processing device is communicatively connected to the robot controller. According to the above aspect, the robot system can obtain the same effects as the information processing system according to an aspect of the present disclosure.

[0168] In addition, all the numbers such as ordinal numbers and quantities used above are exemplified to specifically explain the technology of the present disclosure, and the present disclosure is not limited to the exemplified numbers. In addition, the connection relationships between the components are exemplified to specifically explain the technology of the present disclosure, and the connection relationships that realize the functions of the present disclosure are not limited to these.

[0169] In addition, the division of blocks in the functional block diagram is an example, and multiple blocks may be realized as one block, one block may be divided into multiple blocks, and / or some functions may be transferred to another block. Furthermore, the functions of multiple blocks having similar functions may be processed in parallel or in a time-division manner by a single piece of hardware or software. [Explanation of symbols]

[0170] 1. Robot System 100 Robots 170 Control device 180,180A Information processing device 180a Operation command section 180b Detection processing unit 180c Evaluation processing section 180d Pricing Department 180e Presentation processing unit 180f Accumulation processing section 180g standard determination section 181a 1st memory section 181b Second storage section (storage section) 190 Robot Controller 200 Operation terminal (operation device) 203 Presentation device 400 Learning Device 402 Learning data storage unit 403 Machine Learning Department 500 Information Processing Systems

Claims

1. An information processing device that evaluates an operation of a robot to cause the robot to execute a task, a detection processing unit that performs a detection process to detect operation-related information, which is information related to an operation of the robot that operates according to an operation command input by a user to an operation device, during the execution of the work; an evaluation processing unit that performs an evaluation process, during the execution of the task, of comparing the motion-related information detected by the detection processing unit with a motion-related criterion that is a criterion related to the motion of the robot, and evaluating the motion-related information; a presentation processing unit that performs a presentation process to present the evaluation result of the evaluation processing unit on a presentation device that receives presentation of information while the user is operating the operation device, during the execution of the work; the evaluation processing unit executes the evaluation process at least one of a timing for each evaluation target motion, which is a motion of the robot to be evaluated, and a timing for each evaluation target stage, which is a task stage to be evaluated in the task; when the evaluation process is executed at a timing for each of the evaluation target actions, when the evaluation process is performed on the evaluation target action by the evaluation processing unit, the presentation processing unit causes the presentation device to present the evaluation result regarding the evaluation target action that has been subjected to the evaluation process; When the evaluation process is executed at the timing of each of the evaluation target stages, when the evaluation process is performed on the evaluation target stage by the evaluation processing unit, the presentation processing unit causes the presentation device to present the evaluation result regarding the evaluation target stage that has been subjected to the evaluation process; One of the tasks includes a plurality of the evaluation target actions and a plurality of the evaluation target steps. Information processing device.

2. When the evaluation processing unit executes the evaluation process at the timing of each of the evaluated actions, the evaluation processing unit executes the evaluation process for a first evaluated action by the timing of a second evaluated action that is the next evaluated action after the first evaluated action, The presentation processing unit executes the presentation process of the evaluation result regarding the first evaluated action by a timing of the second evaluated action. The information processing device according to claim 1 .

3. When the evaluation processing unit executes the evaluation process at the timing of each evaluation target stage, the evaluation processing unit executes the evaluation process for the first evaluation target stage by the timing of the second evaluation target stage, which is the evaluation target stage next to the first evaluation target stage; The presentation processing unit executes the presentation process of the evaluation result regarding the first evaluation target stage by the timing of the second evaluation target stage.

3. The information processing device according to claim 1 or 2.

4. and a compensation determination unit that determines a compensation to be given to the user for operating the robot based on the evaluation result. The presentation processing unit causes the presentation device to present the consideration determined by the consideration determination unit and a result of accumulating the consideration during the execution of the work. The information processing device according to any one of claims 1 to 3.

5. The detection processing unit detects, as the operation-related information, at least one of the operation command, an operation command generated for the robot to operate the robot in accordance with the operation command, an actual operation result of the robot, a state of the surrounding environment of the robot, a state of an object acted upon by the robot, and a state of the surrounding environment of the object. The information processing device according to any one of claims 1 to 4.

6. The evaluation processing unit compares and evaluates a plurality of evaluation target features, which are features for evaluating the action-related information, with standards for the plurality of evaluation target features included in the action-related criteria, and evaluates the action-related information based on an evaluation result of the plurality of evaluation target features. The information processing device according to any one of claims 1 to 5.

7. The apparatus further includes a storage unit and a criterion determination unit, The storage unit stores action-related criterion data including the action-related criterion and information on the evaluation result of the action-related information based on the action-related criterion; The criterion determination unit uses information on the first action-related reference data stored in the storage unit and the evaluation result based on the action-related criterion included in the first action-related reference data to generate second action-related reference data by varying the action-related criterion included in the first action-related reference data in accordance with the evaluation result. The information processing device according to any one of claims 1 to 6.

8. The criterion determination unit extracts, from the information stored in the storage unit, information on the evaluation result that satisfies a specified condition and the first action-related reference data that is the action-related reference data used for the evaluation result, and generates the second action-related reference data using the extracted information on the evaluation result and the first action-related reference data. The information processing device according to claim 7.

9. a learning data storage unit that stores information on the action-related criterion and information on the evaluation result of the action-related information based on the action-related criterion as machine learning data in the information processing device according to any one of claims 1 to 8; A machine learning unit; the machine learning unit performs machine learning using information of the evaluation result included in the data for machine learning as input data for machine learning and information of the action-related criterion included in the data for machine learning and used in the evaluation result as teacher data; The machine learning unit receives evaluation information corresponding to the evaluation result information as input data, and receives information on the action-related criterion corresponding to the evaluation information as output data. Learning device.

10. An information processing device according to any one of claims 1 to 8; The learning device according to claim 9, The learning device is configured to receive input data from the operation device or the information processing device, and to output output data corresponding to the input data to the information processing device. Information processing system.

11. An information processing device according to any one of claims 1 to 8; The robot; A robot controller for controlling an operation of the robot, The information processing device is communicatively connected to the robot controller. Robot system.

12. An information processing system according to claim 10; The robot; A robot controller for controlling an operation of the robot, The information processing device is communicatively connected to the robot controller. Robot system.

13. A method for evaluating an operation of a robot to cause the robot to execute a task, comprising: executing, during the execution of the work, a detection process for detecting operation-related information, which is information related to an operation of the robot that operates in accordance with an operation command input by a user to an operation device; executing an evaluation process for comparing and evaluating the motion-related information detected in the detection process with a motion-related criterion that is a criterion related to the motion of the robot, during the execution of the task, at least one of the timing of each evaluation target motion, which is the motion of the robot to be evaluated, and the timing of each evaluation target stage, which is a task stage to be evaluated in the task; When the evaluation process is executed at a timing for each of the evaluation target actions, when the evaluation process is executed for the evaluation target action, a presentation process is executed during the execution of the work, in which an evaluation result for the evaluation target action that has been subjected to the evaluation process is presented on a presentation device that receives presentation of information while the user is operating the operation device; When the evaluation process is executed at the timing of each evaluation target stage, when the evaluation process is executed for the evaluation target stage, a presentation process is executed during the execution of the work, the presentation process being for presenting, on the presentation device, an evaluation result for the evaluation target stage that has been subjected to the evaluation process; One of the tasks includes a plurality of the evaluation target actions and a plurality of the evaluation target steps. Evaluation method.

Images