Work reproduction system, work reproduction method and work reproduction robot

Abstract

To provide a work reproduction system, a work reproduction method and a work reproduction robot which facilitate clarification of an occurrence factor of an abnormal situation.SOLUTION: A work reproduction system according to the present invention comprises a work reproduction robot, a sensor that can sense motion of the work reproduction robot, and a management control device that can communicate with the work reproduction robot and the sensor. The management control device comprises: a leaning part that learns a standard motion model corresponding to predetermined motion of an object to be sensed, on the basis of first sensing information corresponding to the predetermined motion of the object to be sensed; a control part that makes the work reproduction robot perform operation of reproduction once or more while referring to the standard motion model; an input part that inputs information on an accident or on erroneous motion; and a sensing part that senses occurrence of the accident or the erroneous motion, on the basis of second sensing information corresponding to the operation of reproduction performed by the work reproduction robot obtained using the sensor.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a task reproduction system, a task reproduction method, and a task reproduction robot. [Background technology]

[0002] In a workplace where a specific task is performed, an abnormal situation such as an accident may occur due to a worker's mistake or other factors. When an abnormal situation occurs, clarifying the cause is effective in preventing the next abnormal situation from occurring. For example, Patent Document 1 discloses a technology for providing a safety education system that effectively conveys the occurrence of an accident situation, in which accident scene image data of an accident scene image viewed from a viewpoint specified by the viewpoint information is generated based on three-dimensional data of the accident scene where the accident occurred and input viewpoint information, and the image data is displayed on a display unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2007-226515 A Summary of the Invention [Problem to be solved by the invention]

[0004] When an abnormality occurs, it is often caused by a combination of various factors, and in order to clarify the cause of the abnormality, the situation at the site is reproduced. In addition, the relevant people (workers) at the time of the abnormality are placed at the reproduction site and asked to actually perform the actions that would have been performed if the abnormality had occurred. In this case, the situation at the time the abnormality occurred can be reproduced more accurately, making it easier to clarify the cause of the abnormality. However, depending on the situation, it may be difficult to have the relevant people directly reproduce the accident situation.

[0005] Therefore, an object of the present invention is to provide a task reproduction system, a task reproduction method, and a task reproduction robot that make it easy to clarify the cause of an abnormality. [Means for solving the problem]

[0006] The work reproduction system of the present invention comprises a work reproduction robot, a sensor capable of sensing the movement of the work reproduction robot, and a management control device capable of communicating with the work reproduction robot and the sensor, wherein the management control device comprises: a learning unit that learns a standard movement model corresponding to a specified movement of a sensing target based on first sensing information corresponding to the specified movement of the sensing target; a control unit that causes the work reproduction robot to perform a reproduction movement one or more times by referring to the standard movement model; an input unit that inputs accident or malfunction information; and a detection unit that detects the occurrence of the accident or malfunction based on second sensing information corresponding to the reproduction movement of the work reproduction robot obtained using the sensor.

[0007] The work reproduction system of the present invention comprises a work reproduction robot, a sensor capable of sensing the movement of the work reproduction robot, and a management control device capable of communicating with the work reproduction robot and the sensor, wherein the management control device comprises: a learning unit that learns a standard movement model corresponding to a specified movement of a sensing target based on first sensing information corresponding to the specified movement of the sensing target; a control unit that causes the work reproduction robot to perform the reproduction movement one or more times by referring to the standard movement model; a memory unit that stores work manual information or process chart information of the sensing target; and a detection unit that detects the occurrence of a movement different from the work manual information or the process chart information based on second sensing information corresponding to the reproduction movement of the work reproduction robot obtained using the sensor.

[0008] Furthermore, the task reproduction method of the present invention includes a task reproduction robot, a sensor capable of sensing the operation of the task reproduction robot, and a management control device capable of communicating with the task reproduction robot and the sensor, wherein the management control device learns a standard operation model corresponding to a predetermined operation of a sensing target based on first sensing information corresponding to the predetermined operation of a sensing target, causes the task reproduction robot to perform the reproduction operation one or more times by referring to the standard operation model, inputs accident or malfunction information, and detects the occurrence of the accident or malfunction based on second sensing information corresponding to the reproduction operation of the task reproduction robot obtained using the sensor.

[0009] Furthermore, the work reproduction method of the present invention includes a work reproduction robot, a sensor capable of sensing the operation of the work reproduction robot, and a management control device capable of communicating with the work reproduction robot and the sensor, wherein the management control device learns a standard operation model corresponding to a predetermined operation of the sensing target based on first sensing information corresponding to the predetermined operation of the sensing target, causes the work reproduction robot to perform the reproduction operation one or more times by referring to the standard operation model, stores work manual information or process chart information of the sensing target, and detects the occurrence of an operation different from the work manual information or the process chart information based on second sensing information corresponding to the reproduction operation of the work reproduction robot obtained using the sensor.

[0010] Furthermore, the work reproducing robot of the present invention comprises an information processing device capable of communicating with a sensor capable of sensing the operation of the work reproducing robot, and the information processing device comprises: a learning unit that learns a standard operation model corresponding to a predetermined operation of a sensing target based on first sensing information corresponding to the predetermined operation of the sensing target; a control unit that causes the work reproducing robot to perform the reproducing operation one or more times by referring to the standard operation model; an input unit that inputs accident or malfunction information; and a detection unit that detects the occurrence of the accident or malfunction based on second sensing information corresponding to the reproducing operation of the work reproducing robot acquired using the external sensor.

[0011] Furthermore, the work reproduction robot of the present invention includes a sensor capable of sensing the operation of the work reproduction robot and an information processing device capable of communicating with the sensor, and the information processing device is characterized in that it includes: a learning unit that learns a standard operation model corresponding to a predetermined operation of the sensing target based on first sensing information corresponding to the predetermined operation of the sensing target; a control unit that causes the work reproduction robot to perform the reproduction operation one or more times by referring to the standard operation model; a memory unit that stores work manual information or process chart information of the sensing target; and a detection unit that detects the occurrence of an operation different from the work manual information or the process chart information based on second sensing information corresponding to the reproduction operation of the work reproduction robot acquired using the sensor. Effect of the Invention

[0012] According to the present invention, there are provided a task reproduction system, a task reproduction method, and a task reproduction robot that facilitate identifying the cause of an abnormality. [Brief description of the drawings]

[0013] [Figure 1] FIG. 2A is a diagram showing an example of sensing when a worker is made to reproduce an action, and FIG. 2B is a diagram showing an example of sensing when a work reproduction robot is made to reproduce an action, in the work reproduction system of the first embodiment according to the present invention. [Diagram 2] 1 is an example of a humanoid robot in the task reproduction system of embodiment 1 according to the present invention. [Diagram 3] FIG. 1 is a block diagram showing an example of the configuration and functions of a task reproduction system according to a first embodiment of the present invention. [Figure 4] FIG. 2 is a block diagram showing an example of functions of a management control device in the task reproduction system according to the first embodiment of the present invention. [Diagram 5] 1 is an example of a flowchart showing a process of the task reproduction system according to the first embodiment of the present invention. [Figure 6] 7 is an example of a flowchart showing more detailed processing of the worker action learning / standard action model generation processing shown in step S102 of FIG. 5. [Figure 7] 1 is an example of a flowchart showing a process of the task reproduction system according to Modification 1 of Embodiment 1 of the present invention. [Figure 8] FIG. 1A is a diagram showing an example of a system configuration in a task reproduction system according to a second modified example of the first embodiment of the present invention, and FIG. 1B is a diagram showing an example of the humanoid robot shown in FIG. [Figure 9] FIG. 11 is a block diagram showing an example of functions of a humanoid robot in a task reproduction system according to a second modified example of the first embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] The task reproduction system, task reproduction method, and task reproduction robot will be described below with reference to the drawings. However, it should be noted that the technical scope of the present invention is not limited to the embodiments, but extends to the inventions described in the claims and their equivalents.

[0015] (Embodiment 1) FIG. 1 is a diagram for explaining the task reproduction system.

[0016] In this work reproduction system, when an abnormal situation such as an accident or malfunction occurs, the worker 400 is made to perform the action at the time of the occurrence of the abnormal situation, and the second robot sensor 23b (second robot imaging device 24b) senses the predetermined action of the worker 400. In addition, this work reproduction system learns a standard action model corresponding to the predetermined action of the worker 400 based on the first sensing information corresponding to the predetermined action of the worker 400. The standard action model is a model that represents an action corresponding to the predetermined action of the worker 400 and is specified as an action to be taken in a predetermined work item. Thereafter, this work reproduction system refers to the standard action model and makes the first humanoid robot 20a, which is the work reproduction robot, perform the reproduction action one or more times. This work reproduction system inputs the accident or malfunction information, and detects the occurrence of the accident or malfunction based on the second sensing information corresponding to the reproduction action of the first humanoid robot 20a obtained by using the sensor. This makes it possible to analyze problems in the action of the worker 400 and defects in the standard action used in the work of the worker 400.

[0017] FIG. 1(a) is a diagram showing an example of sensing when a worker reproduces an action.

[0018] This task reproduction system includes a first humanoid robot 20a that functions as a task reproduction robot and a second humanoid robot 20b that functions as a mobile robot. Note that the number of humanoid robots is not limited to two.

[0019] The task reproduction system senses the motion of the worker 400 by the second robot sensor 23b (second robot imaging device 24b) provided in the second humanoid robot 20b. An example of the motion of the worker 400 is the motion of the worker 400 when he places a component 320 in a location different from the original placement location 310 when assembling a printed circuit board 300 on the work line 201. The motion of the worker 400 may be recognized by the sensor using a known image recognition technique, or the motion of the worker 400 may be recognized by learning by a learning unit 663 (see FIG. 4). The same applies to the motion reproduced by the task reproduction robot described later.

[0020] The second humanoid robot 20b senses the movement of the worker 400 using the second robot sensor 23b (second robot imaging device 24b). The first sensing information acquired by the second robot sensor 23b (second robot imaging device 24b) is stored in the storage medium 62 (see FIG. 3) of the management control device 60 or the storage device of the second humanoid robot 20b. The management control device 60 learns a standard movement model corresponding to the predetermined movement of the worker 400 based on the first sensing information corresponding to the predetermined movement of the worker 400. The standard movement model is stored in the storage medium 62 of the management control device 60, and / or the storage device 1224 (see FIG. 3) of the first information processing device 25a of the first humanoid robot 20a and / or the storage device of the second information processing device of the second humanoid robot 20b. The predetermined movement includes various movements before and after the occurrence of an abnormal situation, such as a movement to grab an object, a movement to assemble parts, and a movement when handling tools.

[0021] FIG. 1(b) is a diagram showing an example of sensing when the task reproducing robot reproduces a task. The management control device 60 generates a task instruction to operate the first humanoid robot 20a by referring to the stored standard task model. The task instruction is an instruction generated by referring to the standard task model, and is an instruction to operate the task reproducing robot (the first humanoid robot 20a in this embodiment). Then, the management control device 60 operates the first humanoid robot 20a by referring to the task instruction when the first humanoid robot 20a is placed in a reproduction site where the site where the abnormality occurred is reproduced, for example.

[0022] The management control device 60 refers to the standard motion model and has the first humanoid robot 20a perform the reproduction motion at least once, preferably multiple times. For example, in normal line work, the work process is clearly defined, and accidents and malfunctions rarely occur. Therefore, when an abnormal situation occurs, there is a possibility that the occurrence of an accident or malfunction cannot be detected by having the first humanoid robot 20a perform the reproduction motion only once. Therefore, by having the first humanoid robot 20a perform the reproduction motion multiple times, it becomes easier to detect the occurrence of an accident or malfunction.

[0023] The management control device 60 receives the accident or malfunction information and detects the occurrence of an accident or malfunction based on the second sensing information corresponding to the reproduced behavior of the first humanoid robot 20a acquired using the second robot sensor 23b (second robot imaging device 24b). The accident information is information about the accident, such as who performed what action, when and where. The malfunction information is information indicating the malfunction of the relevant person when an abnormal situation occurs and there is an error in the behavior of the relevant person.

[0024] 2 is a diagram showing an example of a humanoid robot in this task reproduction system. The humanoid robot 20, which functions as a task reproduction robot and a mobile robot, includes a robot body 21, a robot movement mechanism 22, a robot sensor 23, a robot imaging device 24 that may be included in the robot sensor 23, an information processing device 25, and a robot arm 26.

[0025] The humanoid robot 20 can move using a robot movement mechanism 22 provided below the robot body 21, and moves, for example, to a workshop 200 upon receiving instructions from outside the humanoid robot 20, such as from a management control device 60, or by referring to a program stored in an information processing device 25.

[0026] The robot main body 21 includes a robot torso 211 and a robot head 212. The robot torso 211 and the robot head 212 constitute a torso / head drive mechanism, and are capable of changing the sensing area 230 (imaging area 240) of the robot sensor 23 (robot imaging device 24). The configuration of the drive mechanism is not particularly limited, and may be configured such that, for example, the robot head 212 rotates a predetermined angle relative to the robot torso 211, or the robot torso 211 rotates a predetermined angle relative to the robot movement mechanism 22, by a servo motor (not shown).

[0027] A robot moving mechanism 22 is provided below the robot body 211, a robot arm 26 is provided on each side of the robot body 211, and a robot sensor 23 is provided in the robot head 212. An information processing device 25 is provided inside the robot body 21.

[0028] The robot movement mechanism 22 may have any configuration, for example, it may be provided with a rotating body driven by a motor, or may have legs that resemble the shape of a human leg. As an example, when the robot movement mechanism 22 is configured to resemble the shape of a human leg, a servo motor is provided at the location corresponding to the human joint, and the movement mechanism is configured by rotating the servo motor by a predetermined angle.

[0029] The robot sensor 23 is provided in the robot head 212 and is capable of sensing the operation of the worker 400 and the work reproduction robot. The robot sensor 23 also sequentially acquires information indicating at least the distance and angle between the robot arm 26 and an object around the humanoid robot 20 on which the humanoid robot 20 works. As an example of the robot sensor 23, a highest performance camera, a thermal camera, a high pixel, telephoto, ultra-wide angle, 360 degree, high performance camera, a radar, a solid-state LiDAR, a LiDAR, a multi-color laser coaxial displacement meter, a vision recognition, or a group of various other sensors may be adopted. These are also examples of the robot imaging device 24. Other examples of the robot sensor 23 include a vibration meter, a hardness meter, a micro vibration meter, an ultrasonic measuring instrument, a vibration measuring instrument, an infrared measuring instrument, an ultraviolet measuring instrument, an electromagnetic wave measuring instrument, a thermometer, a hygrometer, a spot AI weather forecast, a high-precision multi-channel GPS, low altitude satellite information, or long tail incident AI data.

[0030] Examples of sensor information acquired from the robot sensor 23 include images, distance, vibration, heat, smell, color, sound, ultrasonic waves, radio waves, ultraviolet rays, infrared rays, humidity, etc., and preferably the image and distance information is acquired by the robot imaging device 24. The robot sensor 23 (robot imaging device 24) performs these sensing operations, for example, every nanosecond. The sensor information is used, for example, for motion capture of the movements of the worker 400 and the first humanoid robot 20a, a 3D map of the workplace 200, and analysis of navigation, cornering, speed, etc. of the movement and movements of the worker 400 and the first humanoid robot 20a in the workplace 200.

[0031] The robot arm 26 includes a right arm 261 and a left arm 262. The right arm 261 includes a right gripping support part 263 and a right gripping part 265, and the left arm 262 includes a left gripping support part 264 and a left gripping part 266. The right gripping support part 263 is a mechanism for supporting the right gripping part 265, and the left gripping support part 264 is a mechanism for supporting the left gripping part 266, and may be shaped like a human arm, for example. The gripping parts 265 and 266 are mechanisms for gripping parts for work, for example, and may be shaped like a human hand, for example.

[0032] The robot arm 26 constitutes a second drive mechanism. The configuration of the drive mechanism is not particularly limited, and for example, when the robot arm 26 is made to resemble a human shape, a configuration may be adopted in which servo motors are provided at each joint location, such as a location corresponding to a human shoulder, a location corresponding to an elbow, a location corresponding to a wrist, a location corresponding to a finger joint, and the like, and rotated by a predetermined angle.

[0033] The humanoid robot 20 may further be provided with a sensor, for example, on the robot body 211 (see FIG. 8(b)). In this case, the sensor is located at a different height from the robot sensor 23 located on the robot head 212. The different height allows the sensor to sense the movements of the worker 400 and the first humanoid robot 20a from different angles.

[0034] FIG. 3 is a block diagram showing an example of the configuration and functions of the task reproduction system 100 of this embodiment.

[0035] The task reproduction system 100 includes a first humanoid robot 20a, a second humanoid robot 20b, and a management control device 60. The first humanoid robot 20a and the second humanoid robot 20b are each connected to a communication unit 64 of the management control device 60 via wireless or wired communication, and receive instructions from the management control device 60 and transmit information acquired by each sensor. The first humanoid robot 20a and the second humanoid robot 20b may also be connected to each other via wireless or wired communication, and transmit and receive information acquired by each sensor and instructions.

[0036] The first humanoid robot 20a functioning as a task reproduction robot includes a first robot movement mechanism 22a, a first robot sensor 23a, a first robot imaging device 24a included in the first robot sensor 23a, a first information processing device 25a, a first body / head drive mechanism 21a, and a first arm drive mechanism 26a. In this embodiment, the second humanoid robot 20b functioning as a mobile robot also has the same configuration as the first humanoid robot 20a.

[0037] The first information processing device 25a according to this embodiment includes a central processing unit (CPU) 1212, a random access memory (RAM) 1214, and a graphic controller 1216, which are connected to each other by a host controller 1210. The first information processing device 25a also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid state drive, or the like. The first information processing device 25a also includes input / output units such as a read only memory (ROM) 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0038] The CPU 1212 operates according to a program stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into itself, and causes the image data to be displayed on the display device 1218.

[0039] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the first information processing device 25a. The storage device 1224 may also store sensing information. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0040] The ROM 1230 stores therein a boot program, etc., executed by the first information processing device 25a upon activation, and / or a program that depends on the hardware of the first information processing device 25a. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0041] The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the first information processing device 25a, and brings about cooperation between the programs and the above-mentioned various types of hardware resources. An apparatus or method may be configured by realizing an operation or processing of information according to the use of the first information processing device 25a.

[0042] For example, when communication is performed between the first information processing device 25a and an external device, the CPU 1212 may execute a communication program loaded in the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0043] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0044] Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and undergo information processing. The CPU 1212 may execute various types of processing on data read from the RAM 1214, including various types of operations, information processing, conditional decisions, conditional branches, unconditional branches, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write back the results to the RAM 1214. The CPU 1212 may also search for information in files, databases, etc., in the recording medium.

[0045] The above-described program or software module may be stored in a computer-readable storage medium on the first information processing device 25a or in the vicinity of the first information processing device 25a. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable storage medium, thereby providing the program to the first information processing device 25a via the network.

[0046] The blocks in the flowcharts and diagrams in this embodiment may represent stages of a process in which operations are performed or "parts" of an apparatus responsible for performing the operations. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer readable instructions stored on a computer readable storage medium, and / or a processor provided with computer readable instructions stored on a computer readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as, for example, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like, including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0047] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray disks, memory sticks, integrated circuit cards, and the like.

[0048] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0049] Computer readable instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, either locally or over a local area network (LAN), a wide area network (WAN), such as the Internet, etc., to cause the processor of the general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, to execute the computer readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0050] The contents described so far also apply to the second information processing device provided in the second humanoid robot 20b.

[0051] The management control device 60 is a control device that gives instructions to the humanoid robots 20a and 20b in order to realize the task reproduction system 100. In addition, the management control device 60 acquires sensing information accumulated in the storage devices of the humanoid robots 20a and 20b.

[0052] The management control device 60 is composed of a CPU 60A, a RAM 60B, a ROM 60C, an input / output unit (I / O) 60D, a bus 60E such as a data bus or a control bus connecting these, and a communication unit 64. A storage medium 62 is connected to the I / O 60D.

[0053] In addition, a communication unit 64 is connected to the I / O 60D, which transmits and receives sensing information, work manual information, and schedule information between the control system of the humanoid robot 20. The work manual information includes, for example, the name and content of each work item, the order of the work items, and information on the standard work time required for each work item. In addition, the schedule information includes, for example, information indicating the work time and start / end times of the entire work, information indicating the work time and start / end times of each work item, and information indicating the worker for each work item.

[0054] FIG. 4 is a block diagram showing an example of the functions of the management control device 60 in the task reproduction system of this embodiment.

[0055] The management control device 60 includes a storage medium 62 , a communication unit 64 , and a processing unit 66 .

[0056] The storage medium 62 includes, for example, at least one of a semiconductor storage device, a magnetic tape device, a magnetic disk device, and an optical disk device. The storage medium 62 stores a driver program, an operating system program, an application program, data, and the like used for processing in the processing unit 66. For example, the storage medium 62 stores sensing information. The storage medium 62 may also store work manual information and / or process chart information for the operator 400.

[0057] The communication unit 64 has a wireless communication interface circuit such as Wi-Fi (registered trademark) and / or a wired communication interface circuit such as Ethernet (registered trademark). The communication unit 64 transmits and receives various information to and from the humanoid robots 20a and 20b through the interface circuits.

[0058] The processing unit 66 has one or more processors and their peripheral circuits. The processing unit 66 centrally controls the overall operation of the task reproduction system 100, and is, for example, a CPU. The processing unit 66 executes processing by referring to programs (driver programs, operating system programs, application programs, etc.) stored in the storage medium 62. The processing unit 66 can also execute multiple programs (application programs, etc.) in parallel.

[0059] The processing unit 66 includes a determination unit 661, a control unit 662, a learning unit 663, a generation unit 664, an input unit 665, and a detection unit 666. Each of these units is a functional module realized by a program executed by a processor included in the processing unit 66. Alternatively, each of these units may be implemented in the processing unit 66 as firmware.

[0060] The determination unit 661 determines whether or not a sensing target (the worker 400 or the first humanoid robot 20a) is being sensed. As a method of determination, a known image recognition technique may be used, or learning by the learning unit 663 (see FIG. 4) may be used.

[0061] The control unit 662 causes the first humanoid robot 20a to perform the reproduction action one or more times with reference to the standard action model. Furthermore, when it is determined that the sensing target (the worker 400 or the first humanoid robot 20a) is not being sensed, the control unit 662 operates the second body / head driving mechanism and the second robot movement mechanism of the second humanoid robot 20b.

[0062] The learning unit 663 learns a standard action model corresponding to the predetermined action of the worker 400 based on the first sensing information corresponding to the predetermined action of the worker 400. This learning is performed by automatic learning, which is learning in which a learned model is automatically created and / or a judgment / analysis is automatically performed using the learned model, for example.

[0063] The generating unit 664 generates a standard action model by referring to the learning result by the learning unit 663. The generating unit 664 also generates an action instruction.

[0064] The input unit 665 inputs accident or malfunction information. This input may be an input from outside the work reproduction system 100, or the accident or malfunction information may be stored in advance in the storage medium 62, the storage medium 1224 of the first information processing device 25a, and / or the storage medium of the second information processing device, and the information may be read out by an instruction from the management control device 60, thereby inputting the information.

[0065] The detection unit 666 detects the occurrence of an accident or malfunction based on the second sensing information corresponding to the reproduced motion of the first humanoid robot 20a acquired using the second robot sensor 23b (second robot imaging device 24b). In addition, as described below, the detection unit 666 detects the occurrence of motion different from the work manual information or the work schedule information based on the second sensing information corresponding to the reproduced motion of the work reproduction robot acquired using the sensor. Examples of detection targets include differences in changes in each piece of information (data) over time and grasping significant discrepancies between data when comparing data.

[0066] (Processing of the task reproduction system according to the first embodiment of the present invention) FIG. 5 is an example of a flowchart showing the process of the task reproduction system of this embodiment.

[0067] First, the task reproduction system 100 senses the motion of the worker 400 in the work place 200 by using the second robot sensor 23b (second robot imaging device 24b) of the second humanoid robot 20b in response to an instruction from the management control device 60 or an instruction to read out the program stored in the storage medium 62 or the storage device of the second information processing device (step S101). The motion of the worker 400 is a motion that reproduces the motion that the worker 400 actually performed when an abnormality occurs, and preferably, the motion is performed in the work place 200 that faithfully reproduces the situation when the abnormality occurs. The control unit 662 or the second information processing device operates the second body / head driving mechanism and the second robot moving mechanism of the second humanoid robot 20b so that the sensing area 230b (imaging area 240b) of the second robot sensor 23b (second robot imaging device 24b) of the second humanoid robot 20b includes the motion of the worker 400.

[0068] The sensing information (first sensing information) acquired by the second robot sensor 23b (second robot imaging device 24b) is stored in the storage medium 62 via the storage device and / or communication unit 64 of the second information processing device. The storage device and storage medium 62 of each information processing device function as a storage unit.

[0069] The management control device 60 learns a standard action model corresponding to a specific action of the worker 400 based on the first sensing information accumulated in the memory unit, in other words, stored, and generates a standard action model by referring to the learning results (step S102).

[0070] FIG. 6 is an example of a flowchart showing more detailed processing of the worker action learning / standard action model generation processing shown in step S102 of FIG.

[0071] When the first sensing information is acquired, the first sensing information is stored in the storage unit (step S201), and the learning unit 663 learns a standard action model corresponding to a predetermined action of the worker 400 based on the first sensing information (step S202). Then, the generating unit 664 generates a standard action model by referring to the learning result by the learning unit 663 (step S203).

[0072] In addition, in the learning by the learning unit 663, analysis may be performed on the motion capture of the movement of the worker 400, the 3D map of the workplace 200, navigation of the movement and movement of the worker 400 in the workplace 200, cornering, speed, etc., and the optimal movement of the humanoid robot 20, which can also function as a work reproduction robot, may be learned by automatic learning. This makes it possible to analyze a given movement of the worker 400 from multiple aspects at once, and reduces the time and cost required for analyzing and programming the movement of the worker 400.

[0073] Returning to FIG. 5, before and after the standard motion model is generated, the first humanoid robot 20a is placed at a predetermined position (step S103). The first humanoid robot 20a can be placed at the predetermined position, for example, by storing a floor plan of the work place 200, which is an example of the predetermined position, in a storage unit in advance, associating the position of the first humanoid robot 20a with the stored floor plan, and then operating the first moving mechanism 22a of the first humanoid robot 20a to move it to the position. Alternatively, the placement of the first humanoid robot 20a may be based on a position optimized through machine learning. The predetermined position is preferably the work place 200 that faithfully reproduces the situation when an abnormal situation occurs.

[0074] The control unit 662 refers to the standard action model and causes the first humanoid robot 20a to perform the reproduction action one or more times (step S104). In other words, the first humanoid robot 20a performs the reproduction action one or more times based on the action instruction.

[0075] Around the time when the first humanoid robot 20a is actuated (S104), the input unit 665 inputs accident or malfunction information (step S105).

[0076] The task reproduction system 100 senses the reproduction action of the first humanoid robot 20a in the work area 200 using the second robot sensor 23b (second robot imaging device 24b) of the second humanoid robot 20b (step S106). The sensing information (second sensing information) acquired by the second robot sensor 23b (second robot imaging device 24b) is stored in the storage unit.

[0077] The detection unit 666 detects the occurrence of an accident or malfunction based on the second sensing information acquired by using the second robot sensor 23b (second robot imaging device 24b) (step S107).

[0078] (Effects of the task reproduction system according to the first embodiment) According to the task reproduction system 100 of this embodiment, the task reproduction robot is made to perform a reproducing action one or more times with reference to the standard action model, and the occurrence of an accident or malfunction is detected based on the second sensing information corresponding to the reproducing action of the task reproduction robot. As a result, the task reproduction robot reproduces the action of the worker 400 at the time of the occurrence of an abnormal situation, and the occurrence of an accident or malfunction can be detected through the reproducing action of the task reproduction robot, making it easier to clarify the cause of the occurrence of the abnormal situation.

[0079] (Modification 1 of the first embodiment) FIG. 7 is an example of a flowchart showing the processing of the task reproduction system according to the first modification of the first embodiment of the present invention.

[0080] In the process of the task reproduction system according to this modified example, the processes from S101 to S104 are the same, but S105 is omitted, and step S107' is carried out as the process after S106.

[0081] In this work reproduction system, the work manual information or the schedule information of the worker 400 is stored in the memory unit, and the detection unit 666 detects the occurrence of an action different from the work manual information or the schedule information based on the second sensing information corresponding to the reproduced action of the first humanoid robot 20a acquired using the second robot sensor 23b (the second robot imaging device 24b) (S107').

[0082] The work manual information or the schedule information is information that represents the actions and the order that are essentially considered to be proper. Therefore, by comparing the reproduced action of the first humanoid robot 20a performed by referring to the standard action model with the work manual information or the schedule information, it is possible to check whether the standard action has any defects.

[0083] (Modification 2 of the first embodiment) FIG. 8 is a diagram showing an example of a task reproduction system according to the second modification of the present embodiment.

[0084] FIG. 8(a) is a diagram showing an example of a system configuration in a task reproduction system according to a second modification of the first embodiment of the present invention. This task reproduction system is characterized in that a torso sensor 23'' (torso imaging device 24'') is provided in a second humanoid robot 20' functioning as a mobile robot. Specifically, an instruction is given so that the sensing area 230' (imaging area 240') of the head sensor 23' (head imaging device 24') of the second humanoid robot 20' targets the first humanoid robot 20a, and the sensing area 230'' (imaging area 240'') of the torso sensor 23'' (torso imaging device 23'') of the second humanoid robot 20' targets the worker 400. Note that in this task reproduction system, the management control device 60 is not necessarily required as long as the first humanoid robot 20a and the second humanoid robot 20' are configured to be able to communicate with each other.

[0085] Fig. 8(b) is a diagram showing an example of the humanoid robot shown in Fig. 8(a). The second humanoid robot 20' functioning as a mobile robot includes a robot main body 21', a robot moving mechanism 22', a head sensor 23', a head image pickup device 24' included in the head sensor 23', a body sensor 23'', a body image pickup device 24'' included in the body sensor 23'', an information processing device 25', and a robot arm 26'.

[0086] The robot main body 21' includes a robot torso 211' and a robot head 212'. The robot torso 211' and the robot head 212' configure a torso / head drive mechanism 21' (see FIG. 9), and are capable of changing a sensing area 230' (imaging area 240') of the head sensor 23' (head imaging device 24') and a sensing area 230'' (imaging area 240'') of the torso sensor 23'' (torso imaging device 24'').

[0087] Since the head sensor 23' (head imaging device 24') and the torso sensor 23'' (torso imaging device 24'') are positioned at different height positions, the torso sensor 23'' (torso imaging device 24'') and the head sensor 23' (head imaging device 24') sense the movements of the worker 400 and the first robot 20a functioning as a task reproduction robot from different positions.

[0088] The configuration of the information processing device 25' is similar to that of the first information processing device 25a of the first humanoid robot 20a. The robot arm 26' is also similar to that of the first humanoid robot 20a.

[0089] FIG. 9 is a block diagram showing an example of the functions of the humanoid robot 20' in this task reproduction system. In the task reproduction system 100', the information processing device 25' includes an information processing unit 66', a communication interface 1222', and a storage device 1224', and the information processing unit 66' includes a determination unit 661', a control unit 662c', a learning unit 663c', a generation unit 664', an input unit 665', and a detection unit 666'. That is, in the task reproduction system 100', the information processing unit 66' performs the same processing as the processing unit 66 of the management control device 60. The information processing device 25' is configured to be able to communicate with the head sensor 23' (head imaging device 24'), the body sensor 23'' (head imaging device 24''), the body / head driving mechanism 21', the robot moving mechanism 22', and the arm driving mechanism 26'. In addition, the storage unit 1224' may store task manual information or process chart information.

[0090] The second humanoid robot 20' of the work reproduction system 100' is equipped with an information processing unit 66' in the information processing device 25', and the first humanoid robot 20a and the second humanoid robot 20' are configured to be able to communicate with each other, making it possible to configure a work reproduction system without the need for a management control device 60.

[0091] With reference to FIG. 8, for example, the control unit 662' of the second humanoid robot 20' instructs the torso sensor 23'' (torso image pickup device 24'') to sense the worker 400, and the head sensor 23' (head image pickup device 24'') functioning as the second sensor to sense the first humanoid robot 20a functioning as the task reproduction robot. Note that the roles of the head sensor 23' (head image pickup device 24') and the torso sensor 23'' (torso image pickup device 24'') may be reversed. In other words, a configuration may be adopted in which instructions are given so that the head sensor 23' (head image pickup device 24') senses the worker 400, and so that the torso sensor 23'' (torso image pickup device 24'') senses the first humanoid robot 20a.

[0092] The work reproduction system 100' has the worker 400 reproduce the action at the time of the occurrence of the abnormal situation, and then senses the predetermined action of the worker 400 using the torso sensor 23'' (torso image pickup device 24''). The learning unit 663' learns a standard action model corresponding to the predetermined action of the worker 400 based on the first sensing information corresponding to the predetermined action of the worker 400. The control unit 662' causes the first humanoid robot 20a to perform the reproduced action one or more times by referring to the standard action model. The input unit 665' inputs the accident or malfunction information. Then, the detection unit 666' detects the occurrence of the accident or malfunction based on the second sensing information corresponding to the reproduced action of the first humanoid robot 20a acquired using the head sensor 23' (head image pickup device 24'). As a result, the first humanoid robot 20a, which functions as a work reproduction robot, can reproduce the actions of the worker 400 when an abnormality occurs, and the occurrence of an accident or malfunction can be detected through the reproduced actions of the work reproduction robot, making it easier to clarify the cause of the abnormality.

[0093] Moreover, the work reproduction system 100' has the worker 400 reproduce the action when an abnormal situation occurs, and then senses the predetermined action of the worker 400 using the torso sensor 23'' (torso image pickup device 24''). The learning unit 663' learns a standard action model corresponding to the predetermined action of the worker 400 based on the first sensing information corresponding to the predetermined action of the worker 400. The control unit 662' causes the first humanoid robot 20a to perform the reproduction action one or more times with reference to the standard action model. The detection unit 666' detects the occurrence of an action different from the work manual information or the work schedule information based on the second sensing information corresponding to the reproduced action of the first humanoid robot 20a acquired using the head sensor 23' (head image pickup device 24'). In this way, the first humanoid robot 20a functioning as a work reproduction robot can reproduce the action of the worker 400 when an abnormal situation occurs, and defects in the standard action can be grasped through the reproduced action of the work reproduction robot.

[0094] (Effects of Modification 1) According to this task reproduction system, the humanoid robot 20' can constitute a task reproduction system by itself, and therefore, it is possible to provide a task reproduction system that makes it easy to identify the cause of an abnormality even in a place where communication with the management control device 60 is not possible.

[0095] In addition, since the life-like robot 20' is equipped with multiple (two in this modified example) sensors (imaging devices), it is possible to provide a task reproduction system that makes it easy to identify the cause of an abnormality even in a narrow space where the worker 400 and the task reproduction robot cannot be arranged in parallel to reproduce the task.

[0096] In this task reproduction system, the humanoid robot functioning as the mobile robot does not necessarily have to be one, but may be multiple. In this case, as the number of humanoid robots increases, the number of sensors increases by a multiple of the number of humanoid robots, making it possible to obtain a large amount of sensing information at one time.

[0097] Although the embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment of the present invention, and various modifications and applications are possible without departing from the gist of the present invention.

[0098] In the work reproduction system 100 of this embodiment, a configuration has been described in which one mobile robot is arranged for each of the worker 400 and the work reproduction robot. However, the number of mobile robots may be more than this. For example, if there are a large number of mobile robots equipped with sensors and moving mechanisms, it is possible to arrange multiple sensors so as to sense the movements of the worker 400 and the work reproduction robot from different positions, heights, and / or directions. This makes it easier to obtain various data required for learning the movements of the worker 400 and the work reproduction robot, and makes it possible to sense so as to cover the movements of the worker 400 and the work reproduction robot in a comprehensive manner.

[0099] In addition, in the task reproduction system 100 of this embodiment, a configuration has been described in which one mobile robot (humanoid robot) having a sensor and a moving mechanism senses both the worker 400 and the task reproduction robot. With this configuration, the mobile robot can be used for each sensing, reducing the cost and time required for robot production. However, the sensing of the worker 400 and the sensing of the task reproduction robot may be performed by different mobile robots. Furthermore, each sensing does not have to be performed by a mobile robot.

[0100] In the present embodiment, the learning of the predetermined motion of the worker is described as being performed by automatic learning. However, the learning does not necessarily have to be automatic learning, and may be other known machine learning such as deep learning, unsupervised / supervised learning, reinforcement learning, etc. [Explanation of symbols]

[0101] 100, 100', 100'' Work Reproduction System 20, 20a, 20b, 20c, 20' Humanoid robots (mobile robots, robots that reproduce tasks) 22, 22a, 22' movement mechanism 23, 23a, 23b, 23', 23'' Robot Sensors 230, 230a, 230b, 230', 230'' sensing area 24, 24a, 24b, 24', 24'' Imaging device for robots 240, 240a, 240b, 240', 240'' imaging area 25, 25a, 25' Information processing device 60 Management control device 62 Storage medium (storage unit) 1224, 1224' Storage device (storage unit)

Claims

1. A robot that reproduces work A sensor capable of sensing the operation of the task reproduction robot; A management control device capable of communicating with the task reproduction robot and a sensor, The management control device includes: a learning unit configured to learn a standard motion model corresponding to a predetermined motion of a sensing target based on first sensing information corresponding to the predetermined motion of the sensing target; A control unit that causes the task reproducing robot to perform a reproducing motion one or more times by referring to the standard motion model; an input unit for inputting accident or malfunction information; and a detection unit that detects the occurrence of the accident or malfunction based on second sensing information corresponding to the reproduced operation of the task reproduction robot acquired by using the sensor. A task reproduction system comprising:

2. A robot that reproduces work A sensor capable of sensing the operation of the task reproduction robot; A management control device capable of communicating with the task reproduction robot and a sensor, The management control device includes: a learning unit configured to learn a standard motion model corresponding to a predetermined motion of a sensing target based on first sensing information corresponding to the predetermined motion of the sensing target; A control unit that causes the task reproducing robot to perform a reproducing motion one or more times by referring to the standard motion model; A storage unit that stores work manual information or process chart information of the sensing target; and a detection unit that detects the occurrence of an operation different from the operation manual information or the process chart information based on second sensing information corresponding to a reproduction operation of the work reproduction robot acquired using the sensor. A task reproduction system comprising:

3. A robot that reproduces work A sensor capable of sensing the operation of the task reproduction robot; A management control device capable of communicating with the task reproduction robot and a sensor, The management control device includes: learning a standard motion model corresponding to a predetermined motion of a sensing target based on first sensing information corresponding to the predetermined motion of the sensing target; having the task reproducing robot perform a reproducing motion one or more times by referring to the standard motion model; Enter the accident or malfunction information, Detecting the occurrence of the accident or malfunction based on second sensing information corresponding to the reproduced operation of the task reproduction robot acquired by using the sensor. A method for reproducing a task.

4. A robot that reproduces work A sensor capable of sensing the operation of the task reproduction robot; A management control device capable of communicating with the task reproduction robot and a sensor, The management control device includes: learning a standard motion model corresponding to a predetermined motion of a sensing target based on first sensing information corresponding to the predetermined motion of the sensing target; having the task reproducing robot perform a reproducing motion one or more times by referring to the standard motion model; storing work manual information or process chart information of the sensing target; Detecting the occurrence of an action different from the work manual information or the process chart information based on second sensing information corresponding to the reproduced action of the work reproduction robot acquired by using the sensor. A method for reproducing a task.

5. A task reproduction robot, An information processing device capable of communicating with a sensor capable of sensing the operation of the task reproduction robot; The information processing device includes: a learning unit that learns a standard action model corresponding to a predetermined action of a sensing target based on first sensing information corresponding to the predetermined action of the sensing target; A control unit that causes the task reproducing robot to perform a reproducing motion one or more times by referring to the standard motion model; an input unit for inputting accident or malfunction information; and a detection unit that detects the occurrence of the accident or malfunction based on second sensing information corresponding to the reproduced operation of the task reproduction robot acquired by the sensor. A task reproduction robot characterized by the above.

6. A task reproduction robot, An information processing device capable of communicating with a sensor capable of sensing the operation of the task reproduction robot; The information processing device includes: a learning unit that learns a standard action model corresponding to a predetermined action of a sensing target based on first sensing information corresponding to the predetermined action of the sensing target; A control unit that causes the task reproducing robot to perform a reproducing motion one or more times by referring to the standard motion model; A storage unit that stores work manual information or process chart information of the sensing target; a detection unit that detects the occurrence of an operation different from the work manual information or the process chart information based on second sensing information corresponding to a reproduced operation of the work reproduction robot acquired by the sensor; A task reproduction robot characterized by the above.

Images