Operation modification system, operation modification method and working robot

Abstract

To provide an operation modification system, an operation modification method and a working robot, which enable efficient work of a working robot for working using a learning model where an operation of an operator is learned.SOLUTION: An operation modification system includes: a learning part for learning a standard operation model corresponding to a prescribed operation of a sensing object acquired using a sensor, on the basis of sensing information corresponding to the prescribed operation of the sensing object; a model creation part for creating a modification operation model in which execution time of each operation in the standard operation model is set to be shorter than required time of each of the operations at the time of standard operation model creation, by referring to the standard operation model; and a control part for operating the working robot by referring to the modification operation model.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a movement modification system, a movement modification method, and a working robot. [Background technology]

[0002] In recent years, technology has been considered for having a work robot perform work performed by humans, such as line work in a factory. For example, Patent Document 1 discloses a learning model generation method for generating an optimal learning model for each worker according to the tendency of collected data for each worker. The generated learning model can be used to operate a work robot that replaces a worker or works together with a worker. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-135178 A Summary of the Invention [Problem to be solved by the invention]

[0004] There is a certain limit to how fast a worker can work due to the limitations of their physical abilities, so if a work robot is made to perform a learning model that has learned the predetermined movements of a worker, the work robot may not be able to perform the work efficiently.

[0005] Therefore, an object of the present invention is to provide a behavior modification system, a behavior modification method, and a work robot that enable a work robot to work efficiently, the work robot working using a learning model that has learned the work of a worker. [Means for solving the problem]

[0006] The behavior modification system of the present invention comprises a work robot, a sensor, and a management control device capable of communicating with the work robot and the sensor, wherein the management control device comprises a learning unit that learns a standard behavior model corresponding to a specified behavior of the sensing target based on sensing information corresponding to the specified behavior of the sensing target obtained using the sensor, a model generation unit that refers to the standard behavior model and generates a modified behavior model in which the execution time of each behavior in the standard behavior model is set shorter than the time required for each behavior when the standard behavior model was generated, and a control unit that operates the work robot by referring to the modified behavior model.

[0007] In addition, the behavior modification system of the present invention comprises a work robot, a plurality of sensors for sensing a plurality of different sensing targets, and a management control device capable of communicating with the work robot and the plurality of sensors, wherein the management control device comprises a learning unit that learns each of the predetermined behaviors of the plurality of sensing targets and a plurality of standard behavior models corresponding to each of the predetermined behaviors of the plurality of sensing targets based on a plurality of sensing information corresponding to the predetermined behaviors of the plurality of sensing targets obtained using the plurality of sensors, a model generation unit that generates a modified behavior model that integrates at least a portion of the predetermined behaviors of the plurality of sensing targets by referring to the plurality of standard behavior models, and a control unit that operates the work robot by referring to the modified behavior model.

[0008] In addition, the movement modification method of the present invention is characterized in that it learns a standard movement model corresponding to a specified movement of a sensing target based on sensing information corresponding to the specified movement of the sensing target obtained using a sensor, generates a modified movement model by referring to the standard movement model, in which the execution time of each movement in the standard movement model is set shorter than the time required for each movement at the time the standard movement model was generated, and operates the work robot by referring to the modified movement model.

[0009] Furthermore, the work robot of the present invention is characterized in having: a drive mechanism for operating the work robot; a learning unit that learns a standard action model corresponding to a specified movement of a sensing object based on sensing information corresponding to the specified movement of the sensing object obtained using a sensor; a model generation unit that refers to the standard action model and generates a modified action model in which the execution time of each movement in the standard action model is set shorter than the time required for each movement when the standard action model was generated; and a control unit that operates the work robot to control the drive mechanism by referring to the modified action model. Effect of the Invention

[0010] According to the present invention, there is provided a behavior modification method and a working robot that enable a working robot to work efficiently, the working robot working using a learning model that has learned the work of a worker. [Brief description of the drawings]

[0011] [Figure 1] FIG. 2A is a diagram showing an example of a system configuration in a behavior modification system according to the first embodiment of the present invention, and FIG. 2B is a diagram showing an example of the humanoid robot shown in FIG. [Diagram 2] FIG. 2 is a diagram showing an example of a relationship between a standard behavior model and a behavior modification model in the behavior modification system according to the first embodiment of the present invention. [Diagram 3] 1 is a block diagram showing an example of the configuration and functions of an operation modification system according to a first embodiment of the present invention. [Figure 4] 3 is a block diagram showing an example of functions of a management control device in the operation modification system according to the first embodiment of the present invention. FIG. [Diagram 5] 3 is an example of a flowchart showing the processing of the operation modification system according to the first embodiment of the present invention. [Figure 6] 6 is an example of a flowchart showing more detailed processing of the modified action model generation processing shown in step S103 of FIG. 5. [Figure 7]FIG. 1A is a diagram showing an example of a system configuration in a behavior modification system according to a first modified example of the first embodiment of the present invention, and FIG. 1B is a diagram showing an example of the humanoid robot shown in FIG. [Figure 8] FIG. 2 is a block diagram showing an example of functions of a humanoid robot in a behavior modification system according to a first modified example of the first embodiment of the present invention. [Figure 9] 13 is an example of a flowchart showing more detailed processing of a modified action model generation process in the action modification system according to the first modified example of the first embodiment of the present invention. [Figure 10] FIG. 11 is a diagram showing an example of a relationship between a standard action model and an action modification model in the action modification system according to the first modified example of the first embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The motion modification system, the motion modification method, and the working robot will be described below with reference to the drawings. However, it should be noted that the technical scope of the present invention is not limited to the embodiments, but extends to the inventions described in the claims and their equivalents.

[0013] (Embodiment 1) FIG. 1 is a diagram for explaining the behavior modification system.

[0014] 1(a) is a diagram showing an example of a system configuration of a behavior modification system according to a first embodiment of the present invention. This behavior modification system includes a first humanoid robot 20a and a second humanoid robot 20b that function as working robots, and a third humanoid robot 20c that functions as a mobile robot. Note that the number of humanoid robots is not limited to this.

[0015] Each of the humanoid robots 20a-c moves to the vicinity of a worker 400 working on a work line 201 in the work area 200 in response to an instruction from a management control device 60 (see FIG. 3) described later or an instruction from each information processing device provided in each of the humanoid robots 20a-c. Then, the present behavior modification system senses a predetermined behavior of the worker 400 using a third robot sensor 23c (third robot imaging device 24c) provided in the third humanoid robot 20c. The predetermined behavior is diverse, and examples of the predetermined behavior include assembling parts, moving parts, painting a product, and the movement of the worker himself. In addition, when sensing the worker 400, a known image recognition technology may be used, or the worker 400 and his / her predetermined behavior may be recognized by learning by a learning unit 663 (see FIG. 4).

[0016] The present movement modification system learns a standard movement model corresponding to a predetermined movement of the worker 400 based on sensing information corresponding to the predetermined movement of the worker 400 acquired using the third robot sensor 23c (third robot imaging device 24c). The standard movement model is a model that represents the contents of the work items of the worker 400, in other words, a set of each movement specified in the work items. Then, the present movement modification system refers to the standard movement model to generate a modified movement model in which the execution time of each movement in the standard movement model is set shorter than the time required for each movement when the standard movement model was generated.

[0017] This behavior modification system operates the first humanoid robot 20a and the second humanoid robot 20b, which function as working robots, in response to instructions from the management control device 60 or instructions from an information processing device provided in each humanoid robot. At this time, this behavior modification system refers to a modified behavior model.

[0018] This behavior modification system can improve the work efficiency of each humanoid robot by operating the first humanoid robot 20a and the second humanoid robot 20b with reference to a modified behavior model set to shorten the execution time of each behavior. As an example, if there are six workers on one work line 201 in the workplace 200 and they originally completed 100 products per hour, six humanoid robots are placed on the line 201 and operated at 10 times the speed of the standard behavior model. In other words, this behavior modification system operates each humanoid robot with reference to a behavior modification model set to operate in one-tenth the execution time of the behavior in the standard behavior model. This makes it possible to complete 1,000 products per hour.

[0019] In addition, in this behavior modification system, the first humanoid robot 20a and the second humanoid robot 20b functioning as work robots may sense their respective robot behaviors using the first robot sensor 23a (first robot imaging device 24a) and the second robot sensor 23b (second robot imaging device 24b) provided therein, respectively.

[0020] As an example, the first humanoid robot 20a operates the first body / head driving mechanism 21a (see FIG. 3) in response to an instruction from the management control device 60 or an instruction from the first information processing device 25a so that the sensing area 230a (imaging area 240a) of the first robot sensor 23a (first robot imaging device 24a) targets the first gripping parts 265a, 266a of the first humanoid robot 20a. Similarly, the second humanoid robot 20b operates the second body / head driving mechanism in response to an instruction from the management control device 60 or the first information processing device 25a so that the sensing area 230b (imaging area 240b) of the second robot sensor 23b (second robot imaging device 24b) targets the second gripping parts 265b, 266b of the second humanoid robot 20b. This makes it possible to check whether the robot motion of each humanoid robot is a robot motion that refers to the modified motion model.

[0021] 1(b) is a diagram showing an example of the humanoid robot shown in (a). The humanoid robot 20, which functions as a work robot and a mobile robot, includes a robot body 21, a robot movement mechanism 22, a robot sensor 23, a robot imaging device 24 included in the robot sensor 23, an information processing device 25, and a robot arm 26.

[0022] The humanoid robot 20 can move using a robot movement mechanism 22 provided below the robot body 21, and moves to the vicinity of the work line 201 in the workplace 200 upon receiving instructions from outside the humanoid robot 20, such as from a management control device 60, or by referring to a program stored in an information processing device 25.

[0023] The robot main body 21 includes a robot torso 211 and a robot head 212. The robot torso 211 and the robot head 212 constitute a torso / head drive mechanism, and are capable of changing the sensing area 230 (imaging area 240) of the robot sensor 23 (robot imaging device 24). The configuration of the drive mechanism is not particularly limited, and may be configured such that, for example, the robot head 212 rotates a predetermined angle relative to the robot torso 211, or the robot torso 211 rotates a predetermined angle relative to the robot movement mechanism 22, by a servo motor (not shown).

[0024] A robot moving mechanism 22 is provided below the robot body 211, a robot arm 26 is provided on each side of the robot body 211, and a robot sensor 23 is provided in the robot head 212. An information processing device 25 is provided inside the robot body 21.

[0025] The robot movement mechanism 22 may have any configuration, for example, it may be provided with a rotating body driven by a motor, or may have legs that resemble the shape of a human leg. As an example, when the robot movement mechanism 22 is configured to resemble the shape of a human leg, a servo motor is provided at the location corresponding to the human joint, and the movement mechanism is configured by rotating the servo motor by a predetermined angle.

[0026] The robot sensor 23 is preferably provided in the robot head 212 and senses the robotic movements of the worker 400, other working robots, or the humanoid robot 20, preferably the robotic movements of the robot arm 26, and more preferably the robotic movements of the grippers 255 and 256. The robot sensor 23 also sequentially acquires information indicating at least the distance and angle between an object around the humanoid robot 20 on which the humanoid robot 20 works and the robot arm 26. As an example of the robot sensor 23, a high-performance camera, a thermal camera, a high-pixel, telephoto, ultra-wide-angle, 360-degree, high-performance camera, a radar, a solid-state LiDAR, a LiDAR, a multi-color laser coaxial displacement meter, vision recognition, or a variety of other sensor groups may be adopted. These are also examples of the robot imaging device 24. Other examples of the robot sensor 23 include a vibration meter, a hardness meter, a micro vibration meter, an ultrasonic measuring instrument, a vibration measuring instrument, an infrared measuring instrument, an ultraviolet measuring instrument, an electromagnetic wave measuring instrument, a thermometer, a hygrometer, spot AI weather forecasts, high-precision multi-channel GPS, low-altitude satellite information, or long-tail incident AI data.

[0027] Examples of sensing information acquired from the robot sensor 23 include images, distance, vibration, heat, smell, color, sound, ultrasonic waves, radio waves, ultraviolet rays, infrared rays, humidity, etc., and preferably the image and distance information is acquired by the robot imaging device 24. The robot sensor 23 (robot imaging device 24) performs these sensing operations, for example, every nanosecond. The sensing information is used, for example, for motion capture of the movements of the worker 400, a 3D map of the workplace 200, navigation of the movement and movements of the worker 400 in the workplace 200, analysis of cornering, speed, etc.

[0028] The robot arm 26 includes a right arm 261 and a left arm 262. The right arm 261 includes a right gripping support part 263 and a right gripping part 265, and the left arm 262 includes a left gripping support part 264 and a left gripping part 266. The right gripping support part 263 is a mechanism for supporting the right gripping part 265, and the left gripping support part 264 is a mechanism for supporting the left gripping part 266, and may be shaped like a human arm, for example. The gripping parts 265 and 266 are mechanisms for gripping parts for work, for example, and may be shaped like a human hand, for example.

[0029] The robot arm 26 constitutes an arm drive mechanism. The configuration of the drive mechanism is not particularly limited, and for example, when the robot arm 26 is made to resemble a human shape, a configuration may be adopted in which servo motors are provided at each joint location, such as a location corresponding to a human shoulder, a location corresponding to an elbow, a location corresponding to a wrist, a location corresponding to a finger joint, and the like, and rotated by a predetermined angle.

[0030] The humanoid robot 20 may further be provided with a sensor, for example, on the robot body 211 (see FIG. 7(b)). In this case, the sensor is located at a different height from the robot sensor 23 located on the robot head 212. The different height allows the sensor to sense the movements of the worker 400 from different angles.

[0031] FIG. 2 is a diagram showing an example of the relationship between a standard behavior model and a behavior modification model in this behavior modification system.

[0032] As described above, the standard action model is a model that represents a set of actions specified in the work items, and includes multiple actions. As an example, there are a total of 26 actions specified in the work items, and the actions are action A, action B, action C, action B, action C, action Z.

[0033] In contrast, the modified motion model is a model in which the execution time of each motion in the standard motion model is set shorter than the required time of each motion when the standard motion model was generated. For example, the required time of each motion in the standard motion model is set to T for motion A. A Seconds, action B to TB In this case, in the behavior modification model, the same behavior A takes T A t shorter than seconds A sec, T for action B B t shorter than seconds B Seconds are set as the execution time. The same applies to actions C to Z. This makes it possible to shorten the execution time when the working robot is executed using the modified action model compared to when the working robot is executed using the standard action model.

[0034] In the behavior modification model, it is sufficient that the execution time of at least one behavior is less than the required time of the behavior in the standard behavior model, and it is not necessary that the execution time of all the behaviors included in the standard behavior model is less than the required time of the behavior in the standard behavior model. In other words, if the execution time of at least one behavior is less than the required time of the behavior in the standard behavior model, the execution time of each behavior in the standard behavior model is set shorter than the required time of each behavior when the standard behavior model was generated.

[0035] FIG. 3 is a block diagram showing an example of the configuration and functions of the operation modification system 100 of this embodiment.

[0036] The behavior modification system 100 includes a first humanoid robot 20a, a second humanoid robot 20b, a third humanoid robot 20c, and a management control device 60. The first humanoid robot 20a, the second humanoid robot 20b, and the third humanoid robot 20c are each connected to a communication unit 64 of the management control device 60 via wireless or wired communication, and receive instructions from the management control device 60 and transmit information acquired by each sensor. The humanoid robots 20a to c may also be connected to each other via wireless or wired communication, and transmit and receive information acquired by each sensor and instructions.

[0037] The first humanoid robot 20a functioning as a working robot includes a first body / head driving mechanism 21a, a first robot moving mechanism 22a, a first robot sensor 23a, a first robot imaging device 24a included in the first robot sensor 23a, a first information processing device 25a, and a first arm driving mechanism 26a. In this embodiment, the second humanoid robot 20b functioning as a working robot and the third humanoid robot 20c functioning as a mobile robot also have the same configuration as the first humanoid robot 20a.

[0038] The first information processing device 25a according to this embodiment includes a central processing unit (CPU) 1212, a random access memory (RAM) 1214, and a graphic controller 1216, which are connected to each other by a host controller 1210. The first information processing device 25a also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid state drive, or the like. The first information processing device 25a also includes input / output units such as a read only memory (ROM) 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0039] The CPU 1212 operates according to a program stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into itself, and causes the image data to be displayed on the display device 1218.

[0040] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the first information processing device 25a. The storage device 1224 may also store sensing information. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0041] The ROM 1230 stores therein a boot program, etc., executed by the first information processing device 25a upon activation, and / or a program that depends on the hardware of the first information processing device 25a. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0042] The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the first information processing device 25a, and brings about cooperation between the programs and the above-mentioned various types of hardware resources. An apparatus or method may be configured by realizing an operation or processing of information according to the use of the first information processing device 25a.

[0043] For example, when communication is performed between the first information processing device 25a and an external device, the CPU 1212 may execute a communication program loaded in the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0044] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0045] Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and undergo information processing. The CPU 1212 may execute various types of processing on data read from the RAM 1214, including various types of operations, information processing, conditional decisions, conditional branches, unconditional branches, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write back the results to the RAM 1214. The CPU 1212 may also search for information in files, databases, etc., in the recording medium.

[0046] The above-described program or software module may be stored in a computer-readable storage medium on the first information processing device 25a or in the vicinity of the first information processing device 25a. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable storage medium, thereby providing the program to the first information processing device 25a via the network.

[0047] The blocks in the flowcharts and diagrams in this embodiment may represent stages of a process in which operations are performed or "parts" of an apparatus responsible for performing the operations. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer readable instructions stored on a computer readable storage medium, and / or a processor provided with computer readable instructions stored on a computer readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as, for example, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like, including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0048] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray disks, memory sticks, integrated circuit cards, and the like.

[0049] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0050] Computer readable instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, either locally or over a local area network (LAN), a wide area network (WAN), such as the Internet, etc., to cause the processor of the general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, to execute the computer readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0051] The contents described so far also apply to the information processing devices provided in the second humanoid robot 20b and the third humanoid robot 20c.

[0052] The management control device 60 is a control device that issues instructions to each of the humanoid robots 20a to 20c in order to realize the behavior modification system 100. In addition, the management control device 60 acquires sensing information accumulated in the storage device of each information processing device.

[0053] The management control device 60 is composed of a CPU 60A, a RAM 60B, a ROM 60C, an input / output unit (I / O) 60D, a bus 60E such as a data bus or a control bus connecting these, and a communication unit 64. A storage medium 62 is connected to the I / O 60D.

[0054] In addition, a communication unit 64 is connected to the I / O 60D, which transmits and receives sensing information, work manual information, and schedule information between the control system of the humanoid robot 20. The work manual information includes, for example, the name and content of each work item, the order of the work items, and information on the standard work time required for each work item. In addition, the schedule information includes, for example, information indicating the work time and start / end times of the entire work, information indicating the work time and start / end times of each work item, and information indicating the worker for each work item.

[0055] FIG. 4 is a block diagram showing an example of the functions of the management control device 60 in the operation modification system of this embodiment.

[0056] The management control device 60 includes a storage medium 62 , a communication unit 64 , and a processing unit 66 .

[0057] The storage medium 62 includes, for example, at least one of a semiconductor storage device, a magnetic tape device, a magnetic disk device, and an optical disk device. The storage medium 62 stores a driver program, an operating system program, an application program, data, and the like used for processing in the processing unit 66. For example, the storage medium 62 stores sensing information. The storage medium 62 also stores work manual information and / or process chart information for the worker 400.

[0058] The communication unit 64 has a wireless communication interface circuit such as Wi-Fi (registered trademark) and / or a wired communication interface circuit such as Ethernet (registered trademark). The communication unit 64 transmits and receives various information to and from the humanoid robots 20a and 20b through the interface circuits.

[0059] The processing unit 66 has one or more processors and their peripheral circuits. The processing unit 66 centrally controls the overall operation of the operation modification system 100, and is, for example, a CPU. The processing unit 66 executes processing by referring to programs (driver programs, operating system programs, application programs, etc.) stored in the storage medium 62. The processing unit 66 can also execute multiple programs (application programs, etc.) in parallel.

[0060] The processing unit 66 includes a determination unit 661, a control unit 662, a learning unit 663, and a model generation unit 664. Each of these units is a functional module realized by a program executed by a processor included in the processing unit 66. Alternatively, each of these units may be implemented in the processing unit 66 as firmware.

[0061] When there are multiple sensing targets and multiple sensors that sense the targets, the determination unit 661 determines whether the multiple sensors are sensing different targets. For this determination, a known image recognition technique may be used, or a method of referring to learning by the learning unit 663 may be used.

[0062] The control unit 662 operates the first humanoid robot 20a and / or the second humanoid robot 20b, which function as working robots, by referring to the modified motion model generated by the model generation unit 664. Preferably, the control unit 662 operates the robot movement mechanism, the torso / head drive mechanism, and / or the arm drive mechanism of each humanoid robot by referring to the modified motion model.

[0063] The learning unit 663 learns a standard action model corresponding to the predetermined action of the worker 400 based on sensing information corresponding to the predetermined action of the worker 400 acquired using the third robot sensor 23c (third robot imaging device 24c). This learning is performed by automatic learning, which is learning in which a learned model is automatically created and judgment / analysis is automatically performed using the learned model. Note that the learning unit 663 may refer to work manual information and / or work schedule information when generating the standard action model. This makes it possible to have each humanoid robot 20 perform an appropriate action (task) without reflecting any inappropriate predetermined action of the worker 400.

[0064] The model generation unit 664 references the standard behavior model and generates a modified behavior model in which the execution time of each behavior in the standard behavior model is set shorter than the time required for each behavior when the standard behavior model was generated.

[0065] (Processing of the operation modification system according to the first embodiment of the present invention) FIG. 5 is an example of a flowchart showing the processing of the operation modification system of this embodiment.

[0066] First, the third information processing device of the third humanoid robot 20c instructs the third humanoid robot 20c, which functions as a mobile robot, to move to the workshop 200 (step S101) by an instruction from the management control device 60 or an instruction to read a program stored in the storage medium 62 or the storage device of the third information processing device of the third humanoid robot 20c. The movement is caused by the operation of the third robot movement mechanism of the humanoid robot 20c. Note that a movement instruction may also be given at this time to the humanoid robots 20a, 20b, which function as working robots.

[0067] When moving, an instruction is given so that the sensing area 230c (imaging area 240c) of the third robot sensor 23c (third robot imaging device 24c) of the third humanoid robot 20c targets the worker 400. The placement of the third humanoid robot 20c is performed, for example, by storing a floor plan of the workplace 200 in the storage device and / or storage medium 62 of the third humanoid robot 20c in advance and associating the position of the third humanoid robot 20c with the stored floor plan. Alternatively, the placement of the third humanoid robot 20c may be based on a position optimized through machine learning. The same applies to the placement of the first humanoid robot 20a and the second humanoid robot 20b.

[0068] Next, a predetermined movement of the worker 400 on the work line 201 is sensed by the third robot sensor 23c (third robot imaging device 24c) (step S102). In this embodiment, the control unit 662 issues an instruction to set the sensing area 230c (imaging area 240c) of the third robot sensor 23c (third robot imaging device 24c) to target the worker 400, and the third robot movement mechanism and each driving mechanism of the third humanoid robot 20c are operated.

[0069] The sensing information acquired by the third robot sensor 23c (third robot imaging device 24c) is stored in the storage medium 62 via the storage device and / or communication unit 64 of the third humanoid robot 20c. The storage device and storage medium 62 of each humanoid robot function as a storage unit.

[0070] The management control device 60 learns a standard action model corresponding to a predetermined action of the worker 400 based on the sensing information accumulated in the memory unit, in other words, stored, and generates a modified action model by referring to the standard action model, in which the execution time of each action in the standard action model is set shorter than the required time of each action when the standard action model was generated (step S103). As an example, if the required time of one action in the standard action model is 10 seconds, the execution time of the action is set to 5 seconds in the modified action model.

[0071] The control unit 662 operates the first humanoid robot 20a and the second humanoid robot 20b that function as working robots by referring to the generated modified motion model (step S104). By referring to the generated modified motion model, the first humanoid robot 20a and the second humanoid robot 20b can perform the work (predetermined motion) of the worker 400 faster than the predetermined motion. For example, according to the example shown above, the modified motion model is set to perform the motion in half the execution time of the motion in the standard motion model, so that each humanoid robot 20 can perform the work twice as fast as the predetermined motion of the worker 400.

[0072] FIG. 6 is an example of a flowchart showing more detailed processing of the modified motion model processing shown in step S103 of FIG.

[0073] The sensing information is stored in the memory unit (step S201), and the learning unit 663 learns a standard action model corresponding to the specified movement of the worker 400 based on the sensing information corresponding to the specified movement of the worker 400 acquired using the third robot sensor 23c (third imaging device 24c) (step S202).

[0074] The learning unit 663 generates a standard action model based on the learning result (step S203). Note that the learning unit 663 may refer to the work manual information and / or the work schedule information of the worker 400 when generating the standard action model.

[0075] The model generation unit 664 generates a modified motion model by referring to the standard motion model (step S204). The modified motion model is a model in which the execution time of each motion in the standard motion model is set shorter than the time required for each motion when the standard motion model was generated, so that the robot motion of each humanoid robot 20a, 20b operated by referring to the modified motion model is faster than the robot motion of the same robot operated by referring to the standard motion model.

[0076] (Effects of the behavior modification system according to the first embodiment) According to the movement modification system 100 of this embodiment, the work robot can be operated by referring to a modified movement model that is set to be shorter than the time required for each movement when the standard movement model is generated, thereby enabling the work robot to work efficiently.

[0077] Moreover, according to the action modification system 100 of this embodiment, when generating a standard action model, the work manual information and / or the schedule information are referred to, and then the standard action model is generated. The worker 400 does not always perform actions faithful to the work, and in some cases may perform unnecessary actions or omit necessary actions. Therefore, by referring to the work manual information and the schedule information, it is possible to prevent unnecessary or inappropriate predetermined actions by the worker 400 from being reflected in the standard action model.

[0078] (Modification 1 of the first embodiment) FIG. 7 is a diagram showing an example of an operation modification system according to the first modification of the present embodiment.

[0079] FIG. 7(a) is a diagram showing an example of a system configuration of a movement modification system according to a first modification of the first embodiment of the present invention. One of the features of this movement modification system is that it senses and learns each of the predetermined movements of a plurality of sensing targets (workers 400a, 400b). Another feature is that a torso sensor 23'' (torso image capture device 24'') is provided on the humanoid robot 20' that functions as a mobile robot and a working robot. In addition, this movement modification system does not necessarily require a management control device 60, and the movement modification system can be configured by the humanoid robot 20' alone.

[0080] Fig. 7(b) is a diagram showing an example of the humanoid robot shown in Fig. 7(a). The humanoid robot 20' functioning as a mobile working robot includes a robot main body 21', a robot moving mechanism 22', a head sensor 23', a head image pickup device 24' included in the head sensor 23', a body sensor 23'', a body image pickup device 24'' included in the body sensor 23'', an information processing device 25', and a robot arm 26'.

[0081] The robot main body 21' includes a robot torso 211' and a robot head 212'. The robot torso 211' and the robot head 212' configure a torso / head drive mechanism 21' (see FIG. 8), and are capable of changing a sensing area 230' (imaging area 240') of the head sensor 23' (head imaging device 24') and a sensing area 230'' (imaging area 240'') of the torso sensor 23'' (torso imaging device 24'').

[0082] Since the head sensor 23' (head imaging device 24') and the torso sensor 23'' (torso imaging device 24'') are positioned at different height positions, the torso sensor 23'' (torso imaging device 24'') senses the specified movements of each sensing target (workers 400a, 400b) from a different position than the head sensor 23' (head imaging device 24').

[0083] FIG. 8 is a block diagram showing an example of the functions of a humanoid robot in this movement modification system. In the movement modification system 100', the information processing device 25' includes an information processing unit 66', a communication interface 1222', and a storage device 1224', and the information processing unit 66' includes a determination unit 661', a control unit 662', a learning unit 663', and a model generation unit 664'. That is, in the movement modification system 100', the information processing unit 66' performs the same processing as the processing unit 66 of the management control device 60. The information processing device 25' is configured to be able to communicate with the head sensor 23' (head imaging device 24'), the body sensor 23'' (head imaging device 24''), the body / head driving mechanism 21', the robot moving mechanism 22', and the arm driving mechanism 26'.

[0084] In the humanoid robot 20' of the movement modification system 100', an information processing device 25' is provided with an information processing section 66', and therefore the humanoid robot 20' alone constitutes a movement modification system.

[0085] With reference to FIG. 7(a), for example, the control unit 662' of the humanoid robot 20' instructs the torso sensor 23'' (torso image pickup device 24'') to sense the worker 400a, and the head sensor 23' (head image pickup device 24') to sense the worker 400b. In other words, the present movement modification system 100' is provided with a plurality of sensors for sensing a plurality of different sensing targets, and obtains a plurality of pieces of sensing information corresponding to a plurality of predetermined movements of the workers obtained using the plurality of sensors. Note that the roles of the head sensor 23' (head image pickup device 24') and the torso sensor 23'' (torso image pickup device 24'') may be reversed. Furthermore, when each sensor performs sensing, the determination unit 661' may determine whether each sensor is sensing a different target or not.

[0086] FIG. 9 is an example of a flowchart showing more detailed processing of a modified action model generation process in the action modification system according to the first modification of the first embodiment of the present invention.

[0087] This movement modification system is different from the first embodiment in that sensing of predetermined movements of multiple workers is performed by multiple sensors in S102. In addition, this movement modification system includes step S103' instead of S103.

[0088] In step S103', first, each piece of sensing information acquired by the head sensor 23' (head image capture device 24') and the body sensor 23'' (body image capture device 24'') is stored in the storage unit (storage device 1224') (step S201'). The learning unit 663' learns each of the predetermined actions of multiple workers (two in this modified example) based on multiple pieces of sensing information (two in this modified example) (step S202').

[0089] The learning unit 663' also learns a plurality of standard action models (two in this modification) corresponding to the respective predetermined actions of the plurality of workers based on the plurality of pieces of sensing information (step S203'). For example, if the work of worker 400a is made up of actions A to M, while the work of worker 400b is made up of actions N to Z, the standard action models become a first standard action model made up of actions A to M and a second standard action model made up of actions N to Z. When generating the standard action models, the learning unit 663' may refer to work manual information and / or process chart information.

[0090] The model generation unit 664' generates a modified action model that integrates at least a portion of the predetermined actions by a plurality of workers (step S204').

[0091] FIG. 10 is a diagram showing an example of a relationship between a standard behavior model and a behavior modification model in the behavior modification system according to the first modification of the first embodiment of the present invention.

[0092] For example, in a first standard motion model consisting of motions A to M, motion M is a motion of movement for handing over a certain part. Similarly, in a second standard motion model consisting of motions N to Z, motion N is a motion of movement for receiving the part. In this case, motions M and N are unnecessary when performed by a single working robot. Therefore, the model generating unit 664' integrates motions A to L, which are part of the motions in the first standard motion model, and motions O to Z, which are part of the motions in the second standard motion model, to generate a modified motion model consisting of motions A to L and motions O to Z. As a result, in the motion modification system 100, when operating a humanoid robot 20' that also functions as a working robot, it is possible to make a single humanoid robot 20' perform predetermined motions that have been performed by multiple workers by referring to the generated modified motion model, while omitting unnecessary motions as necessary. As a result, it is possible to make the working robot work efficiently.

[0093] As an example of this behavior modification system, if six workers initially complete 100 products per hour on one line 201 in the workplace 200, then by disposing three humanoid robots 20' on the line 201, it becomes possible to complete 100 products per hour. In particular, as described above, when multiple workers perform different tasks, the humanoid robot 20' can perform those tasks collectively, omitting unnecessary actions as necessary.

[0094] The model generation unit 664' may generate a second modified motion model for the modified motion model in which the execution time of each motion is set shorter than the time required for each motion when the modified motion model was generated. In this case, in the example shown above, the humanoid robot 20' can be operated by referring to the second modified motion model set to perform the motion in half the execution time of the motion when the modified motion model was generated, thereby completing 200 products per hour.

[0095] (Effects of Modification 1) According to this behavior modification system, the humanoid robot 20' can constitute a behavior modification system by itself, so that even in a place where communication with the management control device 60 is not possible, for example, the working robot can work efficiently by using a learning model that has learned the work of a worker.

[0096] In addition, since the self-acting robot 20' is equipped with multiple (two in this modified example) sensors (imaging devices), it is possible to perform work learning for workers even in a place that is too small to sense multiple workers, for example.

[0097] Furthermore, according to this motion modification system, multiple standard motion models corresponding to the respective predetermined motions of multiple workers are learned, and a modified motion model is generated by integrating at least some of the predetermined motions of multiple workers by referring to the standard motion models, so that the work (predetermined motions) of multiple workers can be substituted by a smaller number of work robots than the number of workers, thereby improving work efficiency.In addition, a second modified model is generated in which the execution time of each motion in the modified motion model is set shorter than the time required for each motion at the time of generating the modified motion model, and the second modified model is referred to in operating the work robot, making it possible to make the work robot work more efficiently.

[0098] In this behavior modification system, the humanoid robot that functions as both a mobile robot and a work robot does not have to be one as shown in the example, but may be multiple. In this case, as the number of humanoid robots increases, the number of sensors increases by a multiple of the number of humanoid robots, making it possible to obtain more sensing information at one time, and since the number of work robots also increases, it becomes possible to improve work efficiency, for example, when making each work robot perform the same work.

[0099] Although the embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment of the present invention, and various modifications and applications are possible without departing from the gist of the present invention.

[0100] In the description of the movement modification system 100 of this embodiment, the number of mobile robots (humanoid robots) sensing the worker 400 is one. However, the number of mobile robots may be more than one. For example, if there are a large number of mobile robots equipped with sensors and moving mechanisms, it is possible to arrange multiple sensors so as to sense the predetermined movement of the worker 400 from different positions, heights, and / or directions. This makes it easier to obtain various data required for learning the predetermined movement of the worker 400, and makes it possible to sense the predetermined movement of the worker 400 so as to cover the entirety of each predetermined movement of the worker 400.

[0101] In the present embodiment, the standard action model is generated (S103) after sensing (S102) of the worker 400. However, the process from sensing to generation of the standard action model does not necessarily have to be performed continuously. For example, when sensing is performed in S102, sensing information is stored (S201), and learning with reference to the sensing information (S202) may be performed after a predetermined time (24 hours or one week, etc.) has elapsed since the sensing. The same applies to the steps between S101 to S105. Also, contrary to the above example, S103 to S105 may be performed while S102 is being performed. In this case, the working robot is operated with reference to the modified action model while the worker 400 is performing a predetermined action, so that the work efficiency can be further improved.

[0102] In the present embodiment, the learning of the predetermined motion of the worker is described as being performed by automatic learning. However, the learning does not necessarily have to be automatic learning, and may be other known machine learning such as deep learning, unsupervised / supervised learning, reinforcement learning, etc.

[0103] In the present embodiment, the mobile robot and the working robot are described as being the same humanoid robot. In this case, it is possible to use the mobile robot as a working robot, and it is possible to save on the costs involved in manufacturing the robot. However, the mobile robot and the working robot may be different robots.

[0104] In the present embodiment, an example of a modified behavior model that is set to operate in half the execution time or one-tenth the execution time of the behavior in the standard behavior model has been described. However, the present behavior modification system is not particularly limited as long as the execution time of each behavior in the standard behavior model is set shorter than the time required for each behavior when the standard behavior model was generated. [Explanation of symbols]

[0105] 100, 100', 100'' Movement Modification System 20, 20a, 20b, 20c, 20' Humanoid robots (mobile robots, working robots) 23, 23a, 23b, 23c, 23', 23'' Robot Sensors 230, 230a, 230b, 230c, 230', 230'' sensing area 24, 24a, 24b, 24c, 24', 24'' Imaging device for robot 240, 240a, 240b, 240c, 240', 240'' imaging area 25, 25a, 25' Information processing device 60 Management control device 62 Storage medium (storage unit) 1224, 1224' Storage device (storage unit)

Claims

1. A working robot, A sensor; A management control device capable of communicating with the work robot and the sensor, The management control device includes: a learning unit that learns a standard motion model corresponding to a predetermined motion of a sensing target based on sensing information corresponding to a predetermined motion of the sensing target acquired by the sensor; a model generation unit that references the standard behavior model and generates a modified behavior model in which an execution time of each behavior in the standard behavior model is set shorter than a required time of each behavior when the standard behavior model was generated; a control unit that operates the working robot by referring to the modified motion model. A behavior modification system comprising:

2. The management control device further includes a storage unit that stores work manual information or process chart information of the sensing target, The learning unit generates the standard operation model by referring to the sensing information and work manual information or work schedule information of the sensing target. The behavior modification system of claim 1 .

3. A working robot, A plurality of sensors for sensing a plurality of different sensing targets, a management control device capable of communicating with the work robot and the plurality of sensors; The management control device includes: a learning unit that learns each of the predetermined movements of the plurality of sensing targets and a plurality of standard action models corresponding to each of the predetermined movements of the plurality of sensing targets based on a plurality of sensing information corresponding to the predetermined movements of the plurality of sensing targets acquired using the plurality of sensors; a model generation unit that generates a modified action model by integrating at least a part of predetermined actions by the plurality of sensing subjects with reference to the plurality of standard action models; a control unit that operates the working robot by referring to the modified motion model. A behavior modification system comprising:

4. A plurality of the work robots are provided, The control unit operates the plurality of work robots. A behavior modification system according to any one of claims 1 to 3.

5. learning a standard motion model corresponding to a predetermined motion of a sensing target based on sensing information corresponding to the predetermined motion of the sensing target obtained by using a sensor; generating a modified behavior model by referring to the standard behavior model, the execution time of each behavior in the standard behavior model being set shorter than the required time of each behavior at the time of generating the standard behavior model; Operate the working robot by referring to the modified motion model. A method for modifying behavior comprising:

6. A work robot, a drive mechanism for operating the working robot; a learning unit that learns a standard motion model corresponding to a predetermined motion of a sensing target based on sensing information corresponding to the predetermined motion of the sensing target obtained by using a sensor; a model generation unit that references the standard behavior model and generates a modified behavior model in which an execution time of each behavior in the standard behavior model is set shorter than a required time of each behavior when the standard behavior model was generated; a control unit that operates the working robot to control the drive mechanism by referring to the modified motion model; A working robot comprising:

Images