Sensing system, sensing method and mobile robot

Abstract

To solve the problem that it is impossible to sufficiently sense predetermined operation of a worker.SOLUTION: A sensing system according to the present invention includes: a first sensor for sensing predetermined operation of a sensing target; a second sensor for sensing the predetermined operation of the sensing target from a position different from that of the first sensor; a first mobile robot including the first sensor and a first moving mechanism; and a management control device capable of communicating with the first sensor, the second sensor, and the first moving mechanism. The management control device comprises: a determination unit configured to determine, from first information acquired by the first sensor and second information acquired by the second sensor, whether or not a predetermined part of the sensing target that is movable during the predetermined operation is sensed; and a control unit configured to operate the first moving mechanism so that the predetermined part is sensed when the determination unit determines that the predetermined part is not sensed.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a sensing system, a sensing method, and a mobile robot. [Background technology]

[0002] In recent years, technology has been considered for having a work robot perform human tasks, such as line work in a factory. In order to have a work robot perform human tasks, it is necessary, as a prerequisite, to detect (sense) the human task with a sensor and understand the motion of the human. For example, Patent Document 1 discloses a task estimation device that generates a work area image Ic from a cell camera 30 and an overall image Ia from a ceiling camera 80, and performs image analysis. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-113042 A Summary of the Invention [Problem to be solved by the invention]

[0004] The work of a worker may involve, for example, movement or large movements. In the work estimation device of Patent Document 1, since each sensor (camera) is fixedly arranged, the work (predetermined movement) of the worker may not be sensed sufficiently depending on the relationship between the worker and the sensor position.

[0005] Therefore, an object of the present invention is to provide a sensing system, a sensing method, and a mobile robot that can adequately sense a specific motion of a worker. [Means for solving the problem]

[0006] The sensing system of the present invention comprises a first sensor for sensing a predetermined movement of a sensing target, a second sensor for sensing a predetermined movement of the sensing target from a position different from the first sensor, a first mobile robot comprising the first sensor and a first moving mechanism, and a management control device capable of communicating with the first sensor, the second sensor, and the first moving mechanism, wherein the management control device has a determination unit that determines whether a predetermined part that moves when the sensing target makes a predetermined movement is sensed based on first information acquired by the first sensor and second information acquired by the second sensor, and a control unit that operates the first moving mechanism so that the predetermined part is sensed when the determination unit determines that the predetermined part is not sensed.

[0007] Furthermore, the sensing method of the present invention comprises a first mobile robot having a first sensor for sensing a predetermined movement of a sensing object, a second sensor for sensing the predetermined movement of the sensing object from a position different from the first sensor, the first sensor, and a first moving mechanism, and a management control device capable of communicating with the first sensor, the second sensor, and the first moving mechanism, wherein the management control device determines whether or not a predetermined part that moves when the sensing object performs a predetermined movement is sensed from first information acquired by the first sensor and second information acquired by the second sensor, and when it is determined by the determination unit that the predetermined part is not sensed, operates the first moving mechanism so that the predetermined part is sensed.

[0008] The sensing system of the present invention further comprises a first sensor for sensing a predetermined movement of a sensing target, a second sensor for sensing a predetermined movement of the sensing target from a position different from the first sensor, a first mobile robot having the first sensor and a first moving mechanism, and a management control device capable of communicating with the first sensor, the second sensor, and the first moving mechanism, wherein the management control device has a determination unit that determines whether a predetermined part of the sensing target that moves during a predetermined movement is sensed from first information acquired by the first sensor and second information acquired by the second sensor, and determines whether the predetermined part sensed by the first sensor and the predetermined part sensed by the second sensor are identical, and a control unit that, when it is determined that the predetermined part sensed by the first sensor and the predetermined part sensed by the second sensor are identical, operates the first moving mechanism so that the predetermined part sensed by the first sensor and the predetermined part sensed by the second sensor are different.

[0009] In addition, a mobile robot according to the present invention comprises a moving mechanism, a first sensor that senses a sensing target, a second sensor that senses the sensing target from a position different from that of the first sensor, a drive mechanism capable of moving the position of the second sensor, and an information processing unit that controls the first sensor, the second sensor, the moving mechanism, and the drive mechanism, wherein the information processing unit has a determination unit that determines whether a specified part of the sensing target that moves during a specified operation is sensed based on first information acquired by the first sensor and second information acquired by the second sensor, and a control unit that operates the moving mechanism or the drive mechanism so that the specified part is sensed when the determination unit determines that the specified part is not sensed.

[0010] Moreover, a mobile robot according to the present invention comprises a moving mechanism, a first sensor that senses a sensing target, a second sensor that senses the sensing target from a position different from that of the first sensor, a drive mechanism capable of moving the position of the second sensor, and an information processing unit that controls the first sensor, the second sensor, the moving mechanism, and the drive mechanism, wherein the information processing unit has a determination unit that determines whether a predetermined part of the sensing target that moves during a predetermined operation is sensed from first information acquired by the first sensor and second information acquired by the second sensor, and determines whether the predetermined part sensed by the first sensor and the predetermined part sensed by the second sensor are identical, and a control unit that, when it is determined that the predetermined part sensed by the first sensor and the predetermined part sensed by the second sensor are identical, operates the moving mechanism or the drive mechanism so that the predetermined part sensed by the first sensor and the predetermined part sensed by the second sensor are different. Effect of the Invention

[0011] According to the present invention, there are provided a sensing system, a sensing method, and a mobile robot that are capable of sufficiently sensing a predetermined motion of a worker. [Brief description of the drawings]

[0012] [Figure 1] FIG. 2A is a diagram showing an example of a system configuration in a sensing system according to a first embodiment of the present invention, and FIG. 2B is a diagram showing an example of the mobile robot shown in FIG. [Diagram 2] FIG. 2 is a diagram showing an example of a state in which a mobile robot moves in the sensing system according to the first embodiment of the present invention. [Diagram 3] 1 is a block diagram showing an example of the configuration and functions of a sensing system according to a first embodiment of the present invention. [Figure 4] 2 is a block diagram showing an example of the functions of a management control device in the sensing system of the first embodiment of the present invention. FIG. [Diagram 5] 4 is an example of a flowchart showing processing of the sensing system according to the first embodiment of the present invention. [Figure 6] 6 is an example of a flowchart showing more detailed processing of the predetermined part sensing determination processing shown in step S103 of FIG. 5. [Figure 7] 6 is an example of a flowchart showing more detailed processing of the motion information generating process shown in step S104 of FIG. 5. [Figure 8] FIG. 2A is a diagram showing an example of a system configuration in a sensing system according to a first modified example of the first embodiment of the present invention, and FIG. 2B is a diagram showing an example of the mobile robot shown in FIG. [Figure 9] FIG. 11 is a block diagram showing an example of functions of a mobile robot in a sensing system according to a first modified example of the first embodiment of the present invention. [Figure 10] FIG. 13A is a diagram showing an example of a system configuration in a sensing system according to a second modification of the first embodiment of the present invention, and FIG. 13B is a diagram showing an example of a sensor mounting member shown in FIG. [Figure 11] FIG. 11 is a diagram showing an example of the configuration and functions of a sensing system according to a second modification of the first embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The sensing system, the sensing method, and the mobile robot will be described below with reference to the drawings. However, it should be noted that the technical scope of the present invention is not limited to the embodiments, but extends to the inventions described in the claims and their equivalents.

[0014] (Embodiment 1) FIG. 1 is a diagram for explaining the sensing system.

[0015] 1(a) is a diagram showing an example of a system configuration of a sensing system according to a first embodiment of the present invention. The sensing system includes a first humanoid robot 20a and a second humanoid robot 20b that function as mobile robots. The number of humanoid robots is not limited to two.

[0016] Each humanoid robot 20a, 20b moves to the vicinity of a worker 400 working on a work line 201 in a work site 200 in response to an instruction from a management control device 60 (see FIG. 3) described later or an instruction from each information processing device 25a, 25b (see FIG. 3) provided in each humanoid robot 20a, 20b. The sensing system senses a predetermined movement of the worker 400 by a first sensor 23a (first imaging device 24a) provided in the first humanoid robot 20a and a second sensor 23b (second imaging device 24b) provided in the second humanoid robot 20b. The second sensor 23b (second imaging device 24b) senses the predetermined movement of the worker 400 from a position different from that of the first sensor 23a (first imaging device 24a).

[0017] 1(b) is a diagram showing an example of the mobile robot shown in (a). A humanoid robot 20 functioning as a mobile robot includes a robot body 21, a robot movement mechanism 22, a robot sensor 23, a robot imaging device 24 included in the robot sensor 23, an information processing device 25, and a robot arm 26.

[0018] The humanoid robot 20 can move using a robot movement mechanism 22 provided below the robot body 21, and moves to the vicinity of the work line 201 in the workplace 200 upon receiving instructions from outside the humanoid robot 20, such as from a management control device 60, or by referring to a program stored in an information processing device 25.

[0019] The robot main body 21 includes a robot trunk 211 and a robot head 212. The robot trunk 211 and the robot head 212 form a first drive mechanism, and are capable of changing the sensing area 230 (imaging area 240) of the robot sensor 23 (robot imaging device 24). The configuration of the drive mechanism is not particularly limited, and may be configured such that, for example, the robot head 212 rotates a predetermined angle relative to the robot trunk 211, or the robot trunk 211 rotates a predetermined angle relative to the robot movement mechanism 22, by a servo motor (not shown).

[0020] A robot moving mechanism 22 is provided below the robot body 211, a robot arm 26 is provided on each side of the robot body 211, and a robot sensor 23 is provided in the robot head 212. An information processing device 25 is provided inside the robot body 21.

[0021] The robot movement mechanism 22 may have any configuration, for example, it may be provided with a rotating body driven by a motor, or may have legs that resemble the shape of a human leg. As an example, when the robot movement mechanism 22 is configured to resemble the shape of a human leg, a servo motor is provided at the location corresponding to the human joint, and the movement mechanism is configured by rotating the servo motor by a predetermined angle.

[0022] The robot sensor 23 is preferably provided in the robot head 212 and senses the worker 400. The robot sensor 23 also sequentially acquires information indicating at least the distance and angle between an object around the humanoid robot 20 on which the humanoid robot 20 works and the robot arm 26. As an example of the robot sensor 23, a highest performance camera, a thermal camera, a high pixel, telephoto, ultra-wide angle, 360 degree, high performance camera, a radar, a solid-state LiDAR, a LiDAR, a multi-color laser coaxial displacement meter, a vision recognition, or a group of various other sensors may be adopted. These are also examples of the robot imaging device 24. Other examples of the robot sensor 23 include a vibration meter, a hardness meter, a micro vibration meter, an ultrasonic measuring instrument, a vibration measuring instrument, an infrared measuring instrument, an ultraviolet measuring instrument, an electromagnetic wave measuring instrument, a thermometer, a hygrometer, a spot AI weather forecast, a high-precision multi-channel GPS, low altitude satellite information, or long tail incident AI data.

[0023] Examples of sensor information acquired from the robot sensor 23 include images, distance, vibration, heat, smell, color, sound, ultrasonic waves, radio waves, ultraviolet rays, infrared rays, humidity, etc., and the image and distance information is preferably acquired by the robot imaging device 24. The robot sensor 23 (robot imaging device 24) performs these sensing operations, for example, every nanosecond. The sensor information is used, for example, for motion capture of the movements of the worker 400, a 3D map of the workplace 200, navigation of the movement and movements of the worker 400 in the workplace 200, analysis of cornering, speed, etc.

[0024] The robot arm 26 includes a right arm 261 and a left arm 262. The right arm 261 includes a right gripping support part 263 and a right gripping part 265, and the left arm 262 includes a left gripping support part 264 and a left gripping part 266. The right gripping support part 263 is a mechanism for supporting the right gripping part 265, and the left gripping support part 264 is a mechanism for supporting the left gripping part 266, and may be shaped like a human arm, for example. The gripping parts 265 and 266 are mechanisms for gripping parts for work, for example, and may be shaped like a human hand, for example.

[0025] The robot arm 26 constitutes a second drive mechanism. The configuration of the drive mechanism is not particularly limited, and for example, when the robot arm 26 is made to resemble a human shape, a configuration may be adopted in which servo motors are provided at each joint location, such as a location corresponding to a human shoulder, a location corresponding to an elbow, a location corresponding to a wrist, a location corresponding to a finger joint, and the like, and rotated by a predetermined angle.

[0026] The humanoid robot 20 may further be provided with a sensor, for example, on the robot body 211 (see FIG. 8(b)). In this case, the sensor is located at a different height from the robot sensor 23 located on the robot head 212. The different height allows the sensor to sense the movements of the worker 400 from different angles.

[0027] Returning to FIG. 1(a), when the first sensor 23a (first imaging device 24a) and the second sensor 23b (second imaging device 24b) which are the robot sensors (robot imaging devices) sense the motion of the worker 400, the sensing system operates the movement mechanism and the drive mechanism of each humanoid robot 20a, 20b so that the second sensor 23b (second imaging device 24b) senses a predetermined motion of the worker 400 from a position different from that of the first sensor 23a (first imaging device 24a) and so that the sensing areas 230a, 230b (imaging areas 240a, 240b) of each sensor sense different predetermined parts of the worker 400. However, the sensing areas 230a, 230b (imaging areas 240a, 240b) of each sensor do not need to be entirely different, and may be set so that the predetermined parts are different even in part. Examples of the predetermined part include the worker's neck, arm, wrist, etc. Furthermore, the predetermined part may be recognized by each sensor using a known image recognition technique, or the predetermined part may be recognized by learning using the learning unit 663 (see FIG. 4).

[0028] In this embodiment, the movement mechanism 22 and drive mechanism of each humanoid robot are operated so that the sensing area 230a (imaging area 240a) of the first sensor 23a (first imaging device 24a) provided on the first humanoid robot 20a senses the left arm of the worker 400, and the sensing area 230b (imaging area 240b) of the second sensor 23b (second imaging device 24b) provided on the second humanoid robot 20b senses the right arm of the worker 400.

[0029] While sensing is being performed by each sensor, the sensing system determines whether or not a predetermined part that moves when the worker 400 performs a predetermined motion is sensed, based on the first information acquired by the first sensor 23a (first imaging device 24a) and the second information acquired by the second sensor 23b (second imaging device 24b). The predetermined motions are diverse, and include, for example, assembling or moving parts, painting a product, and the movement of the worker himself.

[0030] If it is determined that a specific part of the worker 400 that moves during a specific movement is not being sensed, the sensing system activates the first movement mechanism 22a (see Figure 3) of the first humanoid robot 20a and / or the second movement mechanism 22b (see Figure 3) of the second humanoid robot 20b so that the specific part is sensed.

[0031] FIG. 2 is a diagram showing an example of the sensing system of this embodiment when a mobile robot moves.

[0032] As shown in FIG. 2, when the worker 400 performs a predetermined motion, for example, when viewed from behind the worker 400, the left arm of the worker 400 may be hidden by the back. Therefore, it is determined whether or not a predetermined part that moves when the worker 400 performs a predetermined motion is sensed, and if it is determined that the predetermined part is not sensed, the sensing system operates the first moving mechanism 22a of the first humanoid robot 20a so that the predetermined part is sensed. This allows the first humanoid robot 20a to move to a position where the first sensor 23a (first imaging device 24a) can easily sense, and the first sensor 23a (first imaging device 24a) can adequately sense the predetermined part (the left arm in this embodiment). Note that in FIG. 2, since the position of the right arm also changes with the predetermined motion of the worker 400, the second moving mechanism 22b of the second humanoid robot 20b (second sensor 23b (second imaging device 24b)) that sensed the right arm is also operated.

[0033] The first information and the second information are stored, and the sensing system learns the predetermined motion of the worker based on the stored first information and second information. This learning is performed by automatic learning, which is learning that automatically creates a learned model and automatically performs judgment / analysis using the learned model, for example.

[0034] The sensing system refers to the learning results, as well as the work manual information and / or the work schedule information for the worker 400, and generates operation information for giving operation instructions to a work robot performing the work of the worker 400. This makes it possible to have a work robot perform work that would be performed by a human. Note that the work manual information includes, for example, the name and content of each work item, the order of the work items, and information on the standard work time required for each work item. Furthermore, the work schedule information includes, for example, information indicating the work time and start time / end time of the entire work, information indicating the work time and start time / end time of each work item, information indicating the worker for each work item, and the like.

[0035] FIG. 3 is a block diagram showing an example of the configuration and functions of the sensing system 100 of this embodiment.

[0036] The sensing system 100 includes a first humanoid robot 20a, a second humanoid robot 20b, and a management control device 60. The first humanoid robot 20a and the second humanoid robot 20b are each connected to a communication unit 64 of the management control device 60 via wireless or wired communication, and receive instructions from the management control device 60 and transmit information acquired by each sensor. The first humanoid robot 20a and the second humanoid robot 20b may also be connected to each other via wireless or wired communication, and transmit and receive information acquired by each sensor and instructions.

[0037] The first humanoid robot 20a includes a first moving mechanism 22a, a first sensor 23a which is a sensor for the robot, a first imaging device 24a which is an imaging device for the robot included in the first sensor 23a, a first information processing device 25a, a first driving mechanism, and a second driving mechanism. The second humanoid robot 20b also includes a second moving mechanism 22b, a second sensor 23b which is a sensor for the robot, a second imaging device 24b which is an imaging device for the robot included in the second sensor 23, a second information processing device 25b, and two driving mechanisms. In this embodiment, the first humanoid robot 20a and the second humanoid robot 20b have the same configuration.

[0038] The first information processing device 25a according to this embodiment includes a central processing unit (CPU) 1212, a random access memory (RAM) 1214, and a graphic controller 1216, which are connected to each other by a host controller 1210. The first information processing device 25a also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid state drive, or the like. The first information processing device 25a also includes input / output units such as a read only memory (ROM) 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0039] The CPU 1212 operates according to a program stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into itself, and causes the image data to be displayed on the display device 1218.

[0040] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the first information processing device 25a. The storage device 1224 may also store first information and second information. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0041] The ROM 1230 stores therein a boot program, etc., executed by the first information processing device 25a upon activation, and / or a program that depends on the hardware of the first information processing device 25a. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0042] The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the first information processing device 25a, and brings about cooperation between the programs and the above-mentioned various types of hardware resources. An apparatus or method may be configured by realizing an operation or processing of information according to the use of the first information processing device 25a.

[0043] For example, when communication is performed between the first information processing device 25a and an external device, the CPU 1212 may execute a communication program loaded in the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0044] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0045] Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and undergo information processing. The CPU 1212 may execute various types of processing on data read from the RAM 1214, including various types of operations, information processing, conditional decisions, conditional branches, unconditional branches, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write back the results to the RAM 1214. The CPU 1212 may also search for information in files, databases, etc., in the recording medium.

[0046] The above-described program or software module may be stored in a computer-readable storage medium on the first information processing device 25a or in the vicinity of the first information processing device 25a. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable storage medium, thereby providing the program to the first information processing device 25a via the network.

[0047] The blocks in the flowcharts and diagrams in this embodiment may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer readable instructions stored on a computer readable storage medium, and / or a processor provided with computer readable instructions stored on a computer readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as, for example, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like, including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0048] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray disks, memory sticks, integrated circuit cards, and the like.

[0049] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0050] Computer readable instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, either locally or over a local area network (LAN), a wide area network (WAN), such as the Internet, etc., to cause the processor of the general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, to execute the computer readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0051] The contents described so far also apply to the second information processing device 25b.

[0052] The management control device 60 is a control device that gives instructions to the humanoid robots 20a and 20b in order to realize the sensing system 100. In addition, the management control device 60 acquires the sensor information (first information and second information) accumulated in the storage device 1224.

[0053] The management control device 60 is composed of a CPU 60A, a RAM 60B, a ROM 60C, an input / output unit (I / O) 60D, a bus 60E such as a data bus or a control bus connecting these, and a communication unit 68. A storage medium 62 is connected to the I / O 60D.

[0054] In addition, a communication unit 64 that transmits and receives sensor information, work manual information, process chart information, etc. to and from the control system of the humanoid robot 20 is connected to the I / O 60D.

[0055] FIG. 4 is a block diagram showing an example of the functions of the management control device 60 in the sensing system of this embodiment.

[0056] The management control device 60 includes a storage medium 62 , a communication unit 64 , and a processing unit 66 .

[0057] The storage medium 62 includes, for example, at least one of a semiconductor storage device, a magnetic tape device, a magnetic disk device, and an optical disk device. The storage medium 62 stores a driver program, an operating system program, an application program, data, and the like used for processing in the processing unit 66. For example, the storage medium 62 stores first information and second information. The storage medium 62 also stores work manual information and / or process chart information for the worker 400.

[0058] The communication unit 64 has a wireless communication interface circuit such as Wi-Fi (registered trademark) and / or a wired communication interface circuit such as Ethernet (registered trademark). The communication unit 64 transmits and receives various information to and from the humanoid robots 20a and 20b through the interface circuits.

[0059] The processing unit 66 has one or more processors and their peripheral circuits. The processing unit 66 generally controls the overall operation of the sensing system 100, and is, for example, a CPU. The processing unit 66 executes processing by referring to programs (driver programs, operating system programs, application programs, etc.) stored in the storage medium 62. The processing unit 66 can also execute multiple programs (application programs, etc.) in parallel.

[0060] The processing unit 66 includes a determination unit 661, a control unit 662, a learning unit 663, and a motion information generation unit 664. Each of these units is a functional module realized by a program executed by a processor included in the processing unit 60. Alternatively, each of these units may be implemented in the processing unit 66 as firmware.

[0061] The determination unit 661 determines, from the first information and the second information, whether or not a predetermined part that moves when the worker 400 performs a predetermined motion is sensed. The determination unit 661 also determines whether or not the predetermined part sensed by the first sensor 23a (first imaging device 24a) is the same as the predetermined part sensed by the second sensor 23b (second imaging device 24b).

[0062] When the determining unit 661 determines that the predetermined part of the worker 400 has not been sensed, the control unit 662 operates the first moving mechanism 22a of the humanoid robot 20a and / or the second moving mechanism 22b of the humanoid robot 20b so that the predetermined part is sensed. When the control unit 662 determines that the predetermined part sensed by the first sensor 23a (first imaging device 24a) and the predetermined part sensed by the second sensor 23b (second imaging device 24b) are the same, the control unit 662 operates the first moving mechanism 22a of the humanoid robot 20a and / or the second moving mechanism 22b of the humanoid robot 20b so that the predetermined part sensed by the first sensor 23a (first imaging device 24a) and the predetermined part sensed by the second sensor 23b (second imaging device 24b) are different.

[0063] The learning unit 663 learns the predetermined motion of the worker 400 by referring to the first information and the second information stored in the storage medium 62 and / or the storage device 1224.

[0064] The motion information generating unit 664 generates motion information for giving motion instructions to the humanoid robot 20 functioning as a working robot, with reference to the learning result by the learning unit 663. Note that the motion information generating unit 664 may refer to work manual information and / or work schedule information when generating the motion information.

[0065] (Processing of the sensing system according to the first embodiment of the present invention) FIG. 5 is an example of a flowchart showing the processing of the sensing system of this embodiment.

[0066] First, in response to an instruction from the management control device 60 or an instruction to read out a program stored in the storage medium 62 or the storage device 1224, the information processing device 25 instructs the multiple humanoid robots 20 (two in this embodiment) functioning as mobile robots to move to the workshop 200 (step S101). The movement is caused by the operation of the robot movement mechanism 22 of each humanoid robot 20.

[0067] When moving, an instruction is given so that each sensing area 230 (imaging area 240) of the multiple robot sensors 23 (robot imaging devices 24) senses the worker 400 from a different direction. The placement of such multiple humanoid robots 20 is performed, for example, by storing a floor plan of the workplace 200 in advance in the storage device 1224 and / or the storage medium 62, and associating the position of each humanoid robot 20 with the stored floor plan. Alternatively, the placement of the humanoid robot 20 may be based on a position optimized through machine learning.

[0068] Next, a predetermined movement of the worker 400 on the work line 201 is sensed by the multiple sensors 23a, 23b (multiple imaging devices 24a, 24b) (step S102). In this embodiment, the control unit 662 issues an instruction to cause the sensing area 230a (imaging area 240a) of the first sensor 23a (first imaging device 24a) to sense the left arm of the worker 400 and the sensing area 230b (imaging area 240b) of the second sensor 23b (second imaging device 24b) to sense the right arm of the worker 400, and the movement mechanism 22 and the drive mechanism of each humanoid robot are operated.

[0069] The first information acquired by the first sensor 23a (first imaging device 24a) and the second information acquired by the second sensor 23b (second imaging device 24b) are stored in the storage medium 62 via the storage device 1224 and / or the communication unit 64. The storage device 1224 and the storage medium 62 function as a storage unit.

[0070] When the first information and the second information are acquired, it is determined whether or not a predetermined part of the worker 400 is sensed (step S103). If the determining unit 661 determines that the predetermined part is not sensed, the control unit 662 operates the first moving mechanism 22a and / or the second moving mechanism 22b so that the predetermined part is sensed.

[0071] The management control device 60 learns a specified movement by referring to the first information and second information accumulated, in other words, stored, in the memory device 1224 and / or the storage medium 62, and generates movement information that gives movement instructions to the humanoid robot 20 functioning as a work robot by referring to the learning results (step S104).

[0072] FIG. 6 is an example of a flowchart showing more detailed processing of the predetermined part sensing determination processing shown in step S103 of FIG.

[0073] When the first information and the second information are acquired (step S201), the determination unit 661 determines whether or not a predetermined part that moves when the worker 400 performs a predetermined motion is sensed from the first information and the second information (steps S202 and S203). As an example, the first sensor 23a (first imaging device 24a) is instructed to sense the left arm of the worker 400, and the second sensor 23b (second imaging device 24b) is instructed to sense the right arm of the worker 400, but there may be a case where the left arm or the right arm is hidden behind the back when the worker 400 performs a predetermined motion. Therefore, the determination unit 661 determines whether or not the first sensor 23a (first imaging device 24a) is sensing the left arm as the predetermined part, by referring to the first information and the second information.

[0074] If it is determined that the predetermined part has not been sensed (S203-NO), the control unit 662 operates the first moving mechanism 22a and / or the second moving mechanism 22b so that the predetermined part is sensed (step S206). As an example, the control unit 662 operates the first moving mechanism 22a and moves the first humanoid robot 20a so that the first sensor 23a (first image capture device 24a) can sense the left arm, which is a part of the predetermined part of the worker 400. In addition, the control unit 662 operates the second moving mechanism 22b and moves the second humanoid robot 20b so that the second sensor 23b (second image capture device 24b) can sense the right arm, which is the other part of the predetermined part of the worker 400. This makes it possible to avoid a situation in which the predetermined part is not sensed due to the predetermined action or movement of the worker 400, and to perform sufficient sensing. In addition, since each sensor senses a part of the predetermined area and another part, data (information) required for the worker 400 to learn the work can be efficiently acquired.

[0075] On the other hand, if it is determined that sensing is being performed (S203-YES), the determination unit 661 determines whether or not the predetermined part sensed by the first sensor 23a (first image capture device 24a) and the predetermined part sensed by the second sensor 23b (second image capture device 24b) are the same (steps S204 and S205). As an example, when sensing is started, the first sensor 23a (first image capture device 24a) senses the left arm of the worker 400, and the second sensor 23b (second image capture device 24b) senses the right arm of the worker 400, respectively, but a situation may occur in which the same predetermined part (for example, the back) is sensed due to a predetermined movement of the worker 400. Therefore, the determination unit 661 determines whether or not the predetermined parts sensed by each sensor are the same.

[0076] When it is determined that the predetermined parts sensed by the sensors are the same (S205-YES), the control unit 662 operates the first moving mechanism 22a and / or the second moving mechanism 22b so that the predetermined part sensed by the first sensor 23a (first imaging device 24a) is different from the predetermined part sensed by the second sensor 23b (second imaging device 24b) (S206). As an example, when the first sensor 23a (first imaging device 24a) and the second sensor 23b (second imaging device 24b) are both sensing the back of the worker 400, the control unit 662 operates the first moving mechanism 22a and the second moving mechanism 22b so that the first sensor 23a (first imaging device 24a) senses the left arm of the worker, and the second sensor 23b (second imaging device 24b) senses the right arm of the worker. This makes it possible to prevent a situation in which multiple sensors sense the same predetermined area, and makes it possible to efficiently acquire the data necessary for learning the task.

[0077] FIG. 7 is an example of a flowchart showing more detailed processing of the motion information generating processing shown in step S104 of FIG.

[0078] The first information and the second information are stored in the storage device 1224 and / or the storage medium 62 (step S301), and the learning unit 663 learns the movements of the worker 400 by referring to the first information and the second information stored in the storage device 1224 and / or the storage medium 62 (step S302). In the learning, motion capture of the movements of the worker 400, a 3D map of the workplace 200, navigation of the movement and movements of the worker 400 in the workplace 200, cornering, speed, and the like are analyzed, and the optimal movements of the humanoid robot 20, which can also function as a working robot, are learned by automatic learning. This makes it possible to analyze a given movement of the worker 400 from multiple aspects at once, and reduces the time and cost required for analyzing and programming the movements of the worker 400.

[0079] Thereafter, the motion information generating unit 664 refers to the learning result by the learning unit 663 (step S303) and generates motion information for giving motion instructions to the humanoid robot 20 functioning as a working robot (step S304). When generating the motion information, the motion information generating unit 664 may refer to the work manual information and / or the schedule information of the sensing target (S303). By referring to the generated motion information, the humanoid robot 20 functioning as a working robot is able to perform the work (predetermined motion) of the worker 400.

[0080] (Operation and effect of the sensing system according to the first embodiment) According to the sensing system 100 of this embodiment, when a specific part of the worker 400 to be sensed that moves during a specific movement is not sensed, the first moving mechanism 22a is activated so that the specific part is sensed. This makes it possible to prevent a situation in which the specific part is not sensed due to the specific movement of the worker 400, and enables sufficient sensing to be performed.

[0081] Furthermore, according to the sensing system 100 of this embodiment, the humanoid robot 20, which is a mobile robot, is equipped with a sensor 23 (imaging device 24), while the humanoid robot 20 moves to an appropriate position according to the sensing results as well as pre-stored programs and machine learning results. Therefore, compared to the case where fixed sensors are arranged in the workshop 200, a sufficient sensing environment can be prepared without causing a situation such as rearranging the sensors or increasing the number of sensors.

[0082] Furthermore, according to the sensing system 100 of this embodiment, each of the multiple (two in this embodiment) mobile robots is provided with a movement mechanism, and when a specific part of the worker 400 to be sensed that moves during a specific movement is not sensed, each movement mechanism operates to sense the specific part, making it possible to sense the specific movement of the worker 400 from multiple different angles.

[0083] Furthermore, according to the sensing system 100 of this embodiment, the movement mechanism is operated so that one of the multiple (two in this embodiment) sensors 23 (imaging devices 24) senses a part of a predetermined part of the worker 400, and the movement mechanism is operated so that the other sensor (imaging device) senses the other part of the predetermined part, so that the predetermined movement of the worker 400 can be sensed from multiple different angles and multiple predetermined parts. This makes it possible for multiple mobile robots to work together to perform sensing while moving, and sufficient sensing can be performed.

[0084] Furthermore, according to the sensing system 100 of this embodiment, the sensor information (first information and second information in this embodiment) acquired by each sensor is stored, the predetermined motion of the worker 400 is learned with reference to the stored sensor information, and the learning result is referred to to generate motion information for giving motion instructions to the humanoid robot functioning as a working robot, so that motion information that sufficiently reflects the sensing result is generated. This makes it possible to efficiently manufacture a working robot (humanoid robot 20) that can perform the work of the worker 400.

[0085] Moreover, according to the sensing system 100 of this embodiment, when generating the operation information, the operation information is generated after referring to the work manual information and / or the schedule information. The worker 400 does not always perform the operation faithfully, and in some cases, he may perform an unnecessary operation or omit a necessary operation. Therefore, by referring to the work manual information and the schedule information, it is possible to prevent unnecessary or inappropriate predetermined operations by the worker 400 from being reflected in the operation information.

[0086] (Modification 1 of the first embodiment) FIG. 8 is a diagram illustrating an example of a sensing system according to the first modification of the present embodiment.

[0087] Fig. 8(a) is a diagram showing an example of a system configuration of a sensing system according to a first modified example of the first embodiment of the present invention. This sensing system is characterized in that a humanoid robot 20c functioning as a mobile robot is provided with a torso sensor 23d (torso image capture device 24d) corresponding to a second sensor. In addition, this sensing system does not necessarily require a management control device 60, and the sensing system can be configured by the humanoid robot 20c alone.

[0088] Fig. 8(b) is a diagram showing an example of the mobile robot shown in Fig. 8(a). A humanoid robot 20c functioning as a mobile robot includes a robot main body 21c, a robot moving mechanism 22c, a head sensor 23c, a head imaging device 24c included in the head sensor 23c, a body sensor 23d, a body imaging device 24d included in the body sensor 23d, an information processing device 25c, and a robot arm.

[0089] The robot main body 21c includes a robot trunk 211 and a robot head 212. The robot trunk 211 and the robot head 212 configure a first driving mechanism 21c (see FIG. 9), and are capable of changing a sensing area 230c (imaging area 240c) of the head sensor 23c (head imaging device 24c) and a sensing area 230d (imaging area 240d) of the trunk sensor 23d (trunk imaging device 24d). The robot moving mechanism 22c functions as a first moving mechanism.

[0090] Head sensor 23c (head imaging device 24c) functions as a first sensor, and torso sensor 23d (torso imaging device 24d) functions as a second sensor. Head sensor 23c (head imaging device 24c) and torso sensor 23d (torso imaging device 24d) are disposed at different height positions, so that torso sensor 23d (torso imaging device 24d) functioning as the second sensor senses a predetermined movement of the sensing target from a position different from that of head sensor 23c (head imaging device 24c).

[0091] The configuration of the information processing device 25c is similar to that of the first information processing device 25a of the first humanoid robot 20a. The robot arm is also similar to that of the first humanoid robot 20a.

[0092] 9 is a block diagram showing an example of the functions of a mobile robot in this sensing system. In the sensing system 100', the information processing device 25c includes an information processing unit 66c, a communication interface 1222c, and a storage device 1224c, and the information processing unit 66c includes a determination unit 661c, a control unit 662c, a learning unit 663c, and an operation information generating unit 664c. That is, in the sensing system 100', the information processing unit 66c performs the same processing as the processing unit 66 of the management control device 60. The information processing device 25c is configured to be able to communicate with the head sensor 23c (head imaging device 24c), the body sensor 23d (head imaging device 24d), the first moving mechanism 22c, and the first driving mechanism 21c.

[0093] In the humanoid robot 20c of the sensing system 100', an information processing device 25c is provided with an information processing section 66c, and therefore the humanoid robot 20c alone constitutes a sensing system.

[0094] 8, for example, the control unit 662c of the humanoid robot 20c instructs the head sensor 23c (head image capture device 24c) functioning as a first sensor to sense the left arm of the worker 400, and the torso sensor 23d (torso image capture device 24d) functioning as a second sensor to sense the right arm of the worker 400. Then, the determination unit 661c determines whether or not a predetermined part that moves when the worker 400 performs a predetermined motion is sensed from the sensor information (first information and second information) acquired by each sensor, and if it is determined that the predetermined part is not sensed, the control unit 662c operates the first moving mechanism 22c and / or the first driving mechanism 21c so that the predetermined part is sensed.

[0095] In addition, the determination unit 661c determines whether the specified part sensed by the head sensor 23c (head imaging device 24c) is the same as the specified part sensed by the torso sensor 23d (torso imaging device 24d), and when it is determined that the specified parts sensed by each sensor (imaging device) are the same, the control unit 662c operates the first moving mechanism 22c and / or the first driving mechanism 21c so that the specified parts sensed by each sensor (imaging device) are different.

[0096] (Effects of Modification 1) According to this sensing system, the humanoid robot 20c can constitute a sensing system by itself, so that it is possible to perform sufficient sensing even in places where communication with the management control device 60 is not possible, for example.

[0097] In addition, since the self-type robot 20c is equipped with multiple (two in this modified example) sensors (imaging devices), it is possible to perform sufficient sensing even in a narrow space for sensing the worker 400, for example.

[0098] In this sensing system, the humanoid robot functioning as the mobile robot does not necessarily have to be one, but may be multiple. In this case, as the number of humanoid robots increases, the number of sensors increases by a multiple of the number of humanoid robots, making it possible to obtain a large amount of sensor information at one time.

[0099] (Modification 2 of the first embodiment) FIG. 10 is a diagram illustrating an example of a sensing system according to a second modification of the present embodiment.

[0100] 10(a) is a diagram showing an example of a system configuration of a sensing system according to Modification 2 of Embodiment 1. This sensing system is characterized in that it includes a sensor mounting member 30 in addition to a humanoid robot 20 that functions as a mobile robot.

[0101] 10(b) is a diagram showing an example of the sensor mounting member shown in (a). The sensor mounting member 30 includes a mounting member main body 31, a mounting member moving mechanism 32, a mounting member sensor 33, and a mounting member imaging device 34. The sensor mounting member 30 can be moved by the mounting member moving mechanism 32 provided below the mounting member main body 31. However, the mounting member moving mechanism 32 does not necessarily have to be provided.

[0102] The mounting member body 31 is, for example, a rod-like or cane-like member, and its material is not particularly limited. The length of the mounting member body 31 is longer than the height (back height) of the humanoid robot 20, for example, 2.1 meters. The mounting member body 31 is provided with a mounting member moving mechanism 32 below, preferably at the lower end, and a mounting member sensor 33 above, preferably at the upper end, of the mounting member body 31.

[0103] The mounting member moving mechanism 32 is configured with rotating bodies such as casters, and assists the movement of the sensor mounting member 30 in accordance with the movement of the humanoid robot 20. Note that in this embodiment, it is not assumed that the sensor mounting member 30 moves autonomously, but a mounting member control unit (not shown) that issues instructions to the mounting member moving mechanism 32 may be provided, and the mounting member moving mechanism 32 may be moved based on a signal from the mounting member control unit.

[0104] 10(a), the mounting member sensor 33 (mounting member imaging device 34) functioning as a second sensor is provided above the mounting member main body 31 and senses the worker 400. An example of the mounting member sensor 33 is similar to the robot sensor 23, and an example of the mounting member imaging device 34 is similar to the robot imaging device 24. In addition, an example of acquired sensor information is also similar to the robot sensor 23, and an example of the sensing timing of the sensor information is also similar to the robot sensor 23.

[0105] The mounting member imaging device 34 is included in the mounting member sensor 33. The mounting member sensor 33 including the mounting member imaging device 34 is disposed at a position higher than the height (back height) of the humanoid robot 20. This allows the mounting member sensor 33 to sense the movement of the worker 400 from a position higher than the robot sensor 23.

[0106] FIG. 11 is a diagram showing an example of the configuration and functions of a sensing system according to Variation 2 of Embodiment 1. In the sensing system 100'', the sensor mounting member 30 is configured to be able to communicate with the information processing device 25 of the humanoid robot 20 via wireless or wired communication. However, the sensor mounting member 30 may be configured to be able to communicate with the communication unit 64 of the management control device 60 instead of or in addition to the information processing device 25. The configurations of the humanoid robot 20 and the management control device 60 of the sensing system 100'' are similar to those of the sensing system 100.

[0107] 10, the humanoid robot 20 grips the sensor mounting member 30 with a right gripping part 265 (or a left gripping part 266) which is a part of the robot arm 26 constituting the second driving mechanism. The mounting member sensor 33 (mounting member imaging device 34) of the sensor mounting member 30 can change its sensing area 330 (imaging area 340) by the second driving mechanism.

[0108] In the sensing system 100'', for example, the control unit 662 instructs the robot sensor 23 (robot imaging device 24) functioning as the first sensor to sense the left arm of the worker 400, and the attachment member sensor 33 (attachment member imaging device 34) functioning as the second sensor to sense the right arm of the worker 400. Then, the determination unit 661 determines whether or not a predetermined part that moves when the worker 400 performs a predetermined motion is sensed from the sensor information (first information and second information) acquired by each sensor, and if it is determined that the predetermined part is not sensed, the control unit 662 operates the first moving mechanism 22 and / or the second driving mechanism 26 so that the predetermined part is sensed. Also, if the determination unit 661 determines that the predetermined parts sensed by each sensor (imaging device) are the same, the control unit 662 operates the first moving mechanism 22 and / or the second driving mechanism 26 so that the predetermined parts sensed by each sensor (imaging device) are different.

[0109] (Effects of Modification 2) According to this sensing system, since the mounting member sensor 33 (mounting member imaging device 34) is configured as the second sensor, it is possible to perform sufficient sensing even in a narrow space for sensing the worker 400, for example.

[0110] Furthermore, according to this sensing system, the sensor mounting member 30 has the mounting member sensor 30 (mounting member image capture device 40) disposed at a position higher than the height (back height) of the humanoid robot 20. This allows the movement of the worker 400 to be sensed from a more bird's-eye view position, making it easier to avoid situations where sensing is difficult due to the back of the worker 400, for example, and allows efficient acquisition of data required for learning the work of the worker 400.

[0111] In addition, in this sensing system, the number of humanoid robots functioning as a mobile robot does not need to be one, and the number of sensor mounting members does not need to be one. For example, a plurality of humanoid robots 20 each having two sensor mounting members held by both holding parts 265, 266 may be provided. In this case, too, the number of sensors can be increased, making it possible to obtain a large amount of sensor information at one time.

[0112] Although the embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment of the present invention, and various modifications and applications are possible without departing from the gist of the present invention.

[0113] In the sensing system 100 of this embodiment, two mobile robots (humanoid robots) equipped with sensors and moving mechanisms are used. However, the number of mobile robots may be more than this. For example, if there are a large number of mobile robots equipped with sensors and moving mechanisms, it is possible to arrange multiple sensors so as to sense the predetermined motion of the worker 400 from different positions, heights, and / or directions. This makes it easier to obtain various data required for learning the predetermined motion of the worker 400, and enables sensing to cover each predetermined part of the worker 400 in its entirety.

[0114] In the present embodiment, the first sensor (first imaging device) senses the left arm of the worker, and the second sensor (second imaging device) senses the left arm of the worker. However, the predetermined part to be sensed is not limited to this, and the predetermined part sensed by each sensor is not limited to this. For example, the first sensor (first imaging device) may sense the fingertips of the worker's right hand, and the second sensor (second imaging device) may sense the movement of the worker's neck.

[0115] In the present embodiment, the learning of the predetermined motion of the worker is described as being performed by automatic learning. However, the learning does not necessarily have to be automatic learning, and may be other known machine learning such as deep learning, unsupervised / supervised learning, reinforcement learning, etc.

[0116] In the present embodiment, the mobile robot and the working robot are described as being the same humanoid robot. In this case, it is possible to use the mobile robot as a working robot, and it is possible to save on the costs involved in manufacturing the robot. However, the mobile robot and the working robot may be different robots.

[0117] In addition, in the present embodiment, a worker (person) is used as the sensing target, but the present invention is not limited to this, and the sensing target may be, for example, a robot that can imitate a predetermined operation of a worker. [Explanation of symbols]

[0118] 100, 100', 100'' Sensing System 20, 20a, 20b, 20c Humanoid robots (mobile robots, working robots) 22, 22a, 22b, 22c movement mechanism 23, 23a, 23b, 23c, 23d Robot sensors 230, 230a, 230b, 230c, 230d sensing area 24, 24a, 24b, 24c, 24d Imaging device for robot 240, 240a, 240b, 240c, 240d Imaging area 25, 25a, 25b, 25c Information processing device 60 Management control device 62 Storage medium (storage unit) 1224 Storage device (storage unit)

Claims

1. A first sensor for sensing a predetermined motion of a sensing target; a second sensor for sensing a predetermined motion of the sensing target from a position different from that of the first sensor; a first mobile robot including the first sensor and a first moving mechanism; a management control device capable of communicating with the first sensor, the second sensor, and the first movement mechanism; The management control device includes: a determination unit that determines whether or not a predetermined part that moves during a predetermined operation of the sensing target is sensed based on first information acquired by the first sensor and second information acquired by the second sensor; and a control unit that operates the first moving mechanism so that the predetermined part is sensed when the determination unit determines that the predetermined part is not sensed. A sensing system comprising:

2. the second sensor is disposed on a second mobile robot having a second movement mechanism; When the determination unit determines that the predetermined portion has not been sensed, the control unit operates the first moving mechanism and the second moving mechanism so that the predetermined portion is sensed. The sensing system of claim 1 .

3. The sensing system of claim 2 , wherein the control unit operates the first moving mechanism so that the first sensor senses a portion of the specified area, and operates the second moving mechanism so that the second sensor senses another portion of the specified area.

4. Further comprising a work robot; The management control device includes: a storage unit that stores the first information and the second information; a learning unit that learns the predetermined motion by referring to the first information and the second information stored in the storage unit; and a motion information generating unit that generates motion information for providing motion instructions to the working robot by referring to a learning result by the learning unit. The sensing system according to any one of claims 1 to 3.

5. The storage unit stores in advance work manual information or process chart information of the sensing target, 5. The sensing system according to claim 4, wherein the operation information generation unit generates operation information for providing operation instructions to the work robot by referring to the learning results by the learning unit and work manual information or process chart information of the sensing target.

6. A first sensor for sensing a predetermined motion of a sensing target; a second sensor for sensing a predetermined motion of the sensing target from a position different from that of the first sensor; a first mobile robot including the first sensor and a first moving mechanism; a management control device capable of communicating with the first sensor, the second sensor, and the first movement mechanism; The management control device includes: determining whether or not a predetermined part of the sensing target that moves during a predetermined operation is sensed based on first information acquired by the first sensor and second information acquired by the second sensor; when it is determined that the predetermined portion has not been sensed, actuating the first moving mechanism so that the predetermined portion is sensed; A sensing method comprising:

7. A first sensor for sensing a predetermined motion of a sensing target; a second sensor for sensing a predetermined motion of the sensing target from a position different from that of the first sensor; a first mobile robot including the first sensor and a first moving mechanism; a management control device capable of communicating with the first sensor, the second sensor, and the first movement mechanism; The management control device includes: a determination unit that determines whether or not a predetermined part that moves when the sensing target performs a predetermined operation is sensed based on first information acquired by the first sensor and second information acquired by the second sensor, and determines whether or not the predetermined part sensed by the first sensor is the same as the predetermined part sensed by the second sensor; and a control unit that, when it is determined that the predetermined portion sensed by the first sensor and the predetermined portion sensed by the second sensor are the same, operates the first moving mechanism so that the predetermined portion sensed by the first sensor and the predetermined portion sensed by the second sensor are different. A sensing system comprising:

8. A mobile robot, A moving mechanism; a first sensor for sensing a sensing object; a second sensor that senses the sensing object from a position different from that of the first sensor; a drive mechanism capable of moving a position of the second sensor; an information processing unit that controls the first sensor, the second sensor, the moving mechanism, and the driving mechanism, The information processing unit includes: a determination unit that determines whether or not a predetermined part that moves during a predetermined operation of the sensing target is sensed based on first information acquired by the first sensor and second information acquired by the second sensor; and a control unit that operates the moving mechanism or the driving mechanism so that the predetermined part is sensed when the determination unit determines that the predetermined part is not sensed. A mobile robot characterized by

9. A mobile robot, A moving mechanism; a first sensor for sensing a sensing object; a second sensor that senses the sensing object from a position different from that of the first sensor; a drive mechanism capable of moving a position of the second sensor; an information processing unit that controls the first sensor, the second sensor, the moving mechanism, and the driving mechanism, The information processing unit includes: a determination unit that determines whether or not a predetermined part that moves when the sensing target performs a predetermined operation is sensed based on first information acquired by the first sensor and second information acquired by the second sensor, and determines whether or not the predetermined part sensed by the first sensor is the same as the predetermined part sensed by the second sensor; and a control unit that, when it is determined that the predetermined portion sensed by the first sensor and the predetermined portion sensed by the second sensor are the same, operates the moving mechanism or the driving mechanism so that the predetermined portion sensed by the first sensor and the predetermined portion sensed by the second sensor are different. A mobile robot characterized by

Images