Control system for robot and control program for robot

Abstract

To enable installing, at appropriate positions, respective sensors for acquiring various kinds of information necessary for controlling a robot in a multimodal manner; and to acquire information from the respective sensors in an appropriate state and at appropriate timing.SOLUTION: In a hand tool 50, sensors having respective different information acquisition functions are attached to respective finger parts 22A, 22B, 22C, 22D, and 22E. In other words, sensors for acquiring various kinds of information necessary for controlling a robot are attached. A visual sensor 51A for detecting visual information is attached to a thumb, an auditory sensor 51B for detecting auditory information is attached to an index finger, an olfactory sensor 51C is attached to a middle finger, a tactile sensor 51D for detecting tactile information is attached to a ring finger, and a taste sensor 51E for detecting taste information is attached to a little finger.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a control system for a robot and a control program for a robot.

Background Art

[0002] It has been proposed to move a robot multimodally using generative AI (see Patent Document 1 and Patent Document 2).

[0003] Multimodal means using not only single information such as only image recognition or only language processing, but also multiple pieces of information.

[0004] In particular, regarding the steps of execution, feedback, and adaptation of the robot, a plurality of sensor groups for acquiring various types of information are essential.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, each sensor module of the sensor group has not reached a configuration for acquiring necessary information in which it is installed at an appropriate position and acquires information in an appropriate state and at an appropriate time, nor a specification capable of acquiring the necessary information to be acquired.

[0007] In view of the above facts, an object of the present invention is to provide a robot control system capable of installing each sensor module of a sensor group for acquiring various information necessary for controlling a robot multimodally at an appropriate position and acquiring information from each sensor module in an appropriate state and at an appropriate time, and to obtain a robot control program.

Means for Solving the Problems

[0008] A robot control system according to the present invention is a robot control system including a work actuator attached to a robot for the robot to execute work, and a sensor unit for acquiring a plurality of types of information necessary for the execution of the work of the work actuator, wherein a plurality of types of information are collated from each of the sensor units with attribute information corresponding to the work to create operation control information necessary for operation control and feedback control of the robot, and a control unit for controlling the work of the robot based on the operation control information.

[0009] In the present invention, it further includes a camera for photographing an image of an object of the work to identify the type of the object, and a motion processing unit for specifying the position of the object.

[0010] The camera identifies the photographed object (hereinafter sometimes referred to as a package) based on the photographed image information. That is, it has a role of acquiring information for specifying the type (shape, size, hardness, etc.) of the object.

[0011] The motion processing unit (MoPU) outputs, as position information together with motion information, vector information of the motion of a point indicating the existence position of the object along a predetermined coordinate axis. That is, the motion information output from the MoPU includes only information indicating the motion (moving direction and moving speed) of the center point (or centroid point) of the object on the coordinate axes (x-axis, y-axis, z-axis). That is, it can accurately guide the trajectory when the gripping part approaches the object.

[0012] In the present invention, the acquisition information acquired by the sensor unit is information related to vision, hearing, smell, touch, and taste, and operation control information is obtained by combining a plurality of pieces of acquisition information.

[0013] The control system of the robot according to the present invention is characterized in that a computer is operated as the operation information creation unit and the control unit of the control system of the above-described robot.

[0014] Note that the above summary of the invention does not list all of the necessary features of the present invention. Also, sub-combinations of these feature groups can also be inventions.

Effect of the Invention

[0015] As described above, according to the present invention, each sensor module of the sensor group for acquiring various information necessary for controlling the robot multimodally can be installed at an appropriate position, and information can be acquired from each sensor module in an appropriate state and at an appropriate time.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Best Mode for Carrying Out the Invention

[0017] Hereinafter, the present invention will be described through embodiments of the invention. However, the following embodiments do not limit the invention according to the claims. Also, not all combinations of features described in the embodiments are essential for the solution means of the invention.

[0018] FIG. 1 is a front view of the humanoid robot 1 according to the present embodiment. As shown in FIG. 1, the humanoid robot 1 according to the present embodiment includes an upper body portion 2, a leg portion 3, and a connecting portion 4 that rotatably connects the upper body portion 2 to the leg portion 3.

[0019] The upper body portion 2 has two arm portions 5 and 6. The arm portions 5 and 6 are rotatably attached to the left and right sides of the upper body portion 2. In addition, a gripping portion 20 (details will be described later) for gripping the load 100 is attached to the tips of the arm portions 5 and 6. Note that the number of arm portions is not limited to two, and may be one or three or more.

[0020] The leg portion 3 has two wheels 7 and 8 attached to the lower part thereof, and is configured to be movable on the floor on which the humanoid robot 1 is placed.

[0021] The connecting portion 4 rotatably connects the upper body portion 2 and the leg portion 3. For this reason, the upper body portion 2 can be tilted forward and backward with respect to the leg portion 3. Note that the leg portion 3 has a balance function to prevent the humanoid robot 1 from falling when the upper body portion 2 tilts forward or backward with respect to the leg portion 3 or when the humanoid robot 1 moves.

[0022] In addition, as shown in FIG. 1, the connecting portion 4 has a function of changing the distance between the upper body portion 2 and the leg portion 3. For this reason, the vertical position of the upper body portion 2 with respect to the leg portion 3 can be adjusted as indicated by the arrow A so as to match the height of the workbench in the production line.

[0023] In addition, the humanoid robot 1 according to the present embodiment is driven under the control of a control system 10 installed inside the humanoid robot 1.

[0024] (Structure of the grasping part 20)

[0025] As shown in FIG. 2, the grasping part 20 attached to the tips of the arm parts 5 and 6 has the same hand structure as that of a human (Intelligent Hand System).

[0026] As shown in FIG. 2, the grasping part 20 according to the present embodiment includes a palm part as a base corresponding to a so-called human palm, and five finger parts 22A, 22B, 22C, 22D, and 22E each having a plurality of joints are attached to the palm part. In the present embodiment, the number of fingers of the grasping part 20L is five, but a finger structure with a different number of fingers such as three fingers may also be used.

[0027] A palm sensor 26 is attached to the palm part. The high-resolution camera that constitutes the palm sensor 26 according to the present embodiment identifies what the photographed load 100 is based on the photographed image information.

[0028] In other words, the high-resolution camera has a role of acquiring information for specifying the type (shape, size, hardness, etc.) of the load 100.

[0029] On the other hand, together with the high-resolution camera, the MoPU (Motion Processing Unit) that constitutes the palm sensor 26 of the present embodiment outputs motion information indicating the motion of the photographed load 100 (in this case, the relative motion between the arm parts 5 and 6) from the image of the load 100 photographed at a frame rate of 1000 frames / second or more, for example, at a frame rate of 1000 frames / second or more. When detecting the moving load 100, the frame rate may be increased, and when detecting a fixed object (non-moving load 100), the frame rate may be decreased.

[0030] The MoPU outputs, as movement information, vector information of the movement of the point indicating the position of the package 100 along a predetermined coordinate axis. That is, the movement information output from the MoPU does not include information necessary to identify what the photographed package 100 is (the above-mentioned care products, food), and only includes information indicating the movement (movement direction and movement speed) of the center point (or center of gravity point) of the package 100 on the coordinate axes (x-axis, y-axis, z-axis).

[0031] That is, it is possible to accurately guide the trajectory when the gripping part 20 approaches the package 100.

[0032] The information output from the palm sensor 26 including the high-resolution camera and the MoPU is output to the information processing device 14 (see FIG. 3).

[0033] Based on the information from the palm sensor 26 including the high-resolution camera and the MoPU, the information processing device 14 accurately identifies the position of the package 100, calculates the degree of spread of the finger parts 22A, 22B, 22C when gripping, the strength when grasping, and the adsorption force by the adsorption pad 24, etc., and can accurately control the minute movements of the arm parts 5, 6 and the gripping part 20, and can respond to the picking operations of various packages 100.

[0034] FIG. 3 is a schematic diagram of an example of a control system of the humanoid robot according to the present embodiment. The control system 10 includes a sensor 12 mounted on the humanoid robot, a palm sensor 26 including a high-resolution camera and a MoPU, and an information processing device 14.

[0035] Sensor 12 sequentially acquires information that at least represents the distances and angles between the luggage 100 with which the humanoid robot 1 works and the arm parts 5 and 6, which are around the humanoid robot 1. As the sensor 12, a camera with the highest performance, a solid-state LiDAR (Light Detection And Ranging), a multi-color laser coaxial displacement meter, or various other sensor groups can be adopted. Additionally, as the sensor 12, there are a vibrometer, a thermal camera, a hardness meter, a radar, a LiDAR, a high-pixel, telephoto, ultra-wide-angle, 360-degree, high-performance camera, vision recognition, fine sound, ultrasonic waves, vibration, infrared rays, ultraviolet rays, electromagnetic waves, temperature, humidity, spot AI weather forecast, high-precision multi-channel GPS, low-altitude satellite information, or long-tail incident AI data, etc.

[0036] In addition to the above information, the sensor 12 detects images, distances, vibrations, heat, smells, colors, sounds, ultrasonic waves, ultraviolet rays, or infrared rays, etc. Other information detected by the sensor 12 includes the detection of the center-of-gravity movement of the humanoid robot 1, the detection of the material of the floor on which the humanoid robot 1 is installed, the detection of the outside air temperature, the detection of the outside air humidity, the detection of the vertical, horizontal, and diagonal inclination angles of the floor, the detection of the moisture content, etc.

[0037] The sensor 12 performs these detections, for example, every nanosecond.

[0038] The palm sensor 26 (high-resolution camera and MoPU) is a sensor provided at the gripping parts 20 of the arm parts 5 and 6. Separately from the sensor 12, it has a camera function for photographing the luggage 100 and a position-specifying function for specifying the position of the luggage 100.

[0039] When using one MoPU12, it is possible to acquire vector information of the movement along each of the two coordinate axes (x-axis and y-axis) in the three-dimensional orthogonal coordinate system of the point indicating the existence position of the luggage 100. Utilizing the principle of a stereo camera, when using two MoPU12s, it may output vector information of the movement along each of the three coordinate axes (x-axis, y-axis, and z-axis) in the three-dimensional orthogonal coordinate system of the point indicating the existence position of the luggage 100. The z-axis is the axis along the depth direction (the running of the vehicle).

[0040] The information processing apparatus 14 includes an information acquisition unit 140, a control unit 142, and an information storage unit 144.

[0041] The information acquisition unit 140 acquires information on the load 100 detected by the sensor 12 and the palm sensor 26 (high-resolution camera and MoPU).

[0042] The control unit 142 controls the rotation operation of the connecting part 4, the vertical movement operation, and the operations of the arm parts 5 and 6, etc., using the information acquired by the information acquisition unit 140 from the sensor 12 and AI (Artificial Intelligence).

[0043] As shown in FIG. 4, in the hand tool 50 according to the present embodiment, sensor groups with different information acquisition functions are attached to the respective finger parts 22A, 22B, 22C, 22D, and 22E. In other words, sensor groups for acquiring various information necessary for controlling the robot multimodally are attached.

[0044] In the present embodiment, the following sensors are attached to each of the finger parts 21A to 21E. Finger part 21A (thumb): A visual sensor 51A that detects visual information as attribute information Finger part 21B (index finger): An auditory sensor 51B that detects auditory information as attribute information Finger part 21C (middle finger): An olfactory sensor 51C that detects olfactory information as attribute information Finger part 21D (ring finger): A tactile sensor 51D that detects tactile information as attribute information Finger part 21E (little finger): A gustatory sensor 51E that detects gustatory information as attribute information

[0045] Examples of the visual sensor 51A include a camera, an infrared camera, etc.

[0046] Examples of the auditory sensor 51B include a microphone, etc.

[0047] The olfactory sensor 51C is a sensor that reacts to odors, and the output may be any of current, voltage, light quantity, etc.

[0048] Examples of the tactile sensor 51D include a pressure sensor, an optical sensor S, etc. A mechanical switch may also be used. In addition, as the tactile sensor, those including phenomena felt by the skin, for example, a temperature sensor, a humidity sensor, etc. are assumed.

[0049] Examples of the taste sensor 51E include a bitter taste sensor, a sour taste sensor, a umami taste sensor, a salty taste sensor, astringency taste sensor, etc.

[0050] Note that each of the sensor units 51A to 51E is not limited to the above, and the detection form is also not particularly limited, such as contact or non-contact. Further, if necessary, detection sensors of the same type of attribute may be mounted.

[0051] FIG. 5 is a functional block diagram related to each step (command interpretation → command conversion → robot execution → feedback and adaptation) for controlling a robot multimodally, which is executed in the information processing apparatus 14 (see FIG. 3). Note that each block shown in FIG. 5 is classified by function, and a part or all of the information acquisition control function may be operated by a software program using a microcomputer (including an ASIC or the like).

[0052] Here, multimodal means using not only a single piece of information such as only image recognition or only language processing, but also a plurality of pieces of information.

[0053] More specifically, problem settings of multimodal deep learning may be classified into five categories: Representation, Translation, Alignment, Fusion, and Co-learning.

[0054] Representation is the task of solving how to represent or summarize multimodal data. For example, can text information and audio signal data be handled in the same space, etc.

[0055] Translation is the task of converting data of one modality into data of another modality. For example, generating a description from an image, etc.

[0056] Alignment is the task of clarifying the direct relationship between multiple modalities. For example, linking recipe information (text) and information in an image to accurately rearrange each scene of a video of cooking, etc.

[0057] Fusion is the task of using information from multiple modalities to make a certain prediction. For example, using speech audio and the speaker's mouth movements (video) to accurately predict speech content, etc.

[0058] Co-learning is the task of transferring an inference model, vector representation, etc. created within a certain modality to another modality. For example, Zero-shot learning, etc.

[0059] To control a robot using generative AI, especially natural language processing models, the following steps are included.

[0060] (Command interpretation) The AI interprets the command given in human natural language and converts it into an instruction that the robot can understand.

[0061] (Command conversion) The interpreted command is converted into an appropriate format (e.g., a specific code or signal) according to the robot's control system.

[0062] (Robot Execution) The robot operates according to the converted instructions. This may include physical actions, data collection, or the execution of specific tasks.

[0063] (Feedback and Adaptation) The AI receives feedback from the robot (such as the execution status and problems), analyzes it, and adjusts the actions as necessary.

[0064] In this embodiment, not only the commands given in human natural language are processed linguistically, but also the information obtained from each sensor unit 51A to 51E, that is, the information related to vision, hearing, smell, touch, and taste is utilized to control the humanoid robot 1.

[0065] First, the object attribute determination control function of the information processing device 14 includes a data acquisition unit 70. The data acquisition unit 70 acquires the detection data of each sensor unit 51A to 51E attached to the fingertips of the hand tool 50. The object attribute determination control function of the information processing device 14 includes a data acquisition unit 70. The data acquisition unit 70 acquires the detection data of each sensor unit 51A to 51E attached to the fingertips of the hand tool 50.

[0066] The data acquisition unit 70 is connected to a collation unit 72. The data acquisition unit 72 sends the acquired detection data from each sensor unit 51A to 51E to the collation unit 72.

[0067] The information processing device 14 includes a data generation model 73. The data generation model 73 is a so-called generative AI (Artificial Intelligence). Examples of the data generation model 73 include generative AIs such as ChatGPT (Internet search <URL: https: / / openai.com / blog / chatgpt>), Gemini (Internet search <URL: https: / / japan.googleblog.com / 2023 / 12 / gemini.html>), etc. The data generation model 73 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 73, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is input. The data generation model 73 infers the input inference data according to the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization, etc.

[0068] Here, the information processing device 14 receives, from outside the device, a command given in natural language of a human as a work specification. The data generation model 73 interprets the command and converts it into an instruction sentence for controlling the humanoid robot 1. Note that the data generation model 73 may be provided in a server outside the robot 10, and the server may send an instruction to the information processing device 14 by issuing a command as an instruction in the server.

[0069] In addition, the information acquisition control function of the information processing device 14 includes a work information acquisition unit 74. The work information acquisition unit 74 acquires an instruction sentence for controlling the humanoid robot 1 from the data generation model 73. This work information acquisition unit 74 is connected to a search unit 76 and sends out work information.

[0070] The search unit 76 accesses the work type - attribute information database 78 and reads out the attribute information of the specified work type. The attribute information includes the selection and importance of various sensors, etc., and based on this attribute information, the priority of the acquired information can be determined. The attribute information read by the search unit 76 is sent to the collation unit 72.

[0071] Here, the collation unit 72 collates the acquired information received from the search unit 76 with the attribute information received from the data acquisition unit 70, and sends the collation result to the work control unit (not shown), which is another function of the information processing device 14. That is, in the collation unit 72, the command sentence for controlling the humanoid robot 1 is converted into an appropriate format (for example, a specific code or signal) in the work control unit (not shown). The collation unit 72 is an example of a conversion unit.

[0072] Note that when the humanoid robot 1 operates, as feedback and adaptation processing, the work information acquisition unit 74 receives feedback (for example, execution status and problems) from the work control unit (not shown), and analyzes the content of the feedback in the data generation model 73. Then, the operation is adjusted for the work control unit (not shown).

[0073] Hereinafter, the operation of the humanoid robot 1 in the present embodiment will be described according to the flowchart of FIG. 6.

[0074] Hereinafter, the operation of the present embodiment will be described according to the flowchart of FIG. 6. In step 82, sensor functions are attached to the fingertips of each finger of the hand tool 50. As an example, a visual sensor 51A is attached to the thumb, an auditory sensor 51B is attached to the index finger, an olfactory sensor 51C is attached to the middle finger, a tactile sensor 51D is attached to the ring finger, and a taste sensor 51E is attached to the little finger.

[0075] Note that, if necessary, the relationship between the finger type and the sensor type may be changed, or only the necessary and sufficient number of sensors may be attached.

[0076] In the next step 84, each sensor unit 51A to 51E is confronted with the work object, and the process proceeds to step 86.

[0077] In step 86, each sensor unit 51A to 51E acquires information on an individual object (acquired information).

[0078] In the next step 88, work information is acquired. Then, in step 90, attribute information of the work type is read from the work type - attribute information database (DB), and the process proceeds to step 92 to perform collation between the acquired information and the attribute information. That is, collation between the read attribute information and the detected attribute information is performed.

[0079] In step 94, the collation result (operation control information obtained from a plurality of acquired information) is sent to the work control unit, which is another function of the information processing apparatus 14, and this routine ends.

[0080] FIG. 7 schematically shows an example of the hardware configuration of a computer 1200 that functions as the information processing apparatus 14. The program installed in the computer 1200 causes the computer 1200 to function as one or more "units" of the apparatus according to the present embodiment, or causes the computer 1200 to execute an operation or the one or more "units" associated with the apparatus according to the present embodiment, and / or causes the computer 1200 to execute the process or a stage of the process according to the present embodiment. Such a program may be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0081] The computer 1200 according to the present embodiment includes a CPU 1212, a RAM 1214, and a graphic controller 1216, which are mutually connected by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid state drive, or the like. The computer 1200 also includes input / output units such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0082] The CPU 1212 operates according to programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 acquires image data generated by the CPU 1212 in a frame buffer or the like provided in the RAM 1214 or in itself, and causes the image data to be displayed on the display device 1218.

[0083] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads a program or data from a DVD-ROM or the like and provides it to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0084] ROM 1230 stores therein a boot program or the like executed by computer 1200 at activation, and / or a program dependent on the hardware of computer 1200. Input / output chip 1240 may also be connected to input / output controller 1220 via various input / output units such as a USB port, a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0085] The program is provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The program is read from the computer-readable storage medium, installed in storage device 1224, RAM 1214, or ROM 1230, which is also an example of a computer-readable storage medium, and executed by CPU 1212. The information processing described in these programs is read by computer 1200, resulting in cooperation between the programs and the various types of hardware resources described above. The device or method may be configured by realizing the operation or processing of information according to the use of computer 1200.

[0086] For example, when communication is executed between computer 1200 and an external device, CPU 1212 may execute a communication program loaded in RAM 1214 and instruct communication interface 1222 to perform communication processing based on the processing described in the communication program. Communication interface 1222 reads transmission data stored in a transmission buffer area provided in a recording medium such as RAM 1214, storage device 1224, DVD-ROM, or IC card under the control of CPU 1212, transmits the read transmission data to the network, or writes received data received from the network to a reception buffer area or the like provided on the recording medium.

[0087] In addition, the CPU 1212 may cause all or necessary portions of files or databases stored in an external recording medium such as a storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and may execute various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0088] Various types of information such as various types of programs, data, tables, and databases may be stored in a recording medium and may undergo information processing. The CPU 1212 may perform various types of processing on the data read from the RAM 1214, including various types of operations, information processing, conditional judgments, conditional branches, unconditional branches, search / replacement of information, etc. described throughout the present invention and specified by the instruction sequence of the program, and write back the results to the RAM 1214. In addition, the CPU 1212 may search for information in files, databases, etc. within the recording medium. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 1212 searches for an entry that matches the condition where the attribute value of the first attribute is specified among the plurality of entries, reads the attribute value of the second attribute stored in the entry, and thereby may obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0089] The programs or software modules described above may be stored in a computer-readable storage medium on or near the computer 1200. Also, a recording medium such as a hard disk or RAM provided within a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the program to the computer 1200 via the network.

[0090] In the flowcharts and block diagrams in this embodiment, the blocks may represent stages of a process in which operations are performed or "parts" of a device having a role of performing operations. Specific stages and "parts" may be implemented by a dedicated circuit, a programmable circuit supplied with computer-readable instructions stored on a computer-readable storage medium, and / or a processor supplied with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuit may include digital and / or analog hardware circuits, and may include an integrated circuit (IC) and / or discrete circuits. The programmable circuit may include, for example, a field programmable gate array (FPGA), a programmable logic array (PLA), and other logic operations such as logical product, logical sum, exclusive logical sum, negative logical product, negative logical sum, and other logic operations, flip-flops, registers, and memory elements, and may include a reconfigurable hardware circuit.

[0091] The computer-readable storage medium may include any tangible device capable of storing instructions executable by an appropriate device. As a result, a computer-readable storage medium having instructions stored therein will comprise a product including instructions that can be executed to create means for performing the operations specified in the flowchart or block diagram. Examples of the computer-readable storage medium may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of the computer-readable storage medium may include floppy (registered trademark) disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray (registered trademark) disc, memory stick, integrated circuit card, and the like.

[0092] Computer-readable instructions may include any combination of one or more programming languages, including assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source code or object code written in an object-oriented programming language such as Smalltalk®, JAVA®, C++, and a conventional procedural programming language such as the "C" programming language or similar programming languages.

[0093] The computer-readable instructions may be provided locally or via a wide area network (WAN) such as a local area network (LAN), the Internet, etc., to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, or a programmable circuit, to execute the computer-readable instructions to generate means for performing the operations specified in the flowchart or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0094] As described above, the present invention has been described using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments. It is obvious to those skilled in the art that various changes or improvements can be made to the above embodiments. It is clear from the description of the claims that forms with such changes or improvements may also be included in the technical scope of the present invention.

[0095] In the claims, the specification, and the drawings, the execution order of each process such as the operations, procedures, steps, and stages in the apparatus, system, program, and method shown is not explicitly stated as "before" or "preceding" etc. in particular. It should be noted that, unless the output of the previous process is used in the subsequent process, it can be realized in any order. Regarding the operation flows in the claims, the specification, and the drawings, even if explanations are made using "first," "next," etc. for convenience, it does not mean that it is essential to implement in this order.

[0096] As described above, the present invention has been described using embodiments. However, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is obvious to those skilled in the art that various changes or improvements can be made to the above embodiments. It is clear from the description of the claims that forms with such changes or improvements can also be included in the technical scope of the present invention.

[0097] In the claims, the specification, and the drawings, the execution order of each process such as the operations, procedures, steps, and stages in the apparatus, system, program, and method shown is not explicitly stated as "before" or "preceding" etc. in particular. It should be noted that, unless the output of the previous process is used in the subsequent process, it can be realized in any order. Regarding the operation flows in the claims, the specification, and the drawings, even if explanations are made using "first," "next," etc. for convenience, it does not mean that it is essential to implement in this order.

Explanation of Reference Signs

[0098] 1 Humanoid robot, 2 Upper body part, 3 Leg part, 4 Connecting part, 5, 6 Arm parts, 7, 8 Wheels, 10 Control system, 12 Sensor, 14 Information processing device, 20 Gripping part, 22A, 22B, 22C, 22D, 22E Finger parts, 26 Palm sensor, 51A Visual sensor, 51B Auditory sensor, 51C Olfactory sensor, 51D Tactile sensor, 51E Gustatory sensor, 1200 Computer, 1210 Host controller, 1212 CPU, 1214 RAM, 1216 Graphic controller, 1218 Display device, 1220 Input / output controller, 1222 Communication interface, 1224 Storage device, 1230 ROM, 1240 Input / output chip

Claims

1. A control system for a robot, comprising: a work actuator attached to the robot for the robot to perform work; and a sensor unit that acquires a plurality of types of information necessary for the work actuator to perform work. An operation information creation unit that creates operation control information necessary for operation control and feedback control of the robot by collating a plurality of types of information from each of the sensor units with attribute information corresponding to the work. A control unit that controls the work of the robot based on the operation control information. A control system for a robot having the above components.

2. The control system for a robot according to claim 1, further comprising: a camera that captures an image of the object of the work to identify the type of the object; and a motion processing unit that specifies the position of the object.

3. The acquired information acquired by the sensor unit is information related to vision, hearing, smell, touch, and taste, and operation control information is obtained by combining a plurality of acquired information. The control system for a robot according to claim 2.

4. A control program for a robot that causes a computer to operate as the operation information creation unit and the control unit according to any one of claims 1 to 3.

Images