Control system for robot and control program for robot

Abstract

In a hand tool 50, sensors having different information acquisition functions are attached to respective finger parts 22A, 22B, 22C, 22D, and 22E. In other words, sensors for acquiring various kinds of information necessary for controlling a robot in a multimodal manner are attached. A visual sensor 51A for detecting visual information is attached to the thumb, an auditory sensor 51B for detecting auditory information is attached to the index finger, an olfactory sensor 51C is attached to the middle finger, a tactile sensor 51D for detecting tactile information is attached to the ring finger, and a taste sensor 51E for detecting taste information is attached to the little finger.

Description

Robot control system, robot control program

[0001] The present invention relates to a robot control system and a robot control program.

[0002] It has been proposed to use generative AI to operate robots in a multimodal manner (see Patent Documents 1 and 2).

[0003] Multimodal refers to the use of multiple pieces of information, rather than just a single piece of information such as image recognition or language processing.

[0004] In particular, for the robot's execution, feedback and adaptation steps, multiple sensor groups are essential to acquire a wide range of different information.

[0005] JP 2022-081591 A JP 2022-6610 A

[0006] However, there is still no configuration or specification that allows each sensor module in the sensor group to be installed in the appropriate position, acquire information in the appropriate state, and acquire the necessary information at the appropriate time.

[0007] Taking the above facts into consideration, the present disclosure aims to provide a robot control system and a robot control program that can install each sensor module of a sensor group in an appropriate position to acquire various information necessary for multimodal robot control, and that can acquire information from each sensor module in an appropriate state and at an appropriate time.

[0008] The robot control system according to the present disclosure is a robot control system that includes a work actuator attached to a robot for causing the robot to perform a task, and a sensor unit that acquires multiple types of information necessary for the work actuator to perform the task, and includes an operation information creation unit that creates operation control information necessary for the operation control and feedback control of the robot by comparing the multiple types of information from each of the sensor units with attribute information corresponding to the task, and a control unit that controls the robot's task based on the operation control information.

[0009] The present disclosure is characterized in that it further comprises a camera that takes an image of the object of the work and identifies the type of the object, and a motion processing unit that identifies the position of the object.

[0010] The camera identifies the photographed object (hereinafter, sometimes referred to as luggage) based on the captured image information. That is, it has the role of acquiring information to identify the type of object (shape, size, hardness, etc.).

[0011] The motion processing unit (MoPU) outputs, as position information, vector information of the movement of a point indicating the location of an object along predetermined coordinate axes along with the motion information. That is, the motion information output from the MoPU includes only information indicating the movement (movement direction and movement speed) of the object's center point (or center of gravity) on the coordinate axes (x-axis, y-axis, z-axis). That is, the trajectory of the gripper as it approaches the object can be accurately guided.

[0012] The present disclosure is characterized in that the information acquired by the sensor unit is information relating to sight, hearing, smell, touch, and taste, and that operation control information is obtained by combining a plurality of pieces of acquired information.

[0013] A robot control system according to the present disclosure is characterized in that a computer is operated as the operation information creation unit and the control unit of the robot control system.

[0014] It should be noted that the above summary of the disclosure does not list all of the necessary features of the present disclosure, and subcombinations of these features may also be disclosed.

[0015] As described above, according to the present disclosure, it is possible to install each sensor module of a sensor group for acquiring various information necessary for multimodal control of a robot in an appropriate position, and it is possible to acquire information from each sensor module in an appropriate state and at an appropriate time.

[0016] FIG. 1 is a front view of a humanoid robot according to the present embodiment. FIG. 2 is a front view of the palm side of a gripping unit according to the present embodiment. FIG. 3 is a diagram schematically illustrating an example of a functional configuration of a humanoid robot according to the present embodiment. FIG. 4 is a front view of a gripping unit according to the present embodiment. FIG. 5 is a functional block diagram of information acquisition control in an information processing device for executing a task according to the embodiment. FIG. 6 is a control flowchart showing a processing procedure for an information acquisition task according to the present embodiment. FIG. 7 is a diagram schematically illustrating an example of computer hardware that functions as an information processing device.

[0017] The present disclosure will be described below through the disclosed embodiments, but the following embodiments do not limit the disclosure according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the disclosed solution.

[0018] 1 is a front view of a humanoid robot 1 according to this embodiment. As shown in FIG. 1, the humanoid robot 1 according to this embodiment includes an upper body 2, legs 3, and a connecting portion 4 that connects the upper body 2 to the legs 3 in a rotatable manner.

[0019] The upper body 2 has two arms 5 and 6. The arms 5 and 6 are attached to the left and right of the upper body 2 so as to be freely rotatable. Furthermore, a gripping portion 20 (described in detail later) for gripping luggage 100 is attached to the tip of each of the arms 5 and 6. The number of arms is not limited to two, and may be one or three or more.

[0020] The leg 3 has two wheels 7 and 8 attached to its lower part, allowing the humanoid robot 1 to move on the floor on which it is placed.

[0021] The connecting portion 4 rotatably connects the upper body portion 2 and the legs 3. This allows the upper body portion 2 to lean forward and backward relative to the legs 3. The legs 3 have a balancing function to prevent the humanoid robot 1 from falling over when the upper body portion 2 leans forward or backward relative to the legs 3 or when the humanoid robot 1 moves.

[0022] 1, the connecting portion 4 has a function of changing the distance between the upper body portion 2 and the legs 3. Therefore, the vertical position of the upper body portion 2 relative to the legs 3 can be adjusted as shown by arrow A to match the height of the workbench on the production line.

[0023] Furthermore, the driving of the humanoid robot 1 according to this embodiment is controlled by a control system 10 implemented within the humanoid robot 1.

[0024] (Structure of grip portion 20)

[0025] As shown in FIG. 2, the gripping portion 20 attached to the tip of the arms 5 and 6 has a structure similar to that of a human hand (Intelligent Hand System).

[0026] 2, the grip portion 20 according to this embodiment has a palm portion as a base corresponding to a human palm, and five fingers 22A, 22B, 22C, 22D, and 22E, each having a plurality of joints, are attached to the palm portion. In this embodiment, the grip portion 20L has five fingers, but the grip portion 20L may have a different number of fingers, such as three fingers.

[0027] A palm sensor 26 is attached to the palm of the hand. The high-resolution camera that constitutes the palm sensor 26 according to this embodiment identifies the identity of the photographed baggage 100 based on the photographed image information.

[0028] In other words, the high-resolution camera has the role of acquiring information for identifying the type of luggage 100 (shape, size, hardness, etc.).

[0029] Meanwhile, the MoPU (Motion Processing Unit) constituting palm sensor 26 of this embodiment together with the high-resolution camera outputs, from images of luggage 100 captured at a frame rate of 1000 frames per second or higher, motion information indicating the motion of the captured luggage 100 (in this case, the relative motion between arms 5 and 6) at a frame rate of, for example, 1000 frames per second or higher. Note that the frame rate may be increased when detecting moving luggage 100, and decreased when detecting a stationary object (non-moving luggage 100).

[0030] The MoPU outputs, as motion information, vector information of the movement along predetermined coordinate axes of a point indicating the location of the luggage 100. In other words, the motion information output from the MoPU does not include information necessary to identify what the photographed luggage 100 is (the care product or food), but only includes information indicating the movement (movement direction and movement speed) of the center point (or center of gravity) of the luggage 100 on the coordinate axes (x-axis, y-axis, z-axis).

[0031] In other words, the trajectory of the gripping portion 20 as it approaches the luggage 100 can be accurately guided.

[0032] Information output from the palm sensor 26 including the high-resolution camera and the MoPU is output to the information processing device 14 (see FIG. 3).

[0033] The information processing device 14 uses information from a palm sensor 26 including a high-resolution camera and MoPU to pinpoint the position of the luggage 100 with high precision, calculates the degree of spread of the fingers 22A, 22B, 22C when grasping, the strength of the grip, and the suction force of the suction pad 24, and accurately controls the minute movements of the arms 5, 6 and the gripping part 20, making it possible to handle picking operations for a variety of luggage 100.

[0034] 3 is a schematic diagram of an example of a control system for a humanoid robot according to this embodiment. The control system 10 includes a sensor 12 mounted on the humanoid robot, a palm sensor 26 including a high-resolution camera and an MoPU, and an information processing device 14.

[0035] The sensor 12 sequentially acquires information representing at least the distance and angle between the arms 5, 6 and the load 100 around the humanoid robot 1 on which the humanoid robot 1 is working. The sensor 12 may be a high-performance camera, a solid-state LiDAR (Light Detection and Ranging), a multi-color laser coaxial displacement meter, or a variety of other sensors. Other examples of the sensor 12 include a vibration meter, a thermal camera, a hardness tester, radar, LiDAR, a high-resolution, telephoto, ultra-wide-angle, 360-degree, high-performance camera, vision recognition, minute sounds, ultrasound, vibration, infrared rays, ultraviolet rays, electromagnetic waves, temperature, humidity, spot AI weather forecasts, high-precision multi-channel GPS, low-altitude satellite information, and long-tail incident AI data.

[0036] In addition to the above information, the sensor 12 detects images, distance, vibration, heat, smell, color, sound, ultrasound, ultraviolet light, infrared light, etc. Other information detected by the sensor 12 includes the movement of the center of gravity of the humanoid robot 1, the material of the floor on which the humanoid robot 1 is placed, the outside air temperature, the outside air humidity, the up / down / side / diagonal tilt angle of the floor, the amount of moisture, etc.

[0037] The sensor 12 performs these detections, for example, every nanosecond.

[0038] The palm sensor 26 (high-resolution camera and MoPU) is a sensor provided on the gripping portion 20 of the arm portions 5 and 6, and, separate from the sensor 12, has a camera function for photographing the luggage 100 and a position determination function for determining the position of the luggage 100.

[0039] When one MoPU 12 is used, it is possible to acquire vector information of the movement of a point indicating the location of the luggage 100 along each of two coordinate axes (x-axis and y-axis) in a three-dimensional orthogonal coordinate system. Utilizing the principle of a stereo camera, two MoPUs 12 may be used to output vector information of the movement of a point indicating the location of the luggage 100 along each of three coordinate axes (x-axis, y-axis, and z-axis) in the three-dimensional orthogonal coordinate system. The z-axis is the axis along the depth direction (vehicle travel).

[0040] The information processing device 14 includes an information acquisition unit 140 , a control unit 142 , and an information storage unit 144 .

[0041] The information acquisition unit 140 acquires information about the baggage 100 detected by the sensor 12 and the palm sensor 26 (high-resolution camera and MoPU).

[0042] The control unit 142 uses the information acquired by the information acquisition unit 140 from the sensor 12 and AI (Artificial Intelligence) to control the rotational movement of the connecting unit 4, the movement of the arms 5 and 6, and the like.

[0043] 4, in the hand tool 50 according to this embodiment, a group of sensors with different information acquisition functions is attached to each of the fingers 22A, 22B, 22C, 22D, and 22E. In other words, a group of sensors is attached to acquire various information necessary for multimodal control of the robot.

[0044] In this embodiment, the following sensors are attached to each of the finger portions 21A to 21E. Finger portion 21A (thumb): visual sensor 51A that detects visual information as attribute information Finger portion 21B (index finger): auditory sensor 51B that detects auditory information as attribute information Finger portion 21C (middle finger): olfactory sensor 51C that detects olfactory information as attribute information Finger portion 21D (ring finger): tactile sensor 51D that detects tactile information as attribute information Finger portion 21E (little finger): taste sensor 51E that detects taste information as attribute information

[0045] The visual sensor 51A may be a camera, an infrared camera, or the like.

[0046] The auditory sensor 51B may be a microphone or the like.

[0047] The olfactory sensor 51C is a sensor that reacts to odors, and the output may be any of current, voltage, light intensity, and the like.

[0048] Examples of the tactile sensor 51D include a pressure sensor and an optical sensor. A mechanical switch may also be used. The tactile sensor also includes a sensor that detects a phenomenon felt by the skin, such as a temperature sensor or a humidity sensor.

[0049] Examples of the taste sensor 51E include a bitterness sensor, a sourness sensor, an umami sensor, a saltiness sensor, and an astringency sensor.

[0050] Note that the sensor units (51A to 51E) are not limited to those described above, and the detection method is not particularly limited to contact, non-contact, etc. Furthermore, detection sensors of the same type of attribute may be attached as needed.

[0051] 5 is a functional block diagram of each step (command interpretation → instruction conversion → robot execution → feedback and adaptation) executed in the information processing device 14 (see FIG. 3) when controlling a robot multimodally. Note that the blocks shown in FIG. 5 are classified by function, and some or all of the information acquisition and control functions may be operated by a software program using a microcomputer (including ASIC, etc.).

[0052] Here, multimodal refers to the use of multiple pieces of information, rather than just a single piece of information such as image recognition or language processing.

[0053] More specifically, multimodal deep learning problem settings can be classified into five categories: representation, translation, alignment, fusion, and co-learning.

[0054] Representation is the task of solving how to represent or summarize multimodal data, such as whether text information and audio signal data can be handled in the same space.

[0055] Translation is the task of converting data from one modality into data from another modality, such as generating a description from an image.

[0056] Alignment is the task of identifying direct relationships between multiple modalities, such as linking recipe information (text) with information in images in order to accurately reorder each scene in a cooking video.

[0057] Fusion is the task of using information from multiple modalities to make a prediction, such as using speech audio and a video of a speaker's mouth movements to accurately predict speech content.

[0058] Co-learning is the task of transferring inference models or vector representations created in one modality to another, such as zero-shot learning.

[0059] Controlling a robot using generative AI, specifically a natural language processing model, involves the following steps:

[0060] (Command interpretation) AI interprets commands given in natural human language and converts them into instructions that the robot can understand.

[0061] (Instruction Conversion) The interpreted commands are converted into a suitable format (e.g., specific codes or signals) for the robot's control system.

[0062] Robot Execution: The robot acts according to the translated instructions, which may include performing a physical action, collecting data, or performing a specific task.

[0063] (Feedback and Adaptation) Upon receiving feedback from the robot (e.g., performance status and problems), the AI ​​analyzes it and adjusts its behavior as needed.

[0064] In this embodiment, the humanoid robot 1 is controlled not only by processing commands given in natural human language, but also by utilizing information obtained from each sensor unit (51A to 51E), i.e., information relating to vision, hearing, smell, touch, and taste.

[0065] First, the object attribute determination control function of the information processing device 14 includes a data acquisition unit 70. The data acquisition unit 70 acquires detection data from each sensor unit (51A to 51E) attached to the fingertips of the hand tool 50. The object attribute determination control function of the information processing device 14 includes a data acquisition unit 70. The data acquisition unit 70 acquires detection data from each sensor unit (51A to 51E) attached to the fingertips of the hand tool 50.

[0066] The data acquisition unit 70 is connected to the collation unit 72. The data acquisition unit 70 sends the acquired detection data from each sensor unit (51A to 51E) to the collation unit 72.

[0067] The information processing device 14 includes a data generation model 73. The data generation model 73 is a so-called generative AI (artificial intelligence). Examples of the data generation model 73 include generative AI such as ChatGPT (Internet search <URL: https: / / openai.com / blog / chatgpt>) and geminni (Internet search <URL: https: / / japan.googleblog.com / 2023 / 12 / gemini.html>). The data generation model 73 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 73, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 73 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0068] Here, the information processing device 14 receives commands given in human natural language from outside the device as task specifications. The data generation model 73 interprets the commands and converts them into instruction statements for controlling the humanoid robot 1. Note that the data generation model 73 may be provided in a server external to the control system 10, and the server may convert the commands into instructions and send them to the information processing device 14.

[0069] The information acquisition control function of the information processing device 14 also includes a work information acquisition unit 74. The work information acquisition unit 74 acquires command statements for controlling the humanoid robot 1 from the data generation model 73. The work information acquisition unit 74 is connected to a search unit 76 and sends out work information.

[0070] The search unit 76 accesses the work type-attribute information database 78 and reads out the attribute information of the specified work type. The attribute information includes the selection and importance of various sensors, and this attribute information can be used to determine the priority of the acquired information. The attribute information read out by the search unit 76 is sent to the collation unit 72.

[0071] Here, the collation unit 72 compares the acquired information received from the search unit 76 with the attribute information received from the data acquisition unit 70, and sends the comparison result to a work control unit (not shown), which is another function of the information processing device 14. In other words, the collation unit 72 converts command statements for controlling the humanoid robot 1 into an appropriate format (for example, a specific code or signal) for the work control unit (not shown). The collation unit 72 is an example of a conversion unit.

[0072] During the operation of the humanoid robot 1, as a feedback and adaptation process, the task information acquisition unit 74 receives feedback (e.g., execution status and problems) from the task control unit (not shown), and analyzes the content of the feedback in the data generation model 73. Then, the task control unit (not shown) is made to adjust the operation.

[0073] The operation of the humanoid robot 1 in this embodiment will be described below with reference to the flowchart of FIG.

[0074] The operation of this embodiment will be described below with reference to the flowchart in Fig. 6. In step 82, a sensor function is attached to each fingertip of the hand tool 50. As an example, a visual sensor 51A is attached to the thumb, an auditory sensor 51B to the index finger, an olfactory sensor 51C to the middle finger, a tactile sensor 51D to the ring finger, and a taste sensor 51E to the little finger.

[0075] If necessary, the relationship between the finger type and the sensor type may be changed, and only the necessary and sufficient number of sensors may be attached.

[0076] In the next step 84, each sensor unit (51A to 51E) is placed opposite the work object, and the process proceeds to step 86.

[0077] In step 86, information on each individual object is acquired by each sensor unit (51A to 51E) (acquired information).

[0078] In the next step 88, work information is acquired, and then in step 90, work type attribute information is read from the work type-attribute information database (DB), and the process proceeds to step 92, where the acquired information is compared with the attribute information. That is, the read attribute information is compared with the detected attribute information.

[0079] In step 94, the collation result (operation control information obtained from the plurality of pieces of acquired information) is sent to the work control section, which is another function of the information processing device 14, and this routine ends.

[0080] 7 schematically illustrates an example of the hardware configuration of a computer 1200 functioning as the information processing device 14. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "parts" of the device according to the present embodiment, or can cause the computer 1200 to perform operations associated with the device according to the present embodiment or one or more "parts," and / or can cause the computer 1200 to perform a process according to the present embodiment or steps of the process. Such a program can be executed by the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0081] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communications interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid-state drive, or the like. The computer 1200 also includes input / output units such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0082] The CPU 1212 operates according to programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into the graphics controller 1216 itself, and causes the image data to be displayed on the display device 1218.

[0083] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0084] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or programs that depend on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0085] The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, RAM 1214, or ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or a method may be configured by implementing operations or processing of information in accordance with the use of the computer 1200.

[0086] For example, when communication is performed between computer 1200 and an external device, CPU 1212 may execute a communication program loaded into RAM 1214 and instruct communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of CPU 1212, communication interface 1222 reads transmission data stored in a transmission buffer area provided in RAM 1214, storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes received data received from the network to a reception buffer area or the like provided on the recording medium.

[0087] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and may perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0088] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. The CPU 1212 may perform various types of processing on data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries, each having an attribute value of a first attribute associated with an attribute value of a second attribute, are stored on the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0089] The above-described programs or software modules may be stored in a computer-readable storage medium on or near the computer 1200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0090] The blocks in the flowcharts and block diagrams in this embodiment may represent stages of a process in which an operation is performed or "parts" of a device responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. Dedicated circuitry may include digital and / or analog hardware circuitry, including integrated circuits (ICs) and / or discrete circuits. Programmable circuitry may include reconfigurable hardware circuitry, such as field programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0091] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by an appropriate device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture, including instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic, magnetic, optical, electromagnetic, and semiconductor storage media. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disc read-only memories (CD-ROMs), digital versatile discs (DVDs), Blu-ray discs, memory sticks, integrated circuit cards, and the like.

[0092] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0093] Computer-readable instructions may be provided locally or over a local area network (LAN), a wide area network (WAN) such as the Internet, to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or programmable circuitry, such that the processor or programmable circuitry executes the computer-readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0094] Although the present disclosure has been described above using embodiments, the technical scope of the present disclosure is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included in the technical scope of the present disclosure.

[0095] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order.

[0096] Although the present disclosure has been described above using embodiments, the technical scope of the present disclosure is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included in the technical scope of the present disclosure.

[0097] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order.

[0098] The entire disclosure of Japanese Patent Application No. 2023-209614, filed on December 12, 2023, is incorporated herein by reference.

[0099] 1 Humanoid robot, 2 Upper body, 3 Legs, 4 Connecting portion, 5, 6 Arms, 7, 8 Wheels, 10 Control system, 12 Sensor, 14 Information processing device, 20 Grip portion, 22A, 22B, 22C, 22D, 22E Finger portion, 26 Palm sensor, 51A Visual sensor, 51B Auditory sensor, 51C Olfactory sensor, 51D Tactile sensor, 51E Taste sensor, 1200 Computer, 1210 Host controller, 1212 CPU, 1214 RAM, 1216 Graphics controller, 1218 Display device, 1220 Input / output controller, 1222 Communication interface, 1224 Storage device, 1230 ROM, 1240 Input / output chip

Claims

1. A robot control system comprising: a work actuator attached to a robot for causing the robot to perform a task; and a sensor unit for acquiring multiple types of information necessary for the work actuator to perform the task; an operation information creation unit that creates operation control information necessary for the robot's operation control and feedback control by comparing the multiple types of information from each of the sensor units with attribute information corresponding to the task; and a control unit that controls the robot's task based on the operation control information.

2. The robot control system of claim 1, further comprising a camera for taking an image of the work object and identifying the type of the object, and a motion processing unit for identifying the position of the object.

3. A robot control system according to claim 2, wherein the information acquired by the sensor units is information relating to vision, hearing, smell, touch, and taste, and operation control information is obtained by combining a plurality of pieces of acquired information.

4. A robot control program that causes a computer to operate as the motion information creation unit and the control unit according to any one of claims 1 to 3.

Images