Robot control system, robot control program

Abstract

The hand tool 50 is equipped with sensor groups with different information acquisition functions on each of the fingers 22A, 22B, 22C, 22D, and 22E. In other words, it is equipped with sensor groups for acquiring various information required for multimodal robot control. A visual sensor 51A for detecting visual information is installed on the thumb, an auditory sensor 51B for detecting auditory information is installed on the index finger, an olfactory sensor 51C is installed on the middle finger, a tactile sensor 51D for detecting tactile information is installed on the ring finger, and a taste sensor 51E for detecting taste information is installed on the little finger.

Description

Technical Field

[0001] This invention relates to a robot control system and a robot control program. Background Technology

[0002] A technique for using generative artificial intelligence (AI) to create multimodal mobile robots has been proposed (see Patent Document 1 and Patent Document 2).

[0003] Multimodal learning utilizes not only single pieces of information such as image recognition and language processing, but also a variety of other types of information.

[0004] In particular, multiple sensor arrays are indispensable for the robot's execution, feedback, and adaptation processes, as well as for acquiring a wide range of information from various aspects.

[0005] Existing technical documents Patent documents Patent document 1: Japanese Patent Application Publication No. 2022-081591.

[0006] Patent Document 2: Japanese Patent Application Publication No. 2022-6610. Summary of the Invention

[0007] The problem that the invention aims to solve However, the following structure was not achieved: the sensor modules of the sensor group were positioned in appropriate locations and acquired information in appropriate states and at appropriate times, acquiring the required information, and the required information was not acquired.

[0008] In view of the above facts, the present invention aims to provide a robot control system and a robot control program that can set each sensor module of a sensor group for acquiring various information required for multimodal robot control in appropriate positions, and can acquire information from each sensor module in appropriate states and at appropriate times.

[0009] Methods for solving problems The robot control system of the present invention comprises: a work actuator mounted on the robot and used by the robot to perform a work; and a sensor unit that acquires various information required for the work actuator to perform the work. The robot control system further comprises: a motion information creation unit that compares various information from each of the sensor units with attribute information corresponding to the work to create motion control information required for the robot's motion control and feedback control; and a control unit that controls the robot's work based on the motion control information.

[0010] In this disclosure, the device is characterized by further comprising: a camera that captures images of the object to be worked on to identify the type of the object; and a motion processing unit that determines the position of the object.

[0011] The camera identifies the object being photographed (hereinafter sometimes referred to as cargo) based on the image information it captures. That is, it has the function of acquiring information to determine the type (shape, size, hardness, etc.) of the object.

[0012] The Motion Processing Unit (MoPU) outputs position information along with the motion information, consisting of vector information representing the movement of a point along a specified coordinate axis. That is, the motion information output from the MoPU only includes information representing the movement (direction and speed) of the object's center point (or center of gravity) along the coordinate axes (x-axis, y-axis, z-axis). This allows for high-precision guidance of the handling unit's trajectory as it approaches the object.

[0013] In this disclosure, the information acquired by the sensor unit is related to vision, hearing, smell, touch and taste, and motion control information is obtained by combining multiple pieces of acquired information.

[0014] The robot control system of the present invention is characterized in that a computer is used as the motion information creation unit and the control unit of the robot control system to perform actions.

[0015] Furthermore, the above-described summary does not list all the features required for this disclosure. Additionally, sub-combinations of these feature groups may also be disclosed.

[0016] Invention Effects As explained above, according to this disclosure, the following effects are achieved: each sensor module of the sensor group used to acquire various information required for multimodal robot control is positioned appropriately, and information is acquired from each sensor module in the appropriate state and at the appropriate time. Attached Figure Description

[0017] Figure 1 This is the front view of the humanoid robot involved in this embodiment.

[0018] Figure 2 This is a front view of the palm side of the gripping part according to this embodiment.

[0019] Figure 3 This is a diagram that schematically illustrates an example of the functional structure of the humanoid robot involved in this embodiment.

[0020] Figure 4 This is a front view of the gripping part according to this embodiment.

[0021] Figure 5 It is a block diagram of an information acquisition and control function in an information processing device used to perform the operations involved in the implementation method.

[0022] Figure 6 This is a control flowchart illustrating the processing sequence of the information acquisition operation involved in this embodiment.

[0023] Figure 7 This is a diagram that schematically illustrates an example of computer hardware functioning as an information processing device. Detailed Implementation

[0024] The present disclosure will now be described through the disclosed embodiments, but these embodiments do not limit the scope of the claims. Furthermore, the combinations of features described in the embodiments are not necessarily all necessary for the disclosed technical solution.

[0025] Figure 1 This is a front view of the humanoid robot 1 according to this embodiment. Figure 1 As shown, the humanoid robot 1 according to this embodiment includes: an upper body 2, legs 3, and a connecting part 4 that rotatably connects the upper body 2 to the legs 3.

[0026] The upper body 2 has two arms 5 and 6. The arms 5 and 6 are rotatably mounted on the left and right sides of the upper body 2. In addition, a gripping part 20 for holding the cargo 100 is installed at the front end of the arms 5 and 6 (details will be described later). Furthermore, the number of arms is not limited to two, but may be one or more than three.

[0027] The legs 3 are equipped with two wheels 7 and 8 on their lower part, enabling them to move on the floor provided for the humanoid robot 1.

[0028] The connecting part 4 connects the upper body 2 and the legs 3 in a rotatable manner. Therefore, the upper body 2 can lean forward and backward relative to the legs 3. It should be noted that the legs 3 have a balancing function to prevent the humanoid robot 1 from falling over when the upper body 2 leans forward or backward relative to the legs 3 or when the humanoid robot 1 moves.

[0029] In addition, such as Figure 1 As shown, the connecting part 4 has the function of changing the distance between the upper body 2 and the legs 3. Therefore, the vertical position of the upper body 2 relative to the legs 3 can be adjusted as shown by arrow A to match the height of the workbench on the production line.

[0030] In addition, the humanoid robot 1 involved in this embodiment is driven by a control system 10 installed inside the humanoid robot 1.

[0031] (The structure of the holding part 20) like Figure 2 As shown, the gripping part 20 installed at the front end of the arm parts 5 and 6 is designed with the same structure as a human hand (Intelligent Hand System).

[0032] like Figure 2 As shown, the gripping part 20 according to this embodiment has a palm as the base, which is equivalent to a human palm, and five fingers 22A, 22B, 22C, 22D, and 22E, each with multiple joints, are installed on the palm. It should be noted that although the number of fingers in the gripping part 20L is set to five in this embodiment, it can also be a different number of fingers, such as three fingers.

[0033] A palm sensor 26 is installed in the palm. The high-resolution camera constituting the palm sensor 26 in this embodiment identifies what the captured goods 100 are based on the captured image information.

[0034] In other words, the high-resolution camera has the function of acquiring information for determining the type (shape, size, hardness, etc.) of the goods 100.

[0035] On the other hand, the MoPU (Motion Processing Unit) of the palm sensor 26, which together with the high-resolution camera constitutes this embodiment, outputs motion information representing the motion of the captured cargo 100 (in this case, the relative motion with respect to the arms 5 and 6) from images of the cargo 100 captured at a frame rate of 1000 frames per second or higher, at a frame rate of, for example, 1000 frames per second or higher. It should be noted that the frame rate can be increased when detecting a moving cargo 100, and decreased when detecting a stationary object (the non-moving cargo 100).

[0036] The MoPU outputs motion information as vector information representing the movement of the point where the goods 100 is located along a specified coordinate axis. That is, the motion information output from the MoPU does not include the information needed to identify what the captured goods 100 is (the aforementioned care products, food), but only includes information representing the movement (direction of movement and speed) of the center point (or center of gravity) of the goods 100 along the coordinate axes (x-axis, y-axis, z-axis).

[0037] That is, it can guide the trajectory of the holding part 20 when it approaches the cargo 100 with high precision.

[0038] Information output from the palm sensor 26, which includes a high-resolution camera and a MoPU, is sent to the information processing device 14 (see reference). Figure 3 Output the results.

[0039] The information processing device 14 determines the position of the goods 100 with high precision based on information from the palm sensor 26, which includes a high-resolution camera and MoPU. It calculates the degree of opening of the fingers 22A, 22B, and 22C when gripping, the strength of the grip, and the adsorption force of the adsorption pad 24, and controls the minute movements of the arms 5 and 6 and the gripping part 20 with high precision, so as to cope with the picking operations of various goods 100.

[0040] Figure 3 This is a schematic diagram of an example of the control system of the humanoid robot according to this embodiment. The control system 10 includes: a sensor 12 mounted on the humanoid robot; a palm sensor 26 including a high-resolution camera and a MoPU; and an information processing device 14.

[0041] Sensor 12 successively acquires information representing at least the distance and angle between the cargo 100 being handled by the humanoid robot 1 and the arms 5 and 6, located around the humanoid robot 1. Sensor 12 can be a high-performance camera, a solid-state laser detection and ranging (LiDAR) sensor, a multi-color laser coaxial displacement meter, or other various sensor combinations. Other possible sensors for sensor 12 include vibration meters, thermal cameras, hardness testers, radar, LiDAR, high-resolution telephoto ultra-wide-angle 360-degree high-performance cameras, visual recognition, micro-sound, ultrasound, vibration, infrared, ultraviolet, electromagnetic waves, temperature, humidity, Spot AI weather forecasts, high-precision multi-channel Global Positioning System (GPS), low-altitude satellite information, or long-tail event AI data.

[0042] It should be noted that, in addition to the information mentioned above, sensor 12 also detects images, distance, vibration, heat, odor, color, sound, ultrasound, ultraviolet or infrared radiation, etc. Furthermore, information detected by sensor 12 can include the movement of the humanoid robot 1's center of gravity, detection of the material of the floor on which the humanoid robot 1 is set, detection of external air temperature, detection of external air humidity, detection of the vertical and lateral tilt angles of the floor, and detection of moisture content, etc.

[0043] Sensor 12 performs these detections, for example, every nanosecond.

[0044] The palm sensor 26 (high-resolution camera and MoPU) is a sensor installed on the gripping part 20 of the arm 5 and 6. Unlike the sensor 12, it also has the function of a camera to capture images of the cargo 100 and the function of determining the position of the cargo 100.

[0045] It should be noted that using one MoPU 12, the vector information of the motion of the point representing the location of cargo 100 along each of the two coordinate axes (x-axis and y-axis) in a three-dimensional orthogonal coordinate system can be obtained. Alternatively, using the principle of a stereo camera, two MoPU 12s can be used to output the vector information of the motion of the point representing the location of cargo 100 along each of the three coordinate axes (x-axis, y-axis, and z-axis) in a three-dimensional orthogonal coordinate system. The z-axis is the axis along the depth direction (vehicle travel).

[0046] The information processing device 14 includes an information acquisition unit 140, a control unit 142, and an information storage unit 144.

[0047] The information acquisition unit 140 acquires information about the cargo 100 detected by the sensor 12 and the palm sensor 26 (high-resolution camera and MoPU).

[0048] The control unit 142 uses the information obtained by the information acquisition unit 140 from the sensor 12 and AI (Artificial Intelligence) to control the rotation of the connecting part 4, the vertical movement, and the movements of the arms 5 and 6.

[0049] like Figure 4 As shown, the hand tool 50 of this embodiment is equipped with sensor groups with different information acquisition functions on each of its finger parts 22A, 22B, 22C, 22D, and 22E. In other words, it is equipped with sensor groups for acquiring various information required for multimodal robot control.

[0050] In this embodiment, the sensors shown below are installed on each finger portion 21A to 21E.

[0051] The finger 21A (thumb) is a visual sensor 51A that detects visual information as attribute information. The finger 21B (index finger) is an auditory sensor 51B that detects auditory information as attribute information. The finger 21C (middle finger) is an olfactory sensor 51C that detects olfactory information as attribute information. The 21D finger (ring finger) is a tactile sensor 51D that detects tactile information as attribute information. The finger 21E (little finger) is a taste sensor 51E that detects taste information as attribute information. As a visual sensor, the 51A can include cameras, infrared cameras, etc.

[0052] Examples of auditory sensors 51B include microphones.

[0053] As an olfactory sensor 51C, it can be a sensor that reacts to odors, and its output can be any of the following: current, voltage, light intensity, etc.

[0054] Examples of tactile sensors 51D include pressure sensors and optical sensors S. It should be noted that mechanical switches are also possible. Furthermore, tactile sensors can also detect phenomena felt by the skin, such as temperature sensors and humidity sensors.

[0055] As a taste sensor 51E, examples include bitter taste sensors, sour taste sensors, umami taste sensors, salty taste sensors, and astringent taste sensors.

[0056] It should be noted that the sensor units (51A to 51E) are not limited to those described above, and the detection method is not specifically limited to contact or non-contact. Furthermore, detection sensors with the same properties can be installed as needed.

[0057] Figure 5 In the information processing device 14 (refer to) Figure 3 This is a functional block diagram illustrating the steps involved in multimodal robot control (instruction interpretation → command conversion → robot execution → feedback and adaptation) during the process. It should be noted that... Figure 5 The modules shown are categorized by function. Microcomputers (including ASICs) can also be used to enable some or all of the information acquisition and control functions to be performed through software programs.

[0058] Here, multimodal refers to utilizing not only single pieces of information such as image recognition and language processing, but also multiple types of information.

[0059] More specifically, as a problem setting for multimodal deep learning, it is sometimes classified into five categories: Representation, Translation, Alignment, Fusion, and Co-learning.

[0060] Representation is the task of solving how to represent or summarize multimodal data, such as whether text information and sound signal data can be processed in the same space.

[0061] Translation is the task of converting data from one modality to another, such as generating descriptive statements from images.

[0062] Alignment is the task of making the direct relationships between multiple modalities clear. For example, it can be used to combine recipe information (text) with information in images in order to accurately rearrange the scenes in a video of making a dish.

[0063] Fusion is a task that utilizes information from multiple modalities to make a prediction. Examples include using the speaker's voice and mouth movements (video) to accurately predict the content of a speech.

[0064] Co-learning is the task of transferring inference models, vector representations, etc., created in one modality to another modality. Examples include zero-shot learning.

[0065] Using generative AI, especially natural language processing models, to control robots involves the following steps.

[0066] (Instruction Interpretation) AI interprets instructions given in human natural language and converts them into commands that robots can understand.

[0067] (Command Conversion) The interpreted instructions are converted into appropriate forms (e.g., specific codes, signals) in conjunction with the robot's control system.

[0068] (Robot Execution) The robot performs actions according to the translated commands. This includes physical actions, data collection, or the execution of specific tasks.

[0069] (Feedback and Adaptation) The AI ​​receives feedback from the robot (e.g., performance status, problems), analyzes it, and adjusts its actions as needed.

[0070] In this embodiment, not only are the instructions given in human natural language processed, but the information obtained from each sensor unit (51A-51E), namely information related to vision, hearing, smell, touch and taste, is also flexibly used to control the humanoid robot 1.

[0071] First, the object attribute determination and control function of the information processing device 14 includes a data acquisition unit 70. The data acquisition unit 70 acquires detection data from each sensor unit (51A-51E) installed on the fingertips of the hand tool 50.

[0072] The data acquisition unit 70 is connected to the comparison unit 72. The data acquisition unit 70 sends the detection data acquired from each sensor unit (51A-51E) to the comparison unit 72.

[0073] The information processing device 14 includes a data generation model 73. The data generation model 73 is a so-called generative AI (Artificial Intelligence). Examples of generative AIs include ChatGPT (Internet search <URL:https: / / openai.com / blog / chatgpt>) and gemini (Internet search <URL:https: / / japan.googleblog.com / 2023 / 12 / gemini.html>). The data generation model 73 is obtained through deep learning of a neural network. The data generation model 73 is input with a prompt containing instructions, as well as reasoning data such as sound data (representing sound), text data (representing text), and image data (representing images). The data generation model 73 performs reasoning on the input reasoning data according to the instructions shown in the prompt and outputs the reasoning results in the form of sound data and text data. Here, reasoning refers to, for example, analysis, classification, prediction, and / or generalization.

[0074] Here, the information processing device 14 receives instructions given in human natural language from outside the device as work specifications. The data generation model 73 interprets the instructions and converts them into command statements for controlling the humanoid robot 1. It should be noted that the data generation model 73 can also be set on a server outside the control system 10, and the instructions can be converted into commands and sent to the information processing device 14 in the server.

[0075] In addition, the information processing device 14 includes a job information acquisition unit 74 for its information acquisition and control function. The job information acquisition unit 74 acquires command statements for controlling the humanoid robot 1 from the data generation model 73. This job information acquisition unit 74 is connected to the retrieval unit 76 and sends out the job information.

[0076] The retrieval unit 76 accesses the job type-attribute information database 78 and reads the attribute information for the specified job type. The attribute information includes the selection and importance of various sensors, and can determine the priority of acquired information based on this attribute information. The attribute information read by the retrieval unit 76 is sent to the comparison unit 72.

[0077] Here, in the comparison unit 72, the acquired information received from the retrieval unit 76 is compared with the attribute information received from the data acquisition unit 70, and the comparison result is sent to the operation control unit (not shown), which is another function of the information processing device 14. That is, in the comparison unit 72, the command statements used to control the humanoid robot 1 are converted into an appropriate form (e.g., a specific code or signal) in the operation control unit (not shown). The comparison unit 72 is an example of a conversion unit.

[0078] It should be noted that, during the movement of the humanoid robot 1, as a feedback and adaptation process, the job information acquisition unit 74 receives feedback (e.g., execution status, problem points) from the job control unit (not shown), and parses the content of the feedback in the data generation model 73. Then, the job control unit (not shown) adjusts its movements.

[0079] The following is in accordance with Figure 6 The flowchart is used to illustrate the actions of the humanoid robot 1 in this embodiment.

[0080] The following is in accordance with Figure 6 The flowchart below illustrates the function of this embodiment. In step 82, sensor functions are installed on each fingertip of the hand tool 50. For example, a visual sensor 51A is installed on the thumb, an auditory sensor 51B is installed on the index finger, an olfactory sensor 51C is installed on the middle finger, a tactile sensor 51D is installed on the ring finger, and a taste sensor 51E is installed on the little finger.

[0081] It should be noted that, as needed, the relationship between the type of index and the type of sensor can be changed, and a sufficient number of sensors can be installed.

[0082] In the next step 84, each sensor unit (51A-51E) is positioned opposite the object to be worked on, and the process proceeds to step 86.

[0083] In step 86, information about each object is acquired using each sensor unit (51A-51E).

[0084] In the next step 88, job information is obtained. Then, in step 90, the attribute information of the job type is read from the job type-attribute information database (DB), and the process moves to step 92 to perform a comparison between the obtained information and the attribute information. That is, the read attribute information is compared with the detected attribute information.

[0085] In step 94, the comparison results (motion control information obtained from multiple acquisition information) are sent to the operation control unit, which is another function of the information processing device 14, and the routine ends.

[0086] Figure 7This document provides a summary example of the hardware configuration of a computer 1200 that functions as an information processing device 14. Programs installed in the computer 1200 enable it to function as one or more "units" of the apparatus according to this embodiment, or to perform operations associated with the apparatus or those one or more "units," and / or to execute processes or stages of those processes according to this embodiment. Such programs can be executed by the CPU 1212 to enable the computer 1200 to perform specific operations associated with several or all of the blocks in the flowcharts and block diagrams described in this specification.

[0087] The computer 1200 of this embodiment includes a CPU 1212, RAM 1214, and a graphics controller 1216, which are interconnected via a main controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card driver, which are connected to the main controller 1210 via an input / output controller 1220. The DVD drive can be a DVD-ROM drive or a DVD-RAM drive, etc. The storage device 1224 can be a hard disk drive or a solid-state drive, etc. The computer 1200 also includes input / output units such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0088] The CPU 1212 operates according to the program stored in the ROM 1230 and RAM 1214, thereby controlling each unit. The graphics controller 1216 retrieves the image data generated by the CPU 1212 from the frame buffer provided in RAM 1214 or from itself, so that the image data is displayed on the display device 1218.

[0089] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 within the computer 1200. The DVD drive reads programs or data from a DVD-ROM or similar device and provides them to the storage device 1224. The IC card driver reads programs and data from an IC card and / or writes programs and data to an IC card.

[0090] ROM 1230 stores boot programs and / or programs dependent on the hardware of computer 1200 that are executed by computer 1200 upon activation. Input / output chip 1240 can also connect various input / output units to input / output controller 1220 via USB port, parallel port, serial port, keyboard port, mouse port, etc.

[0091] The program is provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The program is read from the computer-readable storage medium, installed in a storage device 1224, RAM 1214, or ROM 1230 (also examples of computer-readable storage media), and executed by the CPU 1212. Information processing described within these programs is read by the computer 1200, enabling cooperation between the program and the aforementioned types of hardware resources. The apparatus or method can be configured to perform information manipulation or processing according to the use of the computer 1200.

[0092] For example, when communication is performed between computer 1200 and an external device, CPU 1212 can execute a communication program loaded into RAM 1214 and instruct communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of CPU 1212, communication interface 1222 reads transmission data stored in a transmission buffer provided in a recording medium such as RAM 1214, storage device 1224, DVD-ROM, or IC card, and sends the read transmission data to the network, or writes received data received from the network to a receive buffer provided on the recording medium.

[0093] Additionally, the CPU 1212 can read all or a portion of files or databases stored in external recording media such as storage device 1224, DVD drive (DVD-ROM), or IC card from RAM 1214, and perform various types of processing on the data in RAM 1214. Then, the CPU 1212 can write the processed data back to the external recording media.

[0094] Various types of information, such as programs, data, tables, and databases, can be stored in the recording medium and processed. The CPU 1212 can perform various types of processing on data read from the RAM 1214, including various types of operations specified by a sequence of program commands as described anywhere in this invention, such as information processing, conditional judgments, conditional branches, unconditional branches, information search / replacement, etc., and write the results back to the RAM 1214. Furthermore, the CPU 1212 can retrieve information from files, databases, etc., within the recording medium. For example, if multiple entries are stored in the recording medium, each entry having an attribute value of a first attribute associated with an attribute value of a second attribute, the CPU 1212 can retrieve from these multiple entries an entry that matches the condition specifying the attribute value of the first attribute, read the attribute value of the second attribute stored in that entry, thereby obtaining the attribute value of the second attribute associated with the first attribute satisfying a predetermined condition.

[0095] The programs or software modules described above can be stored on or near the computer 1200 in a computer-readable storage medium. Alternatively, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the program to the computer 1200 via the network.

[0096] In the flowcharts and block diagrams of this embodiment, boxes may represent stages of a process for performing an operation or "parts" of a device having the function of performing an operation. Specific stages and "parts" may be mounted via dedicated circuitry, programmable circuitry provided with computer-readable commands stored on a computer-readable storage medium, and / or a processor provided with computer-readable commands stored on a computer-readable storage medium. Dedicated circuitry may include digital and / or analog hardware circuitry and may include integrated circuits (ICs) and / or discrete circuitry. Programmable circuitry may, for example, include reconfigurable hardware circuitry such as field-programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), including logical products, logical sums, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and storage elements.

[0097] Computer-readable storage media can include any tangible device capable of storing commands that can be executed by a suitable device. As a result, a computer-readable storage medium having commands stored therein comprises commands that can be executed to create units for performing operations specified in a flowchart or block diagram. Examples of computer-readable storage media include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media may include floppy disks (registered trademark), magnetic disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random-access memory (SRAM), compactdisc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray discs (registered trademark), memory sticks, integrated circuit cards, etc.

[0098] Computer-readable commands may include assembly commands, instruction set architecture (ISA) commands, machine commands, machine-dependent commands, microcode, firmware commands, state setting data, or source code or object code described by any combination of one or more of the following programming languages, including object-oriented programming languages ​​such as Smalltalk (registered trademark), JAVA (registered trademark), C++, and existing procedural programming languages ​​such as the "C" programming language or similar programming languages.

[0099] To enable a processor or programmable circuit of a general-purpose computer, special-purpose computer, or other programmable data processing device to generate units for performing operations specified in a flowchart or block diagram, computer-readable commands are executed. These computer-readable commands can be provided locally or via a wide area network (WAN) such as a local area network (LAN) or the Internet to the processor or programmable circuit of the general-purpose computer, special-purpose computer, or other programmable data processing device. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0100] Although the present disclosure has been described above using embodiments, the technical scope of the present disclosure is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications or improvements can be made to the above embodiments. It is clear from the description of the technical solutions that methods with such modifications or improvements can also be included within the technical scope of the present disclosure.

[0101] It should be noted that the execution order of actions, processes, steps, and stages in the apparatus, systems, programs, and methods shown in the claims, description, and drawings can be implemented in any order, as long as it is not explicitly stated by "before," "before," etc., and the output of the previous process is not used in the subsequent process. Even if the flow of actions in the claims, description, and drawings is described using terms such as "firstly," "next," etc., for convenience, it does not mean that they must be implemented in that order.

[0102] Although the present disclosure has been described above using embodiments, the technical scope of the present disclosure is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications or improvements can be made to the above embodiments. It is clear from the description of the technical solutions that methods with such modifications or improvements can also be included within the technical scope of the present disclosure.

[0103] It should be noted that the execution order of actions, processes, steps, and stages in the apparatus, systems, programs, and methods shown in the claims, description, and drawings can be implemented in any order, as long as it is not explicitly stated by "before," "before," etc., and the output of the previous process is not used in the subsequent process. Even if the flow of actions in the claims, description, and drawings is described using terms such as "firstly," "next," etc., for convenience, it does not mean that they must be implemented in that order.

[0104] The entire contents of the disclosure of Japanese Patent Application 2023-209614, filed on December 12, 2023, are incorporated herein by reference.

[0105] Explanation of reference numerals in the attached figures 1 Humanoid robot, 2 Upper body, 3 Legs, 4 Connecting parts, 5 & 6 Arms, 7 & 8 Wheels, 10 Control system, 12 Sensors, 14 Information processing device, 20 Grip part, 22A, 22B, 22C, 22D, 22E Fingers, 26 Palm sensor, 51A Vision sensor, 51B Hearing sensor, 51C Olfactory sensor, 51D Tactile sensor, 51E Taste sensor, 1200 Computer, 1210 Main controller, 1212 CPU, 1214 RAM, 1216 Graphics controller, 1218 Display device, 1220 Input / output controller, 1222 Communication interface, 1224 Storage device, 1230 ROM, 1240 Input / output chip.

Claims

1. A control system of a robot, wherein, The robot's control system includes: A task actuator, mounted on a robot and used by the robot to perform tasks; and a sensor unit, which acquires various information required for the task actuator to perform its tasks. The robot's control system has the following features: The motion information creation unit compares various information from each of the sensor units with attribute information corresponding to the task, thereby creating motion control information required for the robot's motion control and feedback control. as well as The control unit controls the robot's operations based on the motion control information.

2. The control system for the robot according to claim 1, wherein, The robot's control system also has: A camera that captures images of the object being worked on to identify the type of the object; as well as A motion processing unit that determines the position of the object.

3. The control system for the robot according to claim 2, wherein, The information acquired by the sensor unit is related to vision, hearing, smell, touch and taste. Motion control information is obtained by combining multiple pieces of acquired information.

4. A control program of a robot, wherein, The robot's control program enables the computer to perform actions as the motion information creation unit and the control unit as described in any one of claims 1 to 3.

Images