Robot, robot control program, information processing device, learning data generation method, operation parameter generation method, program, and learning completion model generation method

Abstract

A humanoid robot (robot) includes: a grip section having a palm section serving as a base for holding a fabric (work object), and a plurality of finger sections extending radially from the palm section; a palm sensor unit that is provided to the palm unit and that detects work object information including the shape and arrangement of the work object; and a control unit that inputs the detection result of the palm sensor unit into a learning model that has been learned using the learning data, and that controls the holding and manipulation of the work object by the gripping unit on the basis of an operation parameter obtained by executing an arithmetic process of the learning model. The learning data represents a combination of a hand motion of a work skilled person during work and a motion of the sewing machine.

Description

Technical Field

[0001] This disclosure relates to robots, robot control programs, information processing devices, methods for generating learning data, methods for generating motion parameters, programs, and methods for generating learned models. Background Technology

[0002] Humanoid robots are used on factory production lines to automate tasks. The posture control of humanoid robots is described in Japanese Patent Application Publication No. 2019-093506.

[0003] Furthermore, WO2011 / 001569 discloses a robotic arm driven by an elastomer actuator and having multiple joints. The robotic arm is controlled by a tip support member and a control unit. The tip support member supports the robotic arm by contacting a support surface disposed on the tip of the robotic arm. The control unit controls the position and posture of the tip of the robotic arm while controlling the contact force between the tip support member and the support surface. Summary of the Invention

[0004] The problem that the invention aims to solve However, tasks such as sewing that require delicate hand movements were traditionally performed by skilled workers.

[0005] Furthermore, even if the robot's gripping mechanism is designed in a finger-like shape, it is still difficult to reproduce the movements of a skilled person's fingers and arms.

[0006] The purpose of this disclosure is to provide a robot capable of operating a workpiece like a skilled operator, a robot control program, an information processing device, a method for generating learning data, a method for generating motion parameters, a program, and a method for generating a learned model.

[0007] Methods for solving problems The robot involved in the first aspect comprises: a gripping part having a palm serving as a base for holding a workpiece and a plurality of fingers extending radially from the palm; a palm sensor part disposed on the palm and detecting workpiece information including the shape and configuration of the workpiece; and a control part that inputs the detection results of the palm sensor part into a learning model completed by learning using learning data, and controls the gripping part to hold and operate the workpiece based on motion parameters obtained by performing computational processing of the learning model, wherein the gripping part is used for gripping in the case of working with the machine tool, and the learning data represents a combination of hand movements of a skilled operator and machine tool movements during operation.

[0008] In this robot, the control unit inputs the detection results from the hand sensor into a learning model that has been trained using learning data. Based on the motion parameters obtained by processing the learning model, the control unit controls the holding and manipulation of the workpiece by the gripping unit when it operates the machine tool. This learning data represents the combination of hand movements and machine tool movements performed by a skilled operator. Therefore, the robot can manipulate the workpiece through the gripping unit like a skilled operator.

[0009] Secondly, based on the robot involved in the first aspect, it has two gripping parts corresponding to the right hand and the left hand; the fingers of one gripping part are five; the workpiece is cloth; the machine tool is a sewing machine; and the operation is sewing.

[0010] This robot, through two gripping parts each with five fingers, can use a sewing machine to sew fabric like a skilled person.

[0011] Thirdly, based on the robot involved in the first aspect, the learning model is a model that has been relearned by utilizing the difference between the target task result and the actual task result performed by the robot.

[0012] In this robot, the learning model can improve the accuracy of the task by relearning from the difference between the target task result and the actual task result performed by the robot.

[0013] Fourthly, based on the robot involved in the first aspect, the parameters provided to the learning model are determined according to the work object and the work method.

[0014] In this robot, since the parameters provided to the learning model are determined according to the workpiece and the work method, the workpiece can be operated more accurately through the gripping part.

[0015] Fifthly, based on the robot involved in the first aspect, the workpiece has an identifier indicating a specific position as a specific location; the control unit controls the holding unit to hold and operate the workpiece based on the detected identifier.

[0016] Sixthly, based on the robot involved in the fifth aspect, the control unit inputs the detection result into a learning model that has been completed after relearning, the learning model being a learning model in a device that outputs the difference between the position of the identifier in the target task result and the position of the identifier in the actual task result.

[0017] Seventhly, based on the robot involved in the fifth aspect, the identifier is an identification code capable of reading character information; the character information includes at least one of the identification information of the work object, the work sequence, or the specific location.

[0018] Eighthly, based on the robot described in the fifth aspect, the control unit controls the holding unit to hold and operate the workpiece based on information from a shooting device that captures images of blind spots that cannot be detected by the hand sensor unit.

[0019] The control program for the robot involved in the ninth aspect enables the computer to operate as the control unit involved in any of the first to eighth aspects.

[0020] The information processing apparatus according to the tenth aspect includes: an acquisition unit that acquires learning data representing a combination of actions of a skilled worker during a task and actions of equipment used by the skilled worker; a specifying unit that specifies, from the acquired learning data, first learning data representing a specific task performed by the skilled worker using a specific piece of equipment, the specific piece of equipment being the specific piece of equipment; and a conversion unit that converts the specified first learning data into second learning data, the second learning data causing a robot equipped with one or more tools corresponding to the specific piece of equipment to perform actions corresponding to the specific task.

[0021] The information processing apparatus of the eleventh aspect, based on the information processing apparatus of the tenth aspect, wherein the conversion unit converts the first learning data based on a conversion table that stores the first learning data and the second learning data in correspondence.

[0022] The information processing apparatus according to the twelfth aspect, based on the information processing apparatus according to the eleventh aspect, wherein the learning data includes data of device tags that assign actions to the device, the device tags being tags that indicate the type of the device; and the conversion unit converts the learning data by changing the device tags of the first learning data into tool tags, the tool tags being tags that indicate the type of the tool corresponding to the device.

[0023] The information processing apparatus according to the thirteenth aspect comprises: a specifying unit that inputs detection results of a sensor unit of a robot into a learning model after learning data representing a combination of actions of a skilled worker during work and actions of equipment used by the skilled worker, and specifies a first action parameter from action parameters obtained by performing computational processing of the learning model, the first action parameter representing a specific task performed by the robot using a specific device, the specific device being a specific device and the specific task being a specific task; and a conversion unit that converts the specified first action parameter into a second action parameter, the second action parameter causing the robot, which is equipped with one or more tools corresponding to the specific device, to perform an action corresponding to the specific task.

[0024] The information processing apparatus according to the fourteenth aspect, based on the information processing apparatus according to the thirteenth aspect, wherein the conversion unit converts the first action parameter based on a conversion table that stores the first action parameter and the second action parameter in correspondence.

[0025] In the method for generating learning data according to the fifteenth aspect, a computer performs the following processing: acquiring learning data representing a combination of actions of a skilled worker during a task and actions of the equipment used by the skilled worker; identifying first learning data from the acquired learning data representing a specific task performed by the skilled worker using a specific device, the specific device being the specific device; and converting the identified first learning data into second learning data, the second learning data causing a robot equipped with one or more tools corresponding to the specific device to perform actions corresponding to the specific task.

[0026] In the method for generating motion parameters according to the sixteenth aspect, the computer performs the following processing: inputting detection results of the sensors of the robot into a learning model that has been trained using learning data representing a combination of actions of a skilled worker during a task and actions of the equipment used by the skilled worker; and identifying a first motion parameter from the motion parameters obtained by performing the computational processing of the learning model, the first motion parameter representing a specific task performed by the robot using a specific device, the specific device being a specific device and the specific task being a specific task; and converting the identified first motion parameter into a second motion parameter, the second motion parameter causing the robot, which has one or more tools corresponding to the specific device, to perform an action corresponding to the specific task.

[0027] The information processing apparatus according to the seventeenth aspect includes: an acquisition unit that acquires information about a workpiece; a switching unit that switches between multiple completed learning models based on the information about the workpiece acquired by the acquisition unit, the multiple completed learning models being learning models learned using learning data representing a combination of hand movements of a skilled worker and machine tool movements during work; and a control unit that inputs detection results from a sensor unit of a robot into the learning model switched by the switching unit, and controls the holding unit of the robot to hold and operate the workpiece based on motion parameters obtained by performing computational processing on the learning model.

[0028] The information processing apparatus according to the eighteenth aspect, based on the information processing apparatus according to the seventeenth aspect, wherein the switching unit determines a working mode from multiple working modes based on the information of the workpiece obtained by the obtaining unit; the control unit controls the robot based on the working mode determined by the switching unit; the multiple working modes include: a first working mode in which only one robot performs the work; and a second working mode in which multiple robots share the work.

[0029] The information processing apparatus involved in the nineteenth aspect, based on the information processing apparatus involved in the eighteenth aspect, wherein the plurality of learned learning models are learning models after learning for each type of workpiece; the first work mode is a cell production mode; and the second work mode is a production line mode.

[0030] The information processing apparatus according to the twentieth aspect includes: a learning unit that uses first learning data representing a combination of actions of a skilled worker during work and actions of the equipment used by the skilled worker to enable a learning model to learn; an acquisition unit that acquires a first work procedure of the robot from a simulation of a work performed by the robot using the equipment, executed using the learning completion model learned by the learning unit; and a modification unit that modifies the first work procedure into a second work procedure based on a predetermined reference, the second work procedure being a modified part of the first work procedure; wherein the learning unit uses second learning data representing a combination of actions of the robot and actions of the equipment obtained from the simulation of the robot's work based on the second work procedure to enable the learning completion model to relearn.

[0031] In the information processing apparatus according to the twenty-first aspect, based on the information processing apparatus according to the twenty-tenth aspect, the predetermined benchmark is a comparison result between the target result of the operation and the operation result when the simulation is performed by the execution unit.

[0032] In the information processing apparatus according to the twenty-second aspect, based on the information processing apparatus according to the twenty-first aspect, the second work step is a work step that removes at least one step from the first work step, which is divided into multiple steps.

[0033] In the information processing apparatus of aspect 23, based on the information processing apparatus of aspect 22, the robot has two gripping parts corresponding to the right hand and the left hand; the fingers of one of the gripping parts are five; the device is a sewing machine; and the operation is sewing.

[0034] In the information processing apparatus according to the twenty-fourth aspect, based on the information processing apparatus according to the twenty-first aspect, the second work step is a work step in which at least one step is added to the first work step.

[0035] The procedures involved in the twenty-fifth aspect enable the computer to operate as an information processing device involved in any of the tenth to fourteenth aspects, or the seventeenth to twenty-fourth aspects.

[0036] In the method for generating a learning completion model according to the twenty-sixth aspect, the computer performs the following: using first learning data representing a combination of actions of a skilled worker during a task and actions of the equipment used by the skilled worker, the learning model learns; obtaining a first task sequence of the robot from a simulation of a task performed by the robot using the equipment, executed using the learning completion model learned by the learning unit; modifying the first task sequence into a second task sequence based on a predetermined benchmark, the second task sequence being a modified part of the first task sequence; and relearning the learning completion model using second learning data representing a combination of actions of the robot and actions of the equipment obtained from the simulation of the robot's task based on the second task sequence.

[0037] It should be noted that the above summary does not list all the essential features of the present invention. Furthermore, sub-combinations of these feature groups may also constitute this disclosure.

[0038] Invention Effects According to this disclosure, a robot capable of operating a workpiece like a skilled operator through its gripping mechanism, a robot control program, an information processing device, a method for generating learning data, a method for generating motion parameters, a program, and a method for generating a learned model can be obtained. Attached Figure Description

[0039] Figure 1 This is a front view of the humanoid robot according to the first embodiment.

[0040] Figure 2 This is a perspective view of the gripping part according to the first embodiment.

[0041] Figure 3 This is a diagram schematically illustrating an example of the functional configuration of the humanoid robot according to the first embodiment.

[0042] Figure 4 This is a diagram schematically illustrating an example of a processing routine executed by an information processing device according to the first embodiment.

[0043] Figure 5 It is shown that... Figure 4 A flowchart illustrating the overall motion coordination of a humanoid robot, showing the control sequence when the robot manipulates the workpiece using its gripping mechanism.

[0044] Figure 6 This is a diagram schematically illustrating an example of computer hardware functioning as an information processing apparatus according to the first embodiment.

[0045] Figure 7 This is a perspective view showing the state in which two pieces of fabric are stacked on a sewing machine according to the first embodiment.

[0046] Figure 8 This is a perspective view showing the state in which the gripping part of the robot according to the first embodiment grips two pieces of cloth respectively.

[0047] Figure 9 This is a perspective view showing the state of the two pieces of fabric being manipulated by the holding part as the sewing progresses, according to the first embodiment.

[0048] Figure 10 This is a second perspective view showing an outline of the state in which two pieces of fabric are stacked on a sewing machine according to the second embodiment.

[0049] Figure 11 This is a schematic diagram illustrating an example of the functional configuration and peripheral devices of the humanoid robot involved in the second embodiment.

[0050] Figure 12 This is a flowchart illustrating the process of outputting the difference in position of the QR code according to the second embodiment.

[0051] Figure 13 This is a flowchart illustrating the relearning process by which the learning device relearns the learning model according to the second embodiment.

[0052] Figure 14This is a flowchart illustrating the process by which an information processing device acquires motion parameters according to the second embodiment.

[0053] Figure 15 This is a diagram schematically illustrating an example of the function of the control system and various devices involved in the third embodiment for controlling the humanoid robot.

[0054] Figure 16 This is a diagram showing an outline of the gripping part that can be exchanged with a robotic tool according to the third embodiment.

[0055] Figure 17 This is a diagram illustrating an example of a state in which the gripping part is replaced with a robotic tool, according to the third embodiment.

[0056] Figure 18 This is a front view of the gripping part according to the third embodiment. Figure 18 (B) is a diagram showing an example of an overview of the operation types of the gripping part involved in the second embodiment.

[0057] Figure 19 This is a diagram that schematically illustrates an example of the data structure of the conversion table involved in the third embodiment.

[0058] Figure 20 This is a flowchart illustrating the process of converting learning data, i.e., the learning data conversion process, involved in the third embodiment.

[0059] Figure 21 This is a diagram that schematically illustrates an example of the functional configuration of the humanoid robot according to the fourth embodiment.

[0060] Figure 22 This is a diagram that schematically illustrates an example of the data structure of the conversion table involved in the fourth embodiment.

[0061] Figure 23 This is a flowchart illustrating the process of motion parameter conversion processing according to the fourth embodiment.

[0062] Figure 24 This is a diagram that schematically illustrates an example of the functional configuration of the humanoid robot according to the fifth embodiment.

[0063] Figure 25 The fifth embodiment illustrates a learning model stored in a storage device after learning for different tasks.

[0064] Figure 26 This is a flowchart illustrating the process for determining the production method according to the fifth embodiment.

[0065] Figure 27 This is a flowchart illustrating the control process of a humanoid robot in a cell production mode, as described in the fifth embodiment.

[0066] Figure 28 This is a flowchart illustrating the control process of the humanoid robot 1 in a production line mode according to the fifth embodiment.

[0067] Figure 29 This is a diagram schematically illustrating an example of the functional configuration of the information processing apparatus according to the sixth embodiment.

[0068] Figure 30 This is a flowchart illustrating the relearning process of relearning the learning model according to the sixth embodiment.

[0069] Figure 31 This is a flowchart illustrating the process of process change handling that improves the efficiency of work processes according to the sixth embodiment.

[0070] Figure 32 This is a flowchart illustrating the process change handling involving the addition of a predetermined step in a variation of the sixth embodiment. Detailed Implementation

[0071] The present disclosure will now be described through embodiments thereof, but these embodiments do not limit the invention as defined in the claims. Furthermore, not all combinations of features described in the embodiments are necessary for the solution of the invention.

[0072] (First Implementation) Figure 1 This is a front view of a humanoid robot, which serves as an example of the robot involved in this embodiment. Figure 1 As shown, the humanoid robot 1 according to this embodiment has an upper body 2 and is positioned near a sewing machine in a sewing factory, etc., and manipulates fabrics 100 and 101, which are examples of workpieces, to perform sewing operations. The humanoid robot 1 has a gripping part 20, a palm sensor part 26, and a control part 142. Here, the sewing machine is an example of a machine tool.

[0073] The upper body 2 has two arms 5 and 6. Arms 5 and 6 are rotatably mounted on the left and right sides of the upper body 2. In addition, a gripping part 20 for holding objects is installed at the front end of arms 5 and 6 (see later description). It should be noted that the number of arms is not limited to two; there may be one or more arms.

[0074] Furthermore, the humanoid robot 1 involved in this embodiment is driven by a control system 10 installed inside the humanoid robot 1.

[0075] (Structure of the holding part 20) like Figure 2 As shown, the gripping part 20 installed at the front end of the arm parts 5 and 6 is formed into a structure similar to a human hand, and the gripping part 20 is rotatably installed on the arm parts 5 and 6 (Intelligent Hand System).

[0076] The gripping portion 20 has a palm portion 20A serving as a base for gripping the fabrics 100 and 101, and multiple finger portions 22A, 22B, 22C, 22D, and 22E extending radially from the palm portion 20A. Each finger portion 22A, 22B, 22C, 22D, and 22E has multiple joints. In this embodiment, the five-finger structure is formed identically (symmetrically) for both the right and left hands.

[0077] Furthermore, for example, a palm sensor unit 26 is mounted on the palm portion 20A of the gripping portion 20. The palm sensor unit 26 detects workpiece information, including the shape and arrangement of the fabrics 100 and 101. Specifically, the palm sensor unit 26 includes a high-resolution camera that identifies the type of fabrics 100 and 101, and a motion processing unit (MoPU) that identifies the position of the fabrics 100 and 101.

[0078] The high-resolution camera of the palm sensor unit 26 constituting this embodiment identifies the state of the captured fabrics 100 and 101 based on the captured image information.

[0079] In other words, a high-resolution camera has the function of obtaining information such as the shape, thickness, and position of a specific fabric 100, 101.

[0080] On the other hand, the MoPU, which together with the high-resolution camera constitutes the palm sensor unit 26 of this embodiment, outputs motion information representing the movement of the captured fabrics 100 and 101 (in this case, the relative movement between them and the arms 5 and 6) based on images of objects captured at a frame rate of 1000 frames per second or higher. It should be noted that when detecting moving fabrics 100 and 101, the frame rate can be increased, and when detecting stationary objects (non-moving fabrics 100 and 101), the frame rate can be decreased.

[0081] The MoPU outputs vector information representing the movement of points along predetermined coordinate axes to indicate the location of an object. That is, the motion information output from the MoPU does not include the information needed to identify the state of the captured fabrics 100 and 101, but only information representing the movement (direction of movement and speed) of the predetermined positions (e.g., edges) of the fabrics 100 and 101 along the coordinate axes (x-axis, y-axis, z-axis).

[0082] That is, it can accurately guide the trajectory of the holding part 20 when it approaches the fabric 100 and 101.

[0083] Information output from the palm sensor section 26, which includes a high-resolution camera and a MoPU, is provided to the information processing device 14.

[0084] The information processing device 14 can accurately identify the state of the fabrics 100 and 101 by using information from the palm sensor unit 26, which includes a high-resolution camera and a MoPU, and calculate the degree of opening of the fingers 22A, 22B, 22C, 22D and 22E when holding the fabric, as well as the strength of the grip, thereby precisely controlling the minute movements of the arms 5 and 6 and the gripping part 20 to cope with the sewing operation of the fabrics 100 and 101.

[0085] Figure 5 This is a schematic diagram of an example of the control system of the humanoid robot according to this embodiment. The control system 10 includes sensors 12 mounted on the humanoid robot, a hand sensor unit 26 including a high-resolution camera and a MoPU, and an information processing device 14.

[0086] Sensor 12 sequentially acquires information about the distance and angle between the fabric 100, 101 around the humanoid robot 1 and the arms 5, 6, which at least indicate the sewing operation performed by the humanoid robot 1, as well as the position of the needle 202 of the sewing machine 200. Figure 7 As sensor 12, at least one of the following can be used: a high-performance camera, a solid-state LiDAR (light detection and ranging), a multi-color laser coaxial displacement meter, or other various sensors. Furthermore, examples of sensors 12 include vibrometers, thermal cameras, hardness testers, radar, LiDAR, high-pixel / telephoto / ultra-wide-angle / 360-degree / high-performance cameras, visual recognition, weak sound, ultrasound, vibration, infrared, ultraviolet, electromagnetic waves, temperature, humidity, spot-based artificial intelligence (AI) weather forecasting, high-precision multi-channel Global Positioning System (GPS), low-altitude satellite information, or long-tail event AI data.

[0087] It should be noted that, in addition to the information mentioned above, sensor 12 also detects images, distance, vibration, heat, odor, color, sound, ultrasound, ultraviolet light, or infrared light. Furthermore, information detected by sensor 12 can include the movement of the humanoid robot 1's center of gravity, detection of the material of the floor on which the humanoid robot 1 is set, detection of external air temperature, detection of external air humidity, detection of the floor's vertical, horizontal, and diagonal tilt angles, and detection of moisture content.

[0088] Sensor 12 performs these detections every nanosecond.

[0089] The palm sensor unit 26 (high-resolution camera and MoPU) is a sensor provided on the gripping part 20 of the arm parts 5 and 6, and is separate from the sensor 12. It has a camera function for photographing the fabrics 100 and 101, and a position-specific function for the specific position of the fabrics 100 and 101.

[0090] It should be noted that using a single MoPU 12 can acquire vector information about the motion of a point representing the position of an object along each of the two coordinate axes (x-axis and y-axis) in a three-dimensional Cartesian coordinate system. Alternatively, utilizing the principle of a stereo camera, two MoPU 12s can be used to output vector information about the motion of a point representing the position of an object along each of the three coordinate axes (x-axis, y-axis, and z-axis) in a three-dimensional Cartesian coordinate system. For example, the y-axis is the axis along the direction of travel of the fabric 100 and 101 on the sewing machine 200. The z-axis is the axis along the thickness direction of the fabric 100 and 101 (the direction of reciprocating motion of the needle 202 of the sewing machine 200). The x-axis is the axis along directions orthogonal to both the y-axis and z-axis.

[0091] The information processing device 14 includes an information acquisition unit 140, a control unit 142, and an information storage unit 144.

[0092] The information acquisition unit 140 acquires information about the object detected by the sensor 12 and the palm sensor unit 26 (high-resolution camera and MoPU).

[0093] The control unit 142 uses information acquired by the information acquisition unit 140 from the sensor 12 and AI to control the rotation of the connecting unit 4, the vertical movement, and the movements of the arms 5 and 6.

[0094] Furthermore, the control unit 142 inputs the detection results from the palm sensor unit 26 into a learning model that has been trained using learning data representing a combination of hand movements of a skilled worker and movements of the sewing machine 200. Based on the motion parameters obtained by performing computational processing on the learning model, the control unit 142 controls the holding and operation of the fabrics 100 and 101 by the gripping unit 20. Specifically, the control unit 142 inputs the information (detection results) acquired by the information acquisition unit 140 from the palm sensor unit 26 (high-resolution camera and MoPU) at various times into the learning model. In addition, motion parameters output from the learning model include, for example, the rotation angles of the joints of the five fingers 22A, 22B, 22C, 22D, and 22E at various times, and the rotation angles of the joints of the arms 5 and 6 at various times. Based on these motion parameters, the control unit 142 controls the holding and operation of the fabrics 100 and 101 by the gripping unit 20. It should be noted that the parameters provided to the learning model can be determined based on the fabric type 100, 101 and the method of operation.

[0095] When sewing is being performed, the information processing device 14 repeatedly executes, for example... Figure 4 The flowchart shown.

[0096] In step S100, the information acquisition unit 140 acquires information about the fabrics 100 and 101 detected by the sensor 12.

[0097] In step S102, the control unit 142 uses the information of the fabrics 100 and 101 obtained in step S100 and AI to control the arms 5 and 6, thereby gripping the fabrics 100 and 101 by the gripping unit 20.

[0098] In step S104, the control unit 142 operates the fabrics 100 and 101 according to the sewing operation.

[0099] According to this embodiment, the control unit 142 inputs the detection results from the palm sensor unit 26 into a learning model that has been trained using learning data representing a combination of hand movements of a skilled worker and movements of the sewing machine 200. Based on the motion parameters obtained by performing computational processing on the learning model, the control unit 142 controls the holding and operation of the fabrics 100 and 101 by the gripping unit 20. Thus, during sewing operations, the fabrics 100 and 101 can be operated by the gripping unit 20 like a skilled worker.

[0100] It should be noted that the learning model can relearn by utilizing the difference between the target task result and the actual task result performed by robot 1. This allows the task accuracy to be gradually improved.

[0101] (Holding and control of fabric 100 and 101 (objects)) Figure 5It is shown that... Figure 4 A flowchart illustrating the sequence of motion coordination of the humanoid robot 1 and the gripping control sequence when the gripping part 20 grips an object.

[0102] In step 150, it is determined whether there is an indication of holding an object. If the indication is affirmative, the process proceeds to step 152, where arms 5 and 6 are moved to align palm 20A with the target object, and the process proceeds to step 154.

[0103] In step 154, the palm 20A faces the object, and the object's information is detected.

[0104] In the next step 156, the detection information obtained by the palm sensor unit 26 (high-resolution camera and MoPU) is analyzed to obtain a detailed understanding of the object's type (shape, size, hardness, etc.) and location, and then proceeds to step 160.

[0105] In step 160, the angles (opening degree) of the fingers 22A, 22B, 22C, 22D and 22E are set according to the state of the object, and then the process proceeds to step 162.

[0106] In step 162, the object is grasped.

[0107] In the next step 164, it is determined whether the object has been successfully grasped. If the determination is positive, the grasped object is transported to the predetermined position, and the process proceeds to step 150 to wait for the next instruction to grasp an object.

[0108] Furthermore, if the result is negative in step 164, proceed to step 166 and perform error handling (e.g., retry or cancellation), then return to step 150.

[0109] As described above, according to this embodiment, three fingers 22A, 22B, 22C, 22D and 22E are provided in the gripping part 20. By bending the palm part 20A and the fingers 22A, 22B, 22C, 22D and 22E of the gripping part 20, the fabrics 100 and 101 can be gripped.

[0110] A palm sensor unit 26 containing a high-resolution camera and a MoPU is installed on the palm part 20A. By installing the gripping part 20 with the above structure on the arms 5 and 6 of the humanoid robot 1, even if the humanoid robot 1 moves quickly, it can firmly grip the fabrics 100 and 101 through the gripping part 20.

[0111] Furthermore, since the palm sensor unit 26 (high-resolution camera and MoPU) is mounted on the palm part 20A, it can capture the fabrics 100 and 101 with high precision and can also handle operations involving minute movements. The gripping force of the holding part 20 can also be adjusted according to the stiffness of the fabrics 100 and 101.

[0112] (Operation control of fabrics 100 and 101) like Figure 7 As shown, this illustrates the operation of overlapping two pieces of fabric 100 and 101 below the needle 202 of a sewing machine 200, and then manipulating the fabric 100 and 101 with the handle 20 to align their edges and sew them together. The fabric 100 and 101 may have different edge contours, for example, for three-dimensional sewing. Furthermore, when the sewing machine 200 feeds the fabric, a stronger force is applied to the lower fabric 101, which may cause the feed rates of the upper and lower fabric 101 to differ. Therefore, if sewing is performed solely by the feeding action of the sewing machine 200, the edges will misalign, making it difficult to sew them neatly together.

[0113] In the humanoid robot 1 of this embodiment, a learning model is used that has been learned by combining learning data representing the hand movements of a skilled sewer and the movements of the sewing machine 200, thereby manipulating the fabrics 100 and 101 through the gripping part 20. Specifically, as Figure 8 As shown, fabrics 100 and 101 are held by two gripping parts 20 respectively. The sewing machine 200 is operated, and when sewing has progressed to a certain stage, the sewing machine 200 is stopped, or its operating speed is reduced. Here, as... Figure 9 As shown, for example, stretch fabric 100 to the right so that the edge of fabric 100 overlaps with the edge of fabric 101, while simultaneously stretching fabric 101 closer to the body to eliminate the difference in feed between fabric 100 and fabric 101. In this state, run sewing machine 200 again. By repeating this action, the edges of fabrics 100 and 101 can be sewn together like a skilled person.

[0114] It should be noted that the constant speed at which the humanoid robot 1 delivers the fabric 100 and 101 can be synchronized with one up-and-down stroke of the needle 200 of the sewing machine 200. This allows for smooth sewing.

[0115] Figure 6 An example of the hardware configuration of a computer 1200, which functions as an information processing device 14 including a control unit 142, is schematically shown. A program installed in the computer 1200 enables the computer 1200 to function as one or more "units" of the apparatus according to this embodiment, or to perform operations associated with the apparatus according to this embodiment or the one or more "units," and / or to perform processes according to this embodiment or stages of those processes. Such a program can be executed by the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the frames in the flowcharts and block diagrams described in this specification.

[0116] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216 interconnected via a host controller 1210. The computer 1200 also includes an input / output unit such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card driver, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive or a DVD-RAM drive, etc. The storage device 1224 may be a hard disk drive or a solid-state drive, etc. The computer 1200 also includes an input / output unit such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0117] The CPU 1212 operates according to the program stored in the ROM 1230 and RAM 1214, thereby controlling the various units. The graphics controller 1216 obtains image data generated by the CPU 1212 from the frame buffer or other data provided in the RAM 1214 or from itself, and displays the image data on the display device 1218.

[0118] Communication interface 1222 communicates with other electronic devices via a network. Storage device 1224 stores programs and data used by CPU 1212 within computer 1200. DVD drive reads programs or data from DVD-ROM, etc., and provides them to storage device 1224. IC card driver reads programs and data from IC card, and / or writes programs and data to IC card.

[0119] The ROM 1230 stores boot programs and other programs that are executed by the computer 1200 at startup, and / or programs that depend on the hardware of the computer 1200. The input / output chip 1240 can also connect various input / output units to the input / output controller 1220 via USB ports, parallel ports, serial ports, keyboard ports, mouse ports, etc.

[0120] The program is provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The program is read from the computer-readable storage medium, installed in a storage device 1224, RAM 1214, or ROM 1230 (also examples of computer-readable storage media), and executed by the CPU 1212. The information processing described within these programs is read by the computer 1200, enabling cooperation between the program and the aforementioned hardware resources of various types. The apparatus or method can be configured to perform information manipulation or processing according to the use of the computer 1200.

[0121] For example, when communication is performed between computer 1200 and external devices, CPU 1212 can execute a communication program loaded into RAM 1214 and, based on the processing described in the communication program, command communication interface 1222 to perform communication processing. Under the control of CPU 1212, communication interface 1222 reads transmission data stored in a transmission buffer provided in RAM 1214, storage device 1224, recording medium such as DVD-ROM or IC card, and sends the read transmission data to the network, or writes received data received from the network into a receive buffer provided on the recording medium, etc.

[0122] Furthermore, the CPU 1212 can read all or necessary portions of files or databases stored in external recording media such as storage device 1224, DVD drive (DVD-ROM), IC card, etc., into RAM 1214, and perform various types of processing on the data in RAM 1214. Next, the CPU 1212 can write the processed data back to the external recording medium.

[0123] Various types of information, such as programs, data, tables, and databases, can be stored in the recording medium for information processing. The CPU 1212 can perform various types of processing on data read from RAM 1214 and write the results back to RAM 1214. These various types of processing include operations, information processing, conditional judgments, conditional branching, unconditional branching, information retrieval / replacement, etc., specified by a sequence of program instructions and described throughout this disclosure. Furthermore, the CPU 1212 can retrieve information from files, databases, etc., within the recording medium. For example, if the recording medium stores multiple entries, each with an attribute value of a first attribute associated with a second attribute value, the CPU 1212 can retrieve from these multiple entries an entry that matches a condition specifying the attribute value of the first attribute, and read the attribute value of the second attribute stored in that entry, thereby obtaining the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0124] The programs or software modules described above can be stored on or near the computer 1200 in a computer-readable storage medium. Alternatively, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the program to the computer 1200 via the network.

[0125] In this embodiment, the blocks in the flowcharts and block diagrams may represent stages of a process for performing an operation or "parts" of a device that performs the operation. Specific stages and "parts" may be implemented by dedicated circuitry, programmable circuitry supplied together with computer-readable instructions stored on a computer-readable storage medium, and / or a processor supplied together with computer-readable instructions stored on a computer-readable storage medium. Dedicated circuitry may include digital and / or analog hardware circuitry, and may also include integrated circuits (ICs) and / or discrete circuitry. Programmable circuitry may include reconfigurable hardware circuitry such as field-programmable gate arrays (FPGAs) and field-programmable gate arrays (PLAs), which include logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, flip-flops, registers, and storage elements.

[0126] Computer-readable storage media can include any tangible device capable of storing instructions executable by a suitable device. As a result, a computer-readable storage medium having instructions stored in a tangible device comprises an article including the instructions, which can be executed to generate units for performing operations specified in a flowchart or block diagram. Examples of computer-readable storage media include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media may include floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random-access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray disc, memory sticks, integrated circuit cards, etc.

[0127] Computer-readable instructions may include any of the following: assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code described in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk (registered trademark), JAVA (registered trademark), C++, and traditional procedural programming languages ​​such as the "C" programming language or similar programming languages.

[0128] Computer-readable instructions may be provided locally or via a wide area network (WAN) such as a local area network (LAN), the Internet, or a processor or programmable circuit of a general-purpose computer, special-purpose computer, or other programmable data processing device, causing the processor or programmable circuit of the general-purpose computer, special-purpose computer, or other programmable data processing device to execute the computer-readable instructions to generate units for performing the operations specified in the flowchart or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0129] (Second Implementation) Next, the second embodiment will be described (omitting or simplifying the parts that are repeated with the above embodiment). The second embodiment is characterized in that the operation performed by the humanoid robot 1 is an operation using a workpiece with a QR code as an identification code. In the second embodiment, as an example, the operation of the humanoid robot 1 sewing fabrics 100 and 101 using a sewing machine 200 will be described.

[0130] like Figure 11 As shown, the control system of the humanoid robot 1 according to the second embodiment includes the humanoid robot 1, a camera 300, and a learning device 400. The humanoid robot 1 and the camera 300, as well as the humanoid robot 1 and the learning device 400, are configured to communicate with each other. It should be noted that in this embodiment, only one humanoid robot 1 is illustrated for one learning device 400, but it is not limited to this; multiple humanoid robots 1 can be connected to one learning device 400.

[0131] Figure 10 This is a second perspective view showing an outline of the state in which two pieces of fabric are stacked on a sewing machine according to the second embodiment.

[0132] like Figure 10As shown, in this embodiment, QR code 110 is printed on fabric 100, and QR code 111 is printed on fabric 101. Furthermore, the camera 300 is separately positioned from the humanoid robot 1 and the sewing machine 200.

[0133] QR codes 110 and 111 are printed at specific locations on the workpiece. These specific locations are those that can be detected by the sensor 12, the hand sensor 26, or the camera 300 during the operation of the humanoid robot 1. In this embodiment, as an example, QR code 110 is printed near the edge of the back side of the fabric 100, and QR code 111 is printed near the edge of the front side of the fabric 101. Here, QR codes 110 and 111 are printed as a corresponding set of QR codes.

[0134] QR codes 110 and 111 are configured to read character information. The character information includes at least one of the following: the type of workpiece, the sequence of operations, or the location information of the workpiece. As an example, in this embodiment, the information represents the type of fabric component, the sewing sequence, and the coordinate position of the component. The QR code may be, for example, a barcode, a QR code (registered trademark), etc. It should be noted that the sequence of operations can be the sequence within an overall process, the sequence of operations within a single workpiece, or a combination of both.

[0135] As an example, QR code 110 contains the information "sleeve component, size 7, x: 10, y: 70, back". Here, "sleeve component" corresponds to "type of fabric component", "size 7" corresponds to "sewing sequence", and "x: 10, y: 70" corresponds to "coordinate position of the component". As another example, QR code 111 contains the information "sleeve component, size 7, x: -10, y: 70, front".

[0136] The imaging device 300 captures images of the humanoid robot 1 performing its tasks and the workpiece. In this embodiment, the imaging device 300 captures images from below the sewing machine table on which the sewing machine 200, on which fabrics 100 and 101 are placed. The imaging device 300 in this embodiment captures areas of the humanoid robot 1 that cannot be detected by the palm sensor unit 26. The image captured from below the sewing machine table is an example of a blind spot, showing the bottom surface of the fabrics 100 and 101 opposite the top of the sewing machine table. The sewing machine table in this embodiment can be transparent or mesh, allowing the imaging device 300 to capture images from below. It should be noted that the imaging device 300, like the palm sensor unit 26, preferably has a high-resolution camera and a MoPU.

[0137] Figure 11 This is a schematic diagram illustrating an example of the functional configuration and peripheral devices of the humanoid robot involved in the second embodiment.

[0138] like Figure 11 As shown, the information processing device 14 in the second embodiment is the same as that in the first embodiment, and includes an information acquisition unit 140, a control unit 142, and an information storage unit 144 as its functional configuration. Hereinafter, the unique operations of the information acquisition unit 140 and the control unit 142 in the second embodiment, as well as the learning device 400 as an example of a peripheral device, will be described.

[0139] The information acquisition unit 140 has the function of acquiring information about the object detected by the sensor 12, the palm sensor unit 26, and the imaging device 300. The object information, that is, the fabric information in this embodiment, includes the printing position and tilt of the QR codes 110 and 111, as well as character information that can be read from the QR codes 110 and 111.

[0140] The control unit 142 has the function of controlling the gripping unit 20 to hold and operate the fabrics 100 and 101 based on the fabric information acquired by the information acquisition unit 140. In this embodiment, the control unit inputs the shape and arrangement of the fabrics 100 and 101 detected by the palm sensor unit 26 to the learning model, and also inputs the information of the fabrics 100 and 101 detected by the palm sensor unit 26 and the imaging device 300, and uses the output motion parameters to control the gripping unit 20. Thus, the information processing device 14 controls the holding and operation of the fabrics 100 and 101.

[0141] The learning device 400 has the function of learning the learning model used in the control unit 142. Furthermore, the learning device 400 of this embodiment is a device for relearning the learning model after it has been learned. The learning device 400 obtains motion parameters output from one or more information processing devices 14 during operation, as well as difference data representing the difference between the target operation result and the actual operation result, and relearns the learning model already used in the control unit 142. It should be noted that the difference data in this embodiment will be explained in the difference data output processing described later.

[0142] The computer that functions as the learning device 400 in this embodiment and Figure 6 The configuration is the same as that of the computer 1200. Then, the learning device 400 uses the program installed in the learning device 400 to relearn the learning model that has completed learning. It should be noted that the relearning process will be explained in the relearning process described later.

[0143] (effect) The information processing device 14 in the second embodiment is the same as that in the above embodiment. As an example, it repeatedly performs sewing operations. Figure 4 The flowchart shown is provided below. Regarding the sewing operation performed by the information processing device 14, the following will use… Figure 4 The specific actions in the second embodiment will be explained.

[0144] exist Figure 4 In step S100, the information acquisition unit 140 acquires information about the fabrics 100 and 101 detected by the sensor 12, the palm sensor unit 26, and the shooting device 300.

[0145] In step S102, the control unit 142 uses the information about the fabrics 100 and 101 obtained in step S100 and the AI ​​to control the arms 5 and 6, thereby gripping the fabrics 100 and 101 by the holding unit 20. The AI ​​in this embodiment is an AI that uses a learning model that has been learned through relearning processing described later.

[0146] In step S104, the control unit 142 operates the fabrics 100 and 101 according to the sewing operation based on the information acquired by the information acquisition unit 140. As an example, the control unit 142 sews the fabrics 100 and 101 in such a way that the position of the QR code 110 overlaps with the position of the QR code 111.

[0147] Figure 12 This is a flowchart illustrating the process of outputting difference data for the positional differences of the QR code according to the second embodiment. The difference data output processing is used to output data for relearning the learning model that has completed its learning process.

[0148] exist Figure 12 In step S201, the CPU 1212 acquires the position of the QR code on the sewn fabric. Specifically, the information acquisition unit 140 acquires fabric information to acquire the position of the QR code. In the second embodiment, as an example, the positions of the QR codes 110 and 111 after sewing fabrics 100 and 101 are acquired.

[0149] In step S202, the CPU 1212 acquires the positions of the target QR codes 110 and 111. In this embodiment, the positions of the target QR codes are predetermined positions set by the user. Furthermore, this position information may also be included in character information that can be read from the QR code.

[0150] In step S203, the CPU 1212 compares the two acquired positions and obtains difference data related to the difference between the compared positions. The difference data may be, for example, data representing the difference in distance and tilt between the acquired QR code and the target QR code. This data can be numerical or image information.

[0151] In step S204, the CPU 1212 outputs the acquired difference data. As an example, the CPU 1212 outputs the difference data to the learning device 400. Then, the CPU 1212 ends the difference data output processing.

[0152] Figure 13 This is a flowchart illustrating the relearning process by which the learning device 400 relearns the learning model, as described in the second embodiment.

[0153] exist Figure 13 In step S301, the learning device 400 acquires difference data. The difference data in this embodiment is... Figure 12 The difference data output in step S204.

[0154] exist Figure 13 In step S302, the learning device 400 uses the acquired difference data to relearn the learning model. The method for relearning the learning model is, for example, reinforcement learning that sets a reward based on the difference represented by the acquired difference data. As an example, the reward is set such that the smaller the difference, the higher the reward.

[0155] In step S303, the learning device 400 outputs the relearned learning model. The learning device 400 then outputs this learning model to the information processing device 14, which acquired the difference data in step S301. Finally, the learning device 400 ends the relearning process.

[0156] Figure 14 This is a flowchart illustrating the process by which an information processing device acquires update processing of motion parameters according to the second embodiment.

[0157] exist Figure 14 In step S401, CPU 1212 determines whether a relearned learning model has been obtained. When CPU 1212 determines that a relearned learning model has been obtained (step S401: Yes), it proceeds to step S402. On the other hand, when CPU 1212 determines that a relearned learning model has not been obtained (step S401: No), CPU 1212 ends the update process.

[0158] In step S402, CPU 1212 updates the learning model to the acquired, fully learned learning model. In this embodiment, the acquired, fully learned learning model is... Figure 13 The learned model is output in step S303. Then, CPU1212 ends the update process. It should be noted that after CPU1212 ends the update process, it can enter... Figure 4 The step S100 shown.

[0159] (Summary of the second implementation method) By printing QR codes 110 and 111 on specific locations on the workpiece, the information acquisition unit 140 can acquire the specific locations of the fabrics 100 and 101. Furthermore, the information processing device 14 can operate the gripping unit 20 based on these specific locations, thereby improving the accuracy of operating the gripping unit 20.

[0160] By printing QR codes 110 and 111 near the edge of the fabric, compared to printing them outside the edge, it ensures that the time required for fabric 100 to cover QR code 111 is extended when sewing fabric 100 and 101. Furthermore, by printing QR codes 110 and 111 in a location invisible to the naked eye during wear, the impact of the printed QR codes on the garment design can be reduced.

[0161] The information processing device 14 outputs the positional difference of the QR code to the learning device 400. The learning device 400 uses the output difference data to relearn the learning model and outputs it to the information processing device 14. The information processing device 14 uses the motion parameters obtained by inputting the detection results of the palm sensor unit 26 into the relearned learning model, thereby improving the accuracy of controlling each part of the humanoid robot 1.

[0162] The learning device 400 uses motion parameters and difference data (hereinafter referred to as learning data) acquired from an information processing device 14 to relearn the learned model. This allows for the correction of individual differences in the sewing machine 200 and / or the humanoid robot 1, as well as errors that occur over time.

[0163] Furthermore, the learning device 400 can use learning data acquired from the information processing devices 14 of multiple identical humanoid robots 1 performing the same task to relearn the completed learning model. In this case, compared to acquiring learning data from a single information processing device 14, more learning data can be acquired over a certain period to relearn the learning model. Moreover, since the completed learning model can be relearned using a large amount of learning data, including individual differences between the sewing machine 200 and / or the humanoid robot 1, and noise caused by errors generated over time, the accuracy and robustness of the completed learning model can be efficiently improved.

[0164] When the detected information in QR codes 110 and 111 contains identification information of the work item, the information processing device 14 can determine whether the combination of the work items is correct or incorrect, thus preventing incorrect combination of the work items. As an example, this situation is when different pieces of fabric are sewn together.

[0165] Furthermore, when the detected information in QR codes 110 and 111 contains a work sequence, the information processing device 14 can reduce the occurrence of skipped work steps. In particular, even when the learning model is relearning, it can perform work according to the work sequence, thus preventing the work sequence from being changed or work steps from being skipped.

[0166] Furthermore, when the information in QR codes 110 and 111 contains the location information of the workpiece, the information processing device 14 can detect the deviation between this location information and the actual location information of the QR code during the work. Therefore, the information processing device 14 can improve the completion rate of the workpiece by correcting this deviation while performing the work.

[0167] The information processing device 14 can acquire image information that cannot be detected by the sensors 12 and 26 by acquiring image information of the workpiece captured by the imaging device 300 from the blind spot of the sensor 12 or the palm sensor unit 26. This image information includes, for example, the overlapping of fabrics 100 and 101 captured from below the sewing machine table, or information about QR codes that can only be detected from below the sewing machine table. By acquiring this image information, the humanoid robot 1 can perform its work more accurately compared to situations where this image information is not available.

[0168] [Remark] In the second embodiment, although a set of QR codes is printed on the fabrics 100 and 101 used as workpieces, the printed content is not limited to QR codes. The printed content may include, for example, marks (patterns) such as dots or crosses, or content with unreadable character information. Furthermore, the number of QR codes printed is not limited to one set; two or more sets may be printed. Additionally, the QR codes do not need to be printed in pairs; they may be printed individually. Moreover, the side with the QR code can be either the front or back of the fabrics 100 and 101, or it may be printed on both sides.

[0169] In the second embodiment, the QR code is printed near the edge of the fabric, but the location of the QR code is not limited to this. The QR code can be printed anywhere on the workpiece that is printable. As an example, the QR code is printed at characteristic locations such as where a label is attached, where the sewing angle changes sharply, or where stitches intersect.

[0170] In the second embodiment, the imaging device 300 is set separately from the humanoid robot 1 and the sewing machine 200. However, as long as the data captured by the imaging device 300 is obtained by the information processing device 14 of the humanoid robot 1, the imaging device 300 can also be integrated with the humanoid robot 1 or the sewing machine 200 (including the sewing machine table).

[0171] In the second embodiment, relearning of the learned model is performed in the learning device 400, but the execution of this relearning is not limited to this. This relearning can be performed by a system consisting of the information processing device 14 and multiple devices.

[0172] In the second embodiment, the learning device 400 has the same configuration as the computer 1200, and therefore has a configuration corresponding to the CPU 1212 and the graphics controller 1216. However, alternatively or additionally, it may have a processor suitable for generating or relearning learning models. This processor is, for example, an AI chip.

[0173] In the second embodiment, reinforcement learning is cited as an example of a relearning method for a learning model that utilizes the differences in output. This reinforcement learning includes deep reinforcement learning, which combines deep learning and reinforcement learning. Furthermore, the learning method for relearning the learning model is not limited to reinforcement learning; it can also be supervised learning, unsupervised learning, or deep learning without reinforcement learning, etc.

[0174] When learning data is acquired from multiple information processing devices 14 to enable the learning model to learn or relearn, reinforcement learning can be known learning methods such as "swarm reinforcement learning" (https: / / www.jstage.jst.go.jp / article / sicejl / 52 / 6 / 52_540 / _pdf) and "multi-agent reinforcement learning system that shares learning experience with a search agent" (https: / / www.jstage.jst.go.jp / article / kikaic1979 / 74 / 739 / 74_739_692 / _pdf). Furthermore, the relearning of the learning model can be performed offline reinforcement learning after learning data has been acquired and accumulated from multiple information processing devices 14.

[0175] The operation of the processor in the above embodiments can be performed not only by one processor, but also by multiple processors working together, or by multiple processors located in physically separate locations working together.

[0176] (Third implementation method) Next, the third embodiment will be described with omissions or simplifications of the parts that are repeated in the above embodiments. The third embodiment is characterized in that the actions performed by a skilled worker using specific sewing equipment (hereinafter referred to as specific actions) are converted into actions performed by the humanoid robot 1 using robotic tools. In the third embodiment, the conversion of learning data by the learning device 400, which is an information processing device, will be described.

[0177] Specifically, the sewing equipment used in this device includes irons, cutting scissors, tweezers, thread cutters, etc. Furthermore, the specific sewing equipment used in this particular device is one of the aforementioned sewing equipment that can be replaced by the robotic tools possessed by the humanoid robot 1.

[0178] The specific actions for a particular job are the actions performed by a skilled worker using the aforementioned specific sewing equipment.

[0179] (Overall composition) Figure 15 This diagram schematically illustrates an example of the control system and functions of each device involved in the third embodiment for controlling the humanoid robot. Figure 15 As shown, the humanoid robot 1 and the learning device 400 are connected and can communicate with each other.

[0180] First, the humanoid robot 1 according to the third embodiment will be described. For example... Figure 1 As shown, the humanoid robot 1 has a gripping part 20. Hereinafter, the configuration of the gripping part 20, which is unique to the third embodiment, will be described.

[0181] (Control Section 20) Figure 16 This is a diagram showing an outline of the gripping part 20 that can be exchanged with the robot tool 21EX.

[0182] like Figure 16 As shown, the gripping part 20 is the part forward of the wrist that is connected to the arm parts 5 and 6. The gripping part 20 can be attached to and detached from the wrist part and can be replaced with the robot tool 21EX described later.

[0183] (Robot Tools 21EX) In the first embodiment, the main purpose of the operation on the fabrics 100 and 101 is to hold the fabrics 100 and 101 with the holding part 20.

[0184] On the other hand, there are cases where the actions performed on fabrics 100 and 101 are not holding but other actions (such as cutting, ironing, etc.).

[0185] In this case, the holding part 20 can also hold the sewing equipment corresponding to various actions and make it face the fabrics 100 and 101. However, when continuing to perform the same actions, the burden of maintaining and controlling the holding state (such as the relative position control between the holding part and the sewing equipment being held) is relatively large.

[0186] Therefore, when the sewing equipment is a specific sewing equipment, the structure is configured such that the gripping part 20 is replaced with a robot tool 21EX, thereby performing the action of the humanoid robot 1 corresponding to a specific action.

[0187] Figure 17 (A) and Figure 17 (B) is a diagram showing an example of a state in which the gripping part 20 is replaced with a robot tool 21EX.

[0188] like Figure 17 As shown in (A), the action of the robot tool 21EXA is cutting, and as a tool, it is equipped with cutting scissors.

[0189] like Figure 17 As shown in (B), the robot tool 21EXB performs an ironing action, and as a tool, it is equipped with an iron.

[0190] It should be noted that, although the illustrations are omitted, sensor units 26 are also installed on each robot tool 21EX.

[0191] (Replacement of the tip of the finger) The tip of the finger of the gripping part 20, as shown in the robot tool 21EX above, can be replaced with a robot tool corresponding to a specific sewing device.

[0192] like Figure 18 As shown in (A), the gripping part 20 in the third embodiment is mainly for gripping the fabrics 100 and 101.

[0193] In this embodiment, the tips of each finger of the gripping part 20 can be attached and detached. Figure 18 (A) is equipped with a robot tool 50A for the aforementioned primary purpose. Figure 18 (B) shows the "finger tip".

[0194] On the other hand, such as Figure 18 As shown in (B), the tip of each finger can be replaced with a robot tool 50B~50F that has a function corresponding to a specific sewing device.

[0195] Robot Tool 50B is a “tweezer”, also known as a pincet, used when gripping or pressing small components such as brand labels.

[0196] Robot tool 50C is a "stick" used when pressing fabric with a rod-shaped component or applying paste to fabric.

[0197] Robot Tool 50D stands for "thread scissors," also known as wire cutters, used when cutting wires.

[0198] Robot Tool 50E is a "camera" used for photographing (especially close-ups) fabric.

[0199] Robot Tool 50F is a "rotary cutter," a so-called rotary cutter used when cutting fabric.

[0200] These robotic tools are particularly well-suited for delicate tasks.

[0201] (Learning Device 400) Next, the learning device 400 of the third embodiment will be described. The computer and... Figure 6 The configuration is the same as that of the computer 1200. Furthermore, the learning device 400 has the function of enabling the learning model used in the control unit 142 of the humanoid robot 1 to learn through a program installed in the learning device 400.

[0202] And, as Figure 15 As shown, the learning device 400 of this embodiment functions as an acquisition unit 402, a specific unit 404, and a conversion unit 406.

[0203] The acquisition unit 402 has the function of acquiring learning data.

[0204] The learning data in this embodiment represents a combination of the actions of a skilled worker during sewing and the actions of the sewing equipment used by that skilled worker. The learning data in this embodiment includes analytical data obtained by analyzing work images captured by image recognition. As an example, the acquisition unit 402 acquires data inferred from the work images, including the movements of the skilled worker's entire skeletal system, including their fingers, hands, and arms, as well as the movements of the sewing equipment.

[0205] In addition, the acquisition unit 402 acquires information about a label indicating the type of sewing equipment, inferred from the work video. Furthermore, besides the information obtained from the work video, the acquisition unit 402 can also acquire information output via motion capture.

[0206] The identification unit 404 has the function of identifying learning data representing a specific action from the acquired learning data as first learning data. In this embodiment, the identification unit 404 identifies data representing the actions of a skilled worker using a specific sewing device from the parsed data of the work image as learning data representing a specific action. As an example, the identification unit 404 identifies learning data representing a specific action when the information on the label of the sewing device used by the skilled worker indicates the type of specific sewing device.

[0207] The conversion unit 406 has the function of converting learning data. The conversion unit 406 converts specific learning data into learning data representing the action of the humanoid robot 1 corresponding to a specific action, which serves as second learning data. In this embodiment, the conversion unit 406 performs the conversion of learning data with reference to the conversion table 410 (described later) stored in the storage device of the learning device 400.

[0208] Figure 19 (A) and Figure 19 (B) is a diagram illustrating an example of a data structure in a conversion table 410 that stores learning data representing specific actions of a skilled worker and learning data representing the actions of the corresponding humanoid robot 1.

[0209] like Figure 19 (A) and Figure 19 As shown in (B), the conversion table 410 of this embodiment is prepared for each type of specific sewing equipment. The conversion table 410 includes, for example, a conversion table 410A for irons and a conversion table 410B for tweezers.

[0210] In each conversion table 410, a record is registered for each learning data point for a specific action that is the object of conversion. Each record is configured to include a field 412 for registering "label information" indicating the type of a specific sewing equipment, a field 414 for registering "learning data indicating a specific action", and a field 416 for registering "learning data indicating the action of humanoid robot 1".

[0211] In this embodiment, the "learning data representing the actions of the humanoid robot 1" stores learning data acquired for each action of the humanoid robot 1 corresponding to a specific action. As an example, this learning data is obtained from an image taken of the humanoid robot 1 replacing the gripper 20 with the robot tool 21EXB. This data is a combination of data inferred from the image representing the movement of the gripper 20 and the skeleton of the arms 5 and 6 of the humanoid robot 1, as well as the movement of the robot tool 21EXB.

[0212] like Figure 19 As shown in (A), in the first row of conversion table 410A, as an example, the label information for "ironing" is registered in field 412, the learning data of a skilled worker "picking up the iron" is registered in field 414, and the learning data for humanoid robot 1 to "replace the gripping part with a robot tool with an ironing function" is registered in field 416. This means that the learning data of a skilled worker picking up the iron is converted into learning data representing the corresponding action of humanoid robot 1.

[0213] Additionally, in the second row of conversion table 410A, as an example, the label information for "ironing" is registered in field 412, the learning data of skilled workers when "ironing" is registered in field 414, and the learning data for humanoid robot 1 to "irond with a robot tool with an iron function" is registered in field 416.

[0214] Furthermore, in the third row of conversion table 410A, as an example, the label information for "iron" is registered in field 412, the learning data of the skilled worker when "putting away the iron" is registered in field 414, and the learning data for the humanoid robot 1 to "replace the gripping part with the original hand" is registered in field 416.

[0215] like Figure 19 As shown in (B), in the first row of another conversion table 410B, as an example, the label information for "tweezers" is registered in field 412, the learning data of skilled workers when "using tweezers" is registered in field 414, and the learning data for humanoid robot 1 to "replace the fingers with robot tools with tweezers function" is registered in field 416.

[0216] Additionally, in the second row of conversion table 410B, as an example, the label information for "tweezers" is registered in field 412, the learning data of skilled workers "pressing with tweezers" is registered in field 414, and the learning data for humanoid robot 1 "pressing with a robot tool with tweezers function" is registered in field 416.

[0217] Furthermore, in the third row of conversion table 410B, as an example, the label information for "tweezers" is registered in field 412, the learning data of the skilled operator when "putting away the tweezers" is registered in field 414, and the learning data for the humanoid robot 1 to "replace the fingers with the original fingers" is registered in field 416.

[0218] Figure 15 The conversion unit 406 shown has the following function: when converting learning data with reference to the conversion table 410, it adjusts parameters so that the action of a specific sewing device becomes the action of the corresponding robot tool. As an example, when converting learning data representing the action of an experienced worker ironing, the conversion unit 406 of this embodiment converts the parameters of the coordinate data so that the coordinate data of the robot tool 21EXB used by the humanoid robot 1 is consistent with the coordinate data of the iron.

[0219] Furthermore, the conversion unit 406 has the function of changing the labels of the learning data. Specifically, the conversion unit 406 relabels the learning data that is marked as the action of a specific sewing device as the action of a robot tool. As an example, the conversion unit 406 converts the label of the learning data marked as "ironing iron" to the label of "robot tool 21EXB".

[0220] (effect) Figure 20 This is a flowchart illustrating the process of converting learning data, i.e., the learning data conversion process, involved in the third embodiment.

[0221] exist Figure 20 In step S1201, the learning device 400 acquires learning data including a label indicating the type of sewing equipment used. For example, the learning device 400 acquires learning data representing the combination of actions performed by a skilled worker using an iron and the actions of that iron. This learning data includes label information for "ironing."

[0222] In step S1202, the learning device 400 refers to the stored conversion table 410 to obtain a list of specific sewing equipment. For example, the learning device 400 obtains information such as irons and tweezers from the label information in the conversion table as the list of specific sewing equipment.

[0223] In step S1203, the learning device 400 determines whether there is learning data in the learning data that represents an action of using a specific sewing device. When the learning device 400 determines that the learning data exists (step S1203: Yes), it proceeds to step S1204. On the other hand, when the learning device 400 determines that the learning data does not exist (step S1203: No), it ends the learning data conversion process.

[0224] In step S1204, the learning device 400 refers to the conversion table 410 to convert the corresponding learning data. Specifically, the learning device 400 reads the conversion table 410 for each tag information from the storage device, and refers to the read conversion table 410 to convert the learning data representing a specific action into learning data representing the action of the humanoid robot 1.

[0225] In step S1205, the learning device 400 changes the label of the learning data for a specific sewing device to the label of the corresponding robot tool. Then, the learning device 400 ends the learning data conversion process.

[0226] (Summary of the third implementation method) The learning device 400 of this embodiment converts learning data representing specific actions into learning data representing actions of the humanoid robot 1 using robotic tools. Therefore, when the learning model is trained, the learning device 400 of this embodiment can use learning data representing actions of the humanoid robot 1 using robotic tools. In other words, by performing the conversion of the learning data in this embodiment, the learning device 400 can preprocess the learning data, enabling it to exhibit actions unique to the humanoid robot 1 that would not be performed by a skilled operator.

[0227] The learning device 400 of this embodiment uses a conversion table 410 to convert learning data of a specific action of a skilled worker into pre-acquired learning data representing the actions of the humanoid robot 1. Therefore, even if a specific action is completely different from the corresponding action of the humanoid robot 1, the learning device 400 of this embodiment can convert the learning data.

[0228] The learning device 400 of this embodiment converts learning data by changing labels representing specific sewing equipment to labels representing corresponding robot tools. Therefore, the learning device 400 of this embodiment can use learning data representing the actions of a specific sewing equipment as learning data representing the actions of the corresponding robot tool. That is, the learning device 400 of this embodiment can treat learning data representing the actions of a specific sewing equipment as learning data representing the actions of a robot tool, thereby enabling the learning model to learn.

[0229] (Fourth Implementation) Next, the fourth embodiment will be described with omissions or simplifications of parts that are repeated in the above embodiments. In the third embodiment, the learning data is converted, while in the fourth embodiment, the characteristic is that the action parameters output from the learned model after learning is completed are converted. In the fourth embodiment, the conversion of action parameters is performed by the information processing device 14, which is an information processing device.

[0230] (Overall composition) Figure 21 This is a diagram that schematically illustrates an example of the functional configuration of the humanoid robot according to the fourth embodiment.

[0231] like Figure 21 As shown, the information processing apparatus 14 in the fourth embodiment, in addition to the functional configuration of the embodiments described above, also includes a specification unit 146 and a conversion unit 148. The specification unit 146 and the conversion unit 148 will be described below.

[0232] The specific unit 146 has the following function: to specify motion parameters (hereinafter referred to as specific motion parameters) that represent the actions of the humanoid robot 1 using a specific sewing device, which are used as first motion parameters. In this embodiment, the specific unit 146 specifies specific motion parameters from the motion parameters output by the learned model. As an example, the specific unit 146 specifies the movement of the handle 20 and arms 5 and 6 for picking up the iron.

[0233] The conversion unit 148 has the function of converting motion parameters. The conversion unit 148 converts specific motion parameters into motion parameters (hereinafter referred to as converted motion parameters) that represent the actions of the humanoid robot 1 using robot tools, which are used as second motion parameters. In this embodiment, the conversion unit 148 performs the motion parameter conversion with reference to the conversion table 510 stored in the storage device 1224.

[0234] Figure 22 (A) and Figure 22 (B) is a diagram illustrating an example of a data structure for a transformation table 510 that stores specific action parameters in correspondence with transformation action parameters.

[0235] like Figure 22 (A) and Figure 22 As shown in (B), the conversion table 510 of this embodiment is prepared for each type of specific sewing equipment. For example, the conversion table 510 includes a conversion table 510A for irons and a conversion table 510B for tweezers.

[0236] In each conversion table 510, a record is registered for each specific motion parameter that is the object of conversion. Each record consists of a field 512 for registering "specific sewing equipment", a field 514 for registering "specific motion parameters", and a field 516 for registering "conversion motion parameters".

[0237] In this embodiment, the "conversion motion parameters" store the motion parameters of the humanoid robot 1 corresponding to specific motion parameters. As an example, the conversion motion parameters are motion parameters that indicate the action of the humanoid robot 1 replacing the gripping part 20 with the robot tool 21EXB.

[0238] like Figure 22 As shown in (A), in the first row of conversion table 510A, as an example, field 512 records information about an "ironing", field 514 records motion parameters for the humanoid robot 1 when it "takes up the iron", and field 516 records motion parameters for the humanoid robot 1 when it "replaces the gripping part with a robot tool that has an ironing function". This means that the motion parameters representing the humanoid robot 1 taking up the iron are converted into motion parameters representing the corresponding motion of the humanoid robot 1.

[0239] Additionally, in the second row of conversion table 510A, as an example, information about an "ironing iron" is registered in field 512, action parameters for the humanoid robot 1 when "ironing" are registered in field 514, and action parameters for the humanoid robot 1 to "irond with a robot tool that has an ironing function" are registered in field 516.

[0240] Furthermore, in the third row of conversion table 510A, as an example, information about "iron" is registered in field 512, action parameters for humanoid robot 1 to "put away the iron" are registered in field 514, and action parameters for humanoid robot 1 to "replace the gripping part with the original hand" are registered in field 516.

[0241] In addition, such as Figure 22 As shown in (B), in the first row of another conversion table 510B, as an example, the information of "tweezers" is registered in field 512, the action parameters of the humanoid robot 1 when "taking the tweezers" are registered in field 514, and the action parameters of the humanoid robot 1 when "replacing the fingers with a robot tool with tweezers function" are registered in field 516.

[0242] Additionally, in the second row of conversion table 510B, as an example, information about "tweezers" is registered in field 512, action parameters for humanoid robot 1 to "press with tweezers" are registered in field 514, and action parameters for humanoid robot 1 to "press with a robot tool that has tweezers function" are registered in field 516.

[0243] Furthermore, in the third row of conversion table 510B, as an example, information about "tweezers" is registered in field 512, action parameters for humanoid robot 1 to "put away the tweezers" are registered in field 514, and action parameters for humanoid robot 1 to "replace the fingers with the original fingers" are registered in field 516.

[0244] like Figure 21 The conversion unit 148 shown has the following function: when converting specific motion parameters into conversion motion parameters, it adjusts the parameters so that the motion of a specific sewing device becomes the motion of the corresponding robot tool. As an example, when converting motion parameters representing the ironing motion of the humanoid robot 1, the conversion unit 148 of this embodiment converts the motion parameters so that the coordinate data of the robot tool 21EXB used by the humanoid robot 1 is consistent with the coordinate data of the iron.

[0245] (effect) Figure 23 This is a flowchart illustrating the process of motion parameter conversion processing according to the fourth embodiment.

[0246] exist Figure 23 In step S1301, CPU 1212 refers to the stored conversion table 510 to obtain a list of specific sewing equipment. CPU 1212 obtains a list of specific sewing equipment such as irons and tweezers.

[0247] In step S1302, CPU 1212 determines whether an action parameter indicating the use of a specific sewing device has been identified from the output action parameters. If CPU 1212 determines that the action parameter has been identified (step S1302: Yes), the process proceeds to step S1303. If CPU 1212 determines that the action parameter has not been identified (step S1303: No), the action parameter conversion process ends.

[0248] In step S1303, CPU 1212 refers to conversion table 510 to convert the corresponding motion parameters. Specifically, CPU 1212 reads the conversion table 510 for each specific sewing device from storage device 1224, and refers to the read conversion table 510 to convert the specific motion parameters into conversion motion parameters. Then, CPU 1212 ends the motion parameter conversion process.

[0249] (Summary of the fourth implementation method) In the fourth embodiment, the information processing device 14 identifies motion parameters representing actions using a specific sewing device from the output motion parameters and converts these specific motion parameters into corresponding motion parameters for the humanoid robot 1. Therefore, the information processing device 14 in this embodiment does not need to perform the conversion of learning data as in the third embodiment, and can convert the specific actions of a skilled worker into the actions of the humanoid robot 1.

[0250] Furthermore, as long as the specific action parameters can be identified, the information processing device 14 can convert the action parameters output from any learning model. Therefore, even if the learned model is updated or changed after learning, the information processing device 14 of this embodiment can still convert the action parameters when the specific action parameters are identified.

[0251] The information processing device 14 of this embodiment uses a conversion table to convert specific motion parameters into corresponding motion parameters for the humanoid robot 1. Therefore, even if the motion of the humanoid robot 1 using a specific sewing device, as represented by the output motion parameters, is completely different from the motion of the humanoid robot 1 using the corresponding robot tool, the information processing device 14 of this embodiment can still convert the motion parameters.

[0252] Furthermore, the information processing device 14 uses a conversion table that corresponds the actions of the humanoid robot 1 using specific sewing equipment to the actions of the humanoid robot 1 using corresponding robotic tools to convert motion parameters. Therefore, the information processing device 14 can convert motion parameters so that the actions of the specific sewing equipment are consistent with the actions of the corresponding robotic tools. That is, the information processing device 14 can control the humanoid robot 1 so that even when using robotic tools, it can perform the same tasks as when using specific sewing equipment.

[0253] [Remark] In the third and fourth embodiments, "equipment" is referred to as "sewing equipment," but "equipment" is not limited to this. "Equipment" includes machinery, appliances, and tools, whether or not they are electric. Equipment is any equipment that can be operated by hand. Examples of equipment include cooking equipment used in cooking operations, manufacturing equipment used in the production of handicrafts and furniture, performance equipment used in stage performances, and playing equipment used in musical instrument performances.

[0254] In the third and fourth embodiments, the tools of the humanoid robot 1 can remain installed until the end of a series of operations, instead of being installed and removed each time it is used. As an example, when the humanoid robot 1 holds fabrics 100 and 101 using robot tools 50A installed on its thumb, index finger, and middle finger, and sews them using the sewing machine 200, the tips of its ring finger and little finger can also remain in a state where robot tools 50B or 50F are installed. By keeping the robot tools installed, the action of changing the robot tools of the humanoid robot 1 can be omitted.

[0255] In the third and fourth embodiments, although the robot tool is installed by replacing the front end of the gripping part 20 or the finger of the humanoid robot 1, the method of installing the robot tool is not limited to this. For example, the robot tool may be pre-built into the gripping part 20 and the finger, and exposed to the outside when needed.

[0256] In the third embodiment, as an example, inferred data of the entire skeleton of a skilled worker, including their fingers, hands, and arms, is used as learning data; however, the learning data is not limited to this. The learning data could also be data on the outline of a skilled worker.

[0257] In the third embodiment, the conversion of learning data is performed by the learning device 400, but is not limited thereto. The conversion of learning data can be performed by a system consisting of the information processing device 14 and multiple devices.

[0258] In the fourth embodiment, the conversion of motion parameters is performed by the information processing device 14, but it is not limited to this. The conversion of motion parameters can be performed by a system consisting of an external server and multiple devices.

[0259] In the third embodiment, the learning device 400 can convert learning data of a specific action into learning data of an action performed by the humanoid robot 1. As an example, the learning device 400 converts learning data obtained by a skilled operator when operating a sewing machine by manipulating a foot pedal into learning data of the humanoid robot 1 sending instruction information to the sewing machine.

[0260] In the fourth embodiment, the information processing device 14 can convert specific motion parameters into motion parameters for the actions performed by the humanoid robot 1. As an example, the information processing device 14 converts the motion parameters of the humanoid robot 1 operating the sewing machine by manipulating the foot pedal into motion parameters for the humanoid robot 1 sending instruction information to the sewing machine.

[0261] (Fifth Implementation) Next, the fifth embodiment will be described, omitting or simplifying the parts that are repeated in the above embodiments. The fifth embodiment is characterized by switching the learned learning model according to the type of workpiece. In the fifth embodiment, the case of the humanoid robot 1 performing a garment-sewing task will be described.

[0262] The workpiece includes raw materials and semi-finished products. Raw materials refer to the state before sewing, while semi-finished products refer to the state after a portion of them has been sewn. As an example, the workpiece in this embodiment is fabric 100 and 101 as raw materials, a workpiece in which one side of fabric 100 and 101 is sewn as a semi-finished product, and a sleeve component sewn from fabric 100 and 101. Furthermore, raw materials and semi-finished products may coexist in the workpiece. In this embodiment, the case where the workpiece is fabric 100 and 101 will be mainly described.

[0263] (Function) Figure 24 This is a diagram that schematically illustrates an example of the functional configuration of the humanoid robot according to the fifth embodiment.

[0264] like Figure 24 As shown, in addition to the functional configuration of the above-described embodiments, the information processing device 14 of this embodiment also functions as a switching unit 146. Hereinafter, the unique operations of the information acquisition unit 140 and the control unit 142, which function as acquisition units in the fifth embodiment, and the switching unit 146 will be described.

[0265] The information acquisition unit 140 has the function of acquiring fabric information, which is information about the workpiece. The fabric information includes information about the type of workpiece. The type of workpiece information includes the type of the final product and information about the type of parts produced from that workpiece. As an example, the information acquisition unit 140 of this embodiment acquires information about fabrics 100 and 101. The information about fabrics 100 and 101 includes, for example, information about a "shirt," which is a garment type of the final product, and information about "sleeves," which is a type of part sewn from fabrics 100 and 101.

[0266] The information acquisition unit 140 acquires fabric information based on information detected from the sensor 12 and the palm sensor unit 26. In this embodiment, the information acquisition unit 140 acquires fabric information based on the detected shape, state, etc., of the workpiece. As an example, the information acquisition unit 140 acquires information about the type of workpiece, such as "shirt" and "sleeve," stored corresponding to the shape of the fabrics 100 and 101. Furthermore, the information acquisition unit 140 can also acquire fabric information by reading an identifier such as a QR code attached to the workpiece.

[0267] The control unit 142 has the following functions: based on the learning model switched by the switching unit 146 (described later), it controls the holding unit 20 to hold and operate the workpiece. Furthermore, the control unit 142 has the function of controlling the humanoid robot 1 based on the working mode determined by the switching unit 146 (described later).

[0268] The switching unit 146 has the function of switching between multiple completed learning models. In this embodiment, the switching unit 146 switches between completed learning models based on the acquired fabric information, after learning for each type of workpiece. This embodiment's switching unit 146 changes the learning model used by the control unit 142 to one stored in... Figure 25 The storage device 1224 described herein can be used to learn any one of the learning models, thereby switching the learning model.

[0269] Figure 25 This illustrates the completed learning model stored in a storage device, after learning for each type of task. For example... Figure 25 As shown, the learning model is, for example, model 1224A for sleeves, model 1224B for the body of the garment, model 1224C for pockets, model 1224D for the collar, model 1224E for the skirt, and model 1224F for trousers.

[0270] Furthermore, the switching unit 146 has the function of determining the work mode. In this embodiment, the switching unit 146 determines the work mode based on the type of garment being the finished product. The work mode in this embodiment is either a unit production mode (as a first work mode) or a production line mode (as a second work mode). For example, if the garment is a shirt, the switching unit 146 determines it to be a unit production mode; if the garment is trousers, the switching unit 146 determines it to be a production line mode. Alternatively, as another example, if the garment is order-made, the switching unit 146 determines it to be a unit production mode; if the garment is ready-made (existing product), the switching unit 146 determines it to be a production line mode.

[0271] The unit production method in this embodiment is a work method in which only one humanoid robot 1 performs multiple tasks. On the other hand, the assembly line production method in this embodiment is a work method in which one humanoid robot 1 performs a task, and then another humanoid robot 1 takes over to perform the task, thereby performing multiple tasks.

[0272] (effect) Figure 26 This is a flowchart illustrating the process for determining the production method according to the fifth embodiment.

[0273] exist Figure 26 In step S2201, CPU 1212 detects the fabric to be sewn.

[0274] In step S2202, CPU 1212 obtains information about the type of garment from the fabric to be sewn.

[0275] In step S2203, CPU 1212 determines whether the type of clothing acquired is produced using a cell manufacturing method. If CPU 1212 determines that the type of clothing acquired is produced using a cell manufacturing method (step S2203: Yes), then proceed to step S2204. On the other hand, if CPU 1212 determines that the type of clothing acquired is not produced using a cell manufacturing method (step S2203: No), then proceed to step S2205.

[0276] In step S2204, CPU 1212 determines the production method as cell production mode. Then, CPU 1212 ends the determination process.

[0277] In step S2205, CPU 1212 determines the production method as assembly line production. Then, CPU 1212 ends the determination process.

[0278] Figure 27This is a flowchart illustrating the control process of the humanoid robot 1 in the case of cell production according to the fifth embodiment.

[0279] exist Figure 27 In step S2301, CPU 1212 removes the fabric. As an example, CPU 1212 controls humanoid robot 1 to remove fabric 100, 101 from a stocker located near humanoid robot 1.

[0280] In step S2302, CPU 1212 obtains the type of part sewn from the taken-out fabric. As an example, CPU 1212 obtains information about "sleeve" as the type of part sewn from fabrics 100 and 101.

[0281] In step S2303, CPU 1212 switches the learned model based on the type of the acquired component. As an example, CPU 1212 switches the learned model to "sleeve learned model".

[0282] In step S2304, CPU 1212 places the fabric onto sewing machine 200. As an example, in this embodiment, CPU 1212 places fabrics 100 and 101 onto sewing machine 200.

[0283] In step S2305, CPU 1212 executes... Figure 4 The sewing process shown.

[0284] exist Figure 27 In step S2306, CPU 1212 determines whether a series of jobs have been completed. If CPU 1212 determines that a series of jobs have not been completed (step S2306: No), then proceed to step S2307. On the other hand, if CPU 1212 determines that a series of jobs have been completed (step S2306: Yes), then the control processing of the cell production mode ends.

[0285] In step S2307, CPU 1212 removes a fabric different from the sewn fabric. In this embodiment, the fabric used for the "body" of the garment, which is another component, is removed. Then, CPU 1212 returns to step S2302 and repeats the process from step S2302 to step S2305. It should be noted that CPU 1212 may also not remove other fabrics used as raw materials, but instead remove components that are semi-finished products, and sew these components together in subsequent operations.

[0286] Figure 28 This is a flowchart illustrating the control process of the humanoid robot 1 in the case of a production line, according to the fifth embodiment. Figure 28The flowchart shown is a flowchart of multiple humanoid robots 1 working together. The process will be repeated as long as the work object is supplied.

[0287] exist Figure 28 In step S2401, the CPU 1212 receives the fabric. As an example, in this embodiment, the CPU 1212 receives the fabric by controlling the holding unit 20 to hold the fabric conveyed from the conveyor belt.

[0288] In step S2402, CPU 1212 obtains the type of part sewn from the received fabric.

[0289] Processing in steps S2403 to S2405 and Figure 27 The processes in steps S2303 to S2305 are the same, so their descriptions are omitted.

[0290] exist Figure 28 In step S2406, CPU 1212 transfers the sewn fabric. As an example, CPU 1212 holds the sewn fabric with the holding part 20 and places the fabric on the conveyor belt. Then, the CPU 1212 of the humanoid robot 1 ends the control processing of the assembly line production method. It should be noted that in the assembly line production method, the humanoid robot 1 located downstream of the conveyor belt performs the control processing related to the next process.

[0291] (Summary of the fifth implementation method) In the fifth embodiment, the information processing device 14 switches to the learned model based on the acquired fabric information, and controls the humanoid robot 1 based on the switched learning model. Therefore, the information processing device 14 can continuously process different types of workpieces. That is, even if different types of workpieces are transported on the same production line, the information processing device 14 can enable the humanoid robot 1 to perform work suitable for each workpiece.

[0292] Furthermore, by switching the learned model through the information processing device 14, the humanoid robot 1 can be responsible for any sewing process on the production line. In other words, even if a humanoid robot 1 malfunctions midway through the production line, the next humanoid robot 1 can take its place and perform the subsequent work.

[0293] In the fifth embodiment, the information processing device 14 determines the production method based on the type of clothing. That is, the information processing device 14 can apply one humanoid robot 1 to different production methods. Specifically, even when multiple humanoid robots 1 capable of assembly line operations are configured, each humanoid robot 1 can switch production methods according to the type of clothing being transported on the assembly line. For example, the information processing device 14 can control the humanoid robot 1 to switch the production method to a unit production method when the type of clothing being transported on the assembly line is small-batch clothing, or to a mass production line production method when the type of clothing being transported on the assembly line is large-batch clothing.

[0294] [Remark] In the fifth embodiment, the switching unit 146 determines the production method based on the type of clothing, but the determined production method is not limited to one method. For example, the switching unit 146 can also control the humanoid robot 1 to switch from cell production to assembly line production after starting sewing in a cell production mode. As an example, the switching unit 146 can produce in a cell production mode before completing a predetermined semi-finished product, and then switch to assembly line production.

[0295] In the fifth embodiment, multiple completed learning models are set as learning models for each component, but the learning models are not limited to this. For example, a completed learning model can be created for each task and switched. Specifically, if there are two or more parts in a component that are sewn together using a sewing machine, a completed learning model can be created for each part and switched.

[0296] In the fifth embodiment, a sewing operation using a sewing machine 200 was described, but the sewing method of the humanoid robot 1 is not limited to the sewing method using a sewing machine 200. For example, the sewing method could also be a method of using the holding part 20 to hold the sewing needle and sew the fabrics 100, 101, that is, a so-called hand sewing method.

[0297] In the fifth embodiment, the method for determining the production method can also be determined using the method described below.

[0298] As an example, the switching unit 146 obtains the type and quantity of fabric in the storage container, and determines that the cell production method can be used if it is determined that the cell production method can be used.

[0299] Furthermore, as another example, the switching unit 146 determines the cell production mode when the production quantity of the workpiece is less than the predetermined quantity, and determines the assembly line production mode when the production quantity is more than the predetermined quantity.

[0300] In addition, as another example, the switching unit 146 determines the cell production mode when the number of processes of the workpiece is less than the predetermined quantity, and determines the assembly line production mode when the number of processes is more than the predetermined quantity.

[0301] (Sixth Implementation Method) Next, the sixth embodiment will be described with omissions or simplifications of the parts that are repeated in the above embodiments. The sixth embodiment is characterized in that the information processing device 34 uses learning data obtained from simulation to relearn the learned model. In the sixth embodiment, as an example, the learning model of the humanoid robot 1 when using the sewing machine 200 to sew fabric 100 and fabric 101 will be described.

[0302] The computer that functions as the information processing device 34 in the sixth embodiment is... Figure 6 The computer 1200 has the same configuration. Furthermore, as described later... Figure 29 As shown, the information processing device 34 is configured to communicate with the control system 10 of the humanoid robot 1. In this embodiment, the information processing device 14 and the information processing device 34 of the control system 10 communicate via a network, so that the learning model completed by the information processing device 34 after relearning is provided to the control system 10 of the humanoid robot 1.

[0303] Figure 29 This diagram schematically illustrates an example of the functional configuration of the information processing apparatus 34 according to the sixth embodiment. The information processing apparatus 34 in this embodiment includes an execution unit 340, a process acquisition unit 342 (as an acquisition unit), a process modification unit 344 (as a modification unit), and a learning unit 346.

[0304] The execution unit 340 has the function of simulating a task performed by a robot using equipment. Here, "equipment" includes machinery, appliances, and tools, regardless of whether they are electric or not. Any equipment that can be operated by hand is acceptable. Examples of equipment include sewing equipment used in sewing operations, cooking equipment used in cooking operations, manufacturing equipment used in the production of handicrafts and furniture, performance equipment used in stage performances, and playing equipment used in musical instrument performances. In this embodiment, the equipment is a sewing machine 200. The execution unit 340 performs the simulation using a learned model. Furthermore, the execution unit 340 performs the simulation based on the work procedures described later. Then, the execution unit 340 outputs the simulation results generated by the simulation. In this embodiment, the execution unit 340 simulates a task performed by a humanoid robot 1 using a sewing machine 200.

[0305] The simulation is performed in a simulation environment with predetermined physical parameters. As an example, physical parameters include physical parameters such as the mass and friction of the workpiece. In addition, physical parameters may also include numerical values ​​representing the state of the workpiece. The state of the workpiece includes, for example, its type (shape, size, hardness, weight, etc.) and location. It should be noted that physical parameters can be set randomly.

[0306] A work process is a work process in which a robot performs work using equipment. In this embodiment, the work process is the work process in which the humanoid robot 1 uses a sewing machine 200 to sew fabric 100 and fabric 101.

[0307] The simulation results are virtual data obtained from the simulation. The simulation results include not only the data that can be obtained after the task is completed, but also the data that can be obtained during the task. The simulation results in this embodiment include, for example, correction data representing the combination of the humanoid robot 1's movements and the sewing machine 200's movements as second learning data, data on the deliverables after sewing fabric 100 and fabric 101, motion parameters of the gripping part 20, 3D coordinate data, detection results from sensor 12, and detection results from palm sensor 26, etc.

[0308] The process acquisition unit 342, as an acquisition unit, has the function of acquiring the robot's work processes. In this embodiment, the process acquisition unit 342 acquires the work processes of the humanoid robot 1 through simulation executed by the execution unit 340. In addition, the process acquisition unit 342 can acquire the work processes of the humanoid robot 1 during actual operation.

[0309] The process change unit 344, which is a change department, has the function of dividing a work process into multiple processes. Specifically, the work process is divided by dividing the work process into predetermined time intervals.

[0310] Furthermore, the process modification unit 344 has the function of modifying a portion of the work process. The process modification unit 344 modifies the work process based on a comparison result between the target data (described later) and the deliverable data of the simulated work result of the work process that has been modified as part of the process. In this embodiment, the process modification unit 344 modifies the work process when the difference between the target data and the deliverable data of the simulated work process after a portion of the process has been deleted is less than a predetermined threshold.

[0311] The target data is either arbitrarily set deliverable data or deliverable data obtained from a simulation based on the work process before the change. In this embodiment, it is the data of the finished product after sewing fabric 100 and fabric 101.

[0312] The data used for comparison can be any data that can be obtained from the simulation results, such as numerical data (size, hardness, weight, density, etc.), image data, and sound data. In this embodiment, as an example, numerical data representing the gap between the fabrics when fabric 100 and fabric 101 are sewn together are compared.

[0313] Changes to the process include supplementing the actions before and after the process that is being changed. A connecting action refers to, for example, when the action of moving the gripping part 20 from its current position to the right and then upward is deleted, the action of moving the gripping part 20 diagonally upward to the right from its current position.

[0314] The learning unit 346 has the following function: it uses learning data obtained from the skilled worker's actions during the job as basic data, which serves as the first learning data, to enable the learning model to learn. Specifically, the learning unit 346 uses learning data representing the combination of the skilled worker's actions during the job and the actions of the equipment used by the skilled worker to enable the learning model to learn.

[0315] Furthermore, the learning unit 346 has the function of relearning the learning model after it has completed learning by using correction data obtained from the simulation. Specifically, the learning unit 346 uses learning data obtained from the simulation, which represents the combination of the robot's actions and the actions of the equipment used by the robot, to relearn the learning model after it has completed learning. In addition, the learning unit 346 can use multiple learning data obtained from simulations with different physical parameters as a dataset for relearning to relearn the learning model after it has completed learning.

[0316] Figure 30 This is a flowchart illustrating the relearning process of relearning the learning model according to the sixth embodiment.

[0317] exist Figure 30 In step S3201, the information processing device 34 uses basic data to enable the learning model to learn. The basic data is learning data representing the combination of the skilled worker's actions during the operation and the actions of the equipment used by the skilled worker.

[0318] In step S3202, the information processing device 34 performs a simulation using the learned model. Specifically, the information processing device 34 uses the learned model from step S3201 to simulate a task performed by the robot using equipment. For example, the information processing device 34 performs a simulation from the beginning to the end of the humanoid robot 1 sewing fabric 100 and fabric 101 using sewing machine 200.

[0319] In step S3203, the information processing device 34 acquires the robot's work sequence through simulation. For example, the information processing device 34 acquires time-series data containing a combination of motion parameters of the humanoid robot 1 and the motion of the sewing machine 200 as the first work sequence.

[0320] In step S3204, the information processing device 34 performs process change processing. Details of the process change processing will be described later.

[0321] In step S3205, the information processing device 34 performs a simulation based on the modified work procedure. In this embodiment, the information processing device 34 performs a simulation based on the work procedure that was modified in step S3204, which is the second work procedure.

[0322] In step S3206, the information processing device 34 acquires correction data from the simulation. For example, the information processing device 34 acquires learning data representing the combination of the movements of the humanoid robot 1 and the sewing machine 200 as correction data from the simulation executed in step S3205.

[0323] In step S3207, the information processing device 34 relearns the learned model by correcting the data. Specifically, the information processing device 34 uses the corrected data obtained in step S3206 to relearn the learned model that was learned in step S3201. Then, the information processing device 34 ends the relearning process.

[0324] Then, the relearned learning model is sent from the information processing device 34 to the control system 10 of the humanoid robot 1.

[0325] Figure 31 This is a flowchart illustrating the process of process change handling that improves the efficiency of work processes according to the sixth embodiment.

[0326] exist Figure 31 In step S3301, the information processing device 34 divides the work process into multiple processes.

[0327] In step S3302, the information processing device 34 deletes any unprotected individual segmented processes. It should be noted that process protection is performed in step S3307, which will be described later.

[0328] In step S3303, the information processing device 34 performs a simulation based on a work process in which at least one step has been removed. This simulation may involve, for example, physical parameters and... Figure 30 The simulation in step S3201 is the same as that in step S3201. That is, by performing a simulation with the same physical parameters, the impact of the changed process on the deliverables is observed.

[0329] exist Figure 31 In step S3304, the information processing device 34 acquires data of deliverables as the result of the operation from the simulation. This simulation is the one performed in step S3303.

[0330] In step S3305, the information processing device 34 determines whether the difference between the target data and the deliverable data is less than a predetermined threshold. When the information processing device 34 determines that the difference is not less than the predetermined threshold (step S3305: No), it proceeds to step S3306. On the other hand, when the information processing device 34 determines that the difference is less than the predetermined threshold (step S3305: Yes), it proceeds to step S3308.

[0331] In step S3306, the information processing device 34 restores the deleted process to its original state. Specifically, the deleted process is the process that was deleted in step S3302.

[0332] In step S3307, the information processing device 34 protects the process of restoring the original state. Protecting the process means, for example, affixing a protection mark to the process of restoring the original state.

[0333] In step S3308, the information processing device 34 determines whether all unprotected split processes have been deleted. If the information processing device 34 determines that all unprotected split processes have not been deleted (step S3308: No), it proceeds to step S3302. On the other hand, if the information processing device 34 determines that all unprotected split processes have been deleted (step S3308: Yes), the process change processing ends.

[0334] (Summary of the sixth implementation method) As described above, in the information processing apparatus 34 according to the sixth embodiment, the learning unit 346 relearns the completed learning model using learning data obtained from the simulation executed by the execution unit 340. Therefore, according to this embodiment, when acquiring the learning data required for relearning the learning model, it is not necessary to actually run the robot, and thus there is no need to prepare robots, equipment, or workpieces. That is, since learning data can be acquired from the simulation, the relearning of the learning model can be performed solely by a computer. Furthermore, since multiple simulations can be executed in parallel, learning data can be acquired efficiently.

[0335] In this embodiment, the learning data is obtained from simulations of work processes where a portion of the process has been removed via the process change unit 344. By using this learning data to relearn the learning model, the work processes of the humanoid robot 1 can be made more efficient. Furthermore, unnecessary actions dependent on the individual skilled worker, included in the learning model learned from the skilled worker's movements, can be removed. These actions include, for example, actions for adjusting posture or preparatory actions for picking up tools.

[0336] [Modifications of the Sixth Embodiment] In the sixth embodiment, the learning model is relearned using learning data obtained from simulations of work processes modified by a process change process that reduces the number of work steps; however, the changes to work processes are not limited to this. Therefore, in a variation, the learning model is relearned using learning data obtained from simulations of work processes with a predetermined number of steps added.

[0337] The predetermined process is the robot's action. The robot's actions include its own movements and the actions of operating the equipment. Examples of robot actions include the humanoid robot 1 shaking its head, raising its arm, or performing flexion and extension movements. As an example, the humanoid robot 1 can wave its hand or shake its head while performing sewing. Furthermore, as other examples, the humanoid robot 1 can sprinkle seasonings such as salt and oil from a height while performing cooking operations, or drain noodles while moving its entire body.

[0338] Figure 32 This is a flowchart illustrating the process change handling involving the addition of a predetermined step in a variation of the sixth embodiment.

[0339] exist Figure 14 In step S3401, the information processing device 34 determines the timing for adding the predetermined process. This timing may be selected, for example, from the acquired work processes or from any predetermined timing.

[0340] In step S3402, the information processing device 34 adds a predetermined process.

[0341] In step S3403, the information processing device 34 performs a simulation based on a work process with at least one predetermined step added.

[0342] In step S3404, the information processing device 34 acquires data of deliverables as the result of the operation from the simulation. This simulation is the one performed in step S3403.

[0343] In step S3405, the information processing device 34 determines whether the difference between the target data and the deliverable data is less than a predetermined threshold. When the information processing device 34 determines that the difference is not less than the threshold (step S3405: ​​No), it proceeds to step S3406. On the other hand, when the information processing device 34 determines that the difference is less than the threshold (step S3405: ​​Yes), the process change processing ends.

[0344] In step S3406, the information processing device 34 deletes the added predetermined process.

[0345] In step S3407, the information processing device 34 changes the timing of adding the predetermined process.

[0346] In a variation of the sixth embodiment, a predetermined step is added to the work process of the humanoid robot 1. This allows for the incorporation of eye-catching movements into the work process. By adding these movements, viewers can appreciate the work process of the humanoid robot 1.

[0347] [Remark] In the sixth embodiment, the simulation is performed by the execution unit 340, but the execution of the simulation is not limited to this. The simulation may be performed by an information processing device that is separate from the information processing device 34, a system consisting of multiple devices, or software, etc.

[0348] In the sixth embodiment, the work process is divided into segments at predetermined time intervals, but the division is not limited to this. The segmentation can be performed in units of action of the humanoid robot 1 (e.g., arm movement, finger angle change, etc.), or continuous actions can be performed as a single unit.

[0349] In the sixth embodiment, the efficiency of the work process is improved by deleting one of the divided work processes, but the efficiency of the work process is not limited to this. It can also be improved by reducing the motion parameters of the humanoid robot 1.

[0350] The learning model can be relearned through simulation as described in the sixth embodiment, after relearning the differences between the actual operation results and those described in the first embodiment. Alternatively, relearning can be performed in the reverse order described above.

[0351] The present disclosure has been described above using embodiments, but the technical scope of the present disclosure is not limited to the scope described in the above embodiments. Those skilled in the art should understand that various changes or improvements can be made to the above embodiments. As can be seen from the claims, embodiments with such changes or improvements can also be included within the technical scope of the present invention.

[0352] It should be noted that the execution order of actions, sequences, steps, and stages in the apparatus, systems, programs, and methods shown in the claims, specification, and drawings is not specifically stated as "before," "earlier than," etc., or can be implemented in any order as long as the output of the preceding process is not used in the subsequent process. Even if the flow of actions in the claims, specification, and drawings is described using terms such as "firstly" or "next" for convenience, it does not mean that the actions must be performed in that order.

[0353] While fabrics 100 and 101 are listed as examples of workpieces, and the work performed by the humanoid robot 1 is set as sewing, the humanoid robot 1 can also be used for work other than sewing. For example, the humanoid robot 1 can be applied to cutting operations using lathes or milling machines, welding operations, etc. That is, since the humanoid robot 1 can use machine tools used by humans, it is possible to automate work using robots while maintaining existing equipment. Furthermore, the robot is not limited to a humanoid shape, as long as it has a gripping part 20.

[0354] The disclosures of Japanese Patent Application No. 2023-63764, filed on April 10, 2023; Japanese Patent Application No. 2023-109653, filed on July 3, 2023; Japanese Patent Application No. 2023-126494, filed on August 2, 2023; Japanese Patent Application No. 2023-133490, filed on August 18, 2023; and Japanese Patent Application No. 2023-133515, all of which are incorporated herein by reference.

[0355] All documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent that each document, patent application, and technical standard is specifically and individually incorporated herein by reference.

Claims

1. A robot, said robot having: The gripping part includes a palm portion that serves as a base for holding the workpiece, and a plurality of fingers that extend radially from the palm portion; A palm sensor unit, disposed on the palm, detects workpiece information including the shape and configuration of the workpiece; and The control unit inputs the detection results from the palm sensor into a learning model that has been completed after learning using learning data, and controls the holding unit to hold and operate the workpiece based on the motion parameters obtained by performing the calculations of the learning model. The holding unit is used to hold the workpiece when operating the machine tool. The learning data represents the combination of hand movements and machine tool movements of a skilled operator during operation.

2. The robot according to claim 1, wherein, It has two gripping parts corresponding to the right and left hands; The fingers of one of the gripping parts are five; The workpiece is fabric; The machine tool is a sewing machine; The task described is sewing.

3. The robot according to claim 1, wherein, The learning model is a model that has been relearned by utilizing the difference between the target task result and the actual task result performed by the robot.

4. The robot according to claim 1, wherein, The parameters provided to the learning model are determined based on the task and the task method.

5. The robot according to claim 1, wherein, The workpiece has an identifier indicating a specific location as a specific position; The control unit controls the holding unit to hold and operate the workpiece based on the detected identifier.

6. The robot according to claim 5, wherein, The control unit inputs the detection result into the learning model that has been completed after relearning. The learning model is a learning model in the device that outputs the difference between the position of the identifier in the target task result and the position of the identifier in the actual task result.

7. The robot according to claim 5, wherein, The identifier is a recognition code that can read character information; The character information includes at least one of the following: the identification information of the work object, the work sequence, or the specific location.

8. The robot according to claim 5, wherein, The control unit controls the holding unit to hold and operate the workpiece based on information from the shooting device that captures images of blind spots that cannot be detected by the palm sensor unit.

9. A robot control program that causes a computer to operate as a control unit as described in any one of claims 1 to 8.

10. An information processing apparatus, the information processing apparatus comprising: The acquisition unit acquires learning data representing a combination of actions performed by a skilled worker and actions performed by the equipment used by the skilled worker during the work. A specific unit, from the acquired learning data, identifies first learning data representing a specific task performed by a skilled worker using a specific device, wherein the specific device is the specific device. as well as The conversion unit converts the specific first learning data into second learning data, which enables a robot equipped with one or more tools corresponding to the specific device to perform an action corresponding to the specific task.

11. The information processing apparatus according to claim 10, wherein, The conversion unit converts the first learning data based on a conversion table that stores the first learning data and the second learning data in correspondence.

12. The information processing apparatus according to claim 11, wherein, The learning data includes data on device labels assigned to the actions of the device, wherein the device labels are labels indicating the type of the device; The conversion unit converts the learning data by changing the device label of the first learning data to a tool label, wherein the tool label is a label indicating the type of the tool corresponding to the device.

13. An information processing apparatus, the information processing apparatus comprising: A specific unit inputs detection results from the robot's sensors into a learning model that has been trained using learning data representing a combination of actions of a skilled worker during a task and actions of the equipment used by the skilled worker. From the action parameters obtained through the computational processing of the learning model, the specific unit identifies a first action parameter. This first action parameter represents a specific task performed by the robot using a specific piece of equipment, where the specific equipment is specific and the specific task is specific. The conversion unit converts the specified first action parameters into second action parameters, which cause the robot equipped with one or more tools corresponding to the specific device to perform an action corresponding to the specific task.

14. The information processing apparatus according to claim 13, wherein, The conversion unit converts the first action parameter based on a conversion table that stores the first action parameter and the second action parameter in correspondence.

15. A method for generating learning data, wherein, The computer performs the following processing: Acquire learning data representing the combination of actions of a skilled worker during a task and the actions of the equipment used by the skilled worker; From the acquired learning data, first learning data is specifically identified representing a specific task performed by a skilled user using a specific device, wherein the specific device is the specific device. The first specific learning data is converted into second learning data, which enables a robot equipped with one or more tools corresponding to the specific device to perform actions corresponding to the specific task.

16. A method for generating motion parameters, wherein, The computer performs the following processing: In a learning model that has been completed after learning using learning data representing a combination of actions of a skilled worker and actions of the equipment used by the skilled worker, the detection results of the sensors of the robot are input, and a first action parameter is identified from the action parameters obtained by performing the computational processing of the learning model. The first action parameter represents a specific task performed by the robot using a specific device, where the specific device is a specific device and the specific task is a specific task. The first specific action parameter is converted into a second action parameter, which causes the robot, equipped with one or more tools corresponding to the specific device, to perform an action corresponding to the specific task.

17. An information processing apparatus, the information processing apparatus comprising: The acquisition unit acquires information about the workpiece; A switching unit, based on information about the workpiece acquired by the acquisition unit, switches between multiple completed learning models, wherein the multiple completed learning models are learning models trained using learning data representing combinations of hand movements and machine tool movements performed by skilled workers during work; and The control unit inputs the detection results of the sensor unit of the robot into the learning model after being switched by the switching unit, and controls the holding unit of the robot to hold and operate the workpiece based on the motion parameters obtained by performing the calculation processing of the learning model.

18. The information processing apparatus according to claim 17, wherein, The switching unit determines one operation mode from multiple operation modes based on the information of the work object obtained by the obtaining unit; The control unit controls the robot based on the operating mode determined by the switching unit; The multiple operating methods include: A first operating mode in which only one of the robots performs the task; as well as A second operation mode in which the task is performed by multiple robots.

19. The information processing apparatus according to claim 18, wherein, The multiple completed learning models are learning models after learning for each type of the aforementioned task; The first operating method is cell production; The second operating method is the assembly line production method.

20. An information processing apparatus, the information processing apparatus comprising: The learning unit uses first learning data representing a combination of the actions of a skilled worker during a task and the actions of the equipment used by the skilled worker to enable the learning model to learn. The acquisition unit obtains the first operation sequence of the robot from a simulation of an operation performed by the robot using the device, executed using a learning completion model learned by the learning unit; as well as The modification unit, based on a predetermined standard, modifies the first work step into a second work step, wherein the second work step is a modified part of the first work step; wherein... The learning unit uses second learning data, which represents a combination of the robot's actions and the device's actions, obtained from simulations of the robot's operations based on the second work process, to enable the learning-completed model to relearn.

21. The information processing apparatus according to claim 20, wherein, The predetermined benchmark is the target result of the task as the objective, and the comparison result of the task result when the simulation was performed.

22. The information processing apparatus according to claim 21, wherein, The second work step is a work step that removes at least one step from the first work step, which is divided into multiple steps.

23. The information processing apparatus according to claim 22, wherein, The robot has two gripping parts corresponding to the right and left hands; Five fingers are provided on one of the gripping parts; The device is a sewing machine; The task described is sewing.

24. The information processing apparatus according to claim 21, wherein, The second work process is a work process in which at least one additional process is added to the first work process.

25. A program that causes a computer to operate as an information processing apparatus according to any one of claims 10 to 14, or claims 17 to 24.

26. A method for generating a learned complete model, wherein, Computer execution: The learning model learns by using first learning data that represents a combination of the actions of a skilled worker during a task and the actions of the equipment used by the skilled worker. The first work procedure of the robot is obtained from the simulation of the operation performed by the robot using the device, which is executed by using the learned completion model; Based on a predetermined benchmark, the first work procedure is changed to a second work procedure, wherein the second work procedure is a part of the first work procedure that has been changed. The learning-completed model is relearned using second learning data, which represents a combination of the robot's actions and the device's actions, obtained from simulations of the robot's operations based on the second work process.

Images