Computer system and robot control method

Abstract

To achieve automatic control of a robot by using an existing model without performing relearning.SOLUTION: A computer system generates a transformation formula for transforming measurement data acquired by a sensor such that a second dynamical system matches a first dynamical system on the basis of a difference between the first dynamical system, representing work in a learning environment, and the second dynamical system, representing work in a control environment, transforms the measurement data by using the transformation formula, inputs the transformed measurement data to a model, and controls a robot on the basis of an output from the model.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a method for controlling a robot and a robot control device.

Background Art

[0002] There is a technology for automatically controlling a robot that performs operations including gripping and moving a workpiece, such as an assembly operation. In automatic control, a model that takes measurement data acquired from a sensor attached to the robot as input is used. The model is generated by machine learning. For example, the technology described in Patent Document 1 is known.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] [[ID=Z39]]When an existing model is utilized, it is premised that the actual working environment (control environment) is the same as the working environment (learning environment) in machine learning. However, when the control environment, such as the sensor attachment position, the shape and orientation of the workpiece, and the position of the positioning pin, is different from the learning environment, the magnitude and direction of the force applied to the workpiece by the robot change, so retraining of the model becomes necessary.

[0005] In order to reuse an existing model, a method of converting the measurement data input to the model can be considered. As a technique for converting measurement data, the technique described in Patent Document 2 is known.

[0006] Patent Document 1 discloses a state detection device that includes "an acceleration information acquisition unit 110 that acquires detected acceleration from an acceleration sensor 200, a correction value acquisition unit 120 that acquires a first coordinate transformation correction value, a coordinate transformation unit 130 that performs coordinate transformation processing of the detected acceleration from the acceleration sensor coordinate system to the motion analysis coordinate system based on the first coordinate transformation correction value, and a motion analysis unit 140 that performs motion analysis processing based on the detected acceleration after the coordinate transformation processing."

[0007] The technology described in Patent Document 2 can eliminate the difference in coordinate systems between environments. However, even with the technology described in Patent Document 2, it is not possible to eliminate the difference in the dynamical systems representing the operations between environments.

[0008] This invention provides a technology for automatically controlling tasks performed by a robot using an existing model by transforming measurement data to eliminate differences in the dynamic systems of work between different environments. [Means for solving the problem]

[0009] To solve the above problems, for example, the configuration described in the claims may be adopted.

[0010] A typical example of the invention disclosed in this application is as follows: A computer system for controlling a robot that performs tasks including gripping and moving a workpiece using a model generated by machine learning, comprising a computer having a computing device, a storage device connected to the computing device, and a network interface connected to the computing device, which generates a first transformation formula for transforming measurement data indicating the operating state of the robot so that the second dynamic system matches the first dynamic system based on the difference between a first dynamic system representing work in a learning environment and a second dynamic system representing work in a control environment, and when the measurement data is acquired, the measurement data is transformed using the first transformation formula, the transformed measurement data is input to the model, and the robot is controlled based on the output of the model. [Effects of the Invention]

[0011] According to the present invention, by transforming measurement data to eliminate differences in the dynamic systems of work between environments, it is possible to automatically control the work performed by a robot using an existing model. Problems, configurations, and effects other than those described above will be clarified by the following description of embodiments. [Brief explanation of the drawing]

[0012] [Figure 1] This is a diagram showing an example of the system configuration of Example 1. [Figure 2A] This figure shows an example of the data structure of the equipment management information in Example 1. [Figure 2B] This figure shows an example of the data structure of the equipment management information in Example 1. [Figure 3] This figure shows an example of the data structure of the orbital management information in Example 1. [Figure 4] This figure shows an example of the data structure of the model management information in Example 1. [Figure 5] This figure shows an example of the data structure of the conversion formula management information in Example 1. [Figure 6] This figure shows an example of the data structure of work management information in Example 1. [Figure 7] This is a flowchart illustrating an example of the robot control process performed by the computer in Example 1. [Figure 8] This figure shows an example of a screen displayed by the computer in Example 1. [Figure 9] This figure shows an example of a screen displayed by the computer in Example 1. [Figure 10] This is a flowchart illustrating an example of the conversion formula (A) calculation process performed by the computer in Example 1. [Figure 11A] This figure shows a specific example of the conversion formula (A) calculation process performed by the computer in Example 1. [Figure 11B] This figure shows a specific example of the conversion formula (A) calculation process performed by the computer in Example 1. [Figure 11C] This figure shows a specific example of the conversion formula (A) calculation process performed by the computer in Example 1. [Figure 12] It is a flowchart for explaining an example of the conversion formula (B) calculation process executed by the computer of Example 1. [Figure 13A] It is a diagram showing a specific example of the conversion formula (B) calculation process executed by the computer of Example 1. [Figure 13B] It is a diagram showing a specific example of the conversion formula (B) calculation process executed by the computer of Example 1. [Figure 13C] It is a diagram showing a specific example of the conversion formula (B) calculation process executed by the computer of Example 1. [Figure 13D] It is a diagram showing a specific example of the conversion formula (B) calculation process executed by the computer of Example 1. [Figure 13E] It is a diagram showing a specific example of the conversion formula (B) calculation process executed by the computer of Example 1. [Figure 13F] It is a diagram showing a specific example of the conversion formula (B) calculation process executed by the computer of Example 1.

Mode for Carrying Out the Invention

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not construed as being limited to the description of the embodiments shown below. It will be easily understood by those skilled in the art that the specific configuration can be changed without departing from the spirit or gist of the present invention.

[0014] In the configuration of the invention described below, the same or similar configurations or functions are denoted by the same reference numerals, and duplicate descriptions are omitted.

[0015] Expressions such as "first", "second", "third", etc. in this specification and the like are attached to identify components, and do not necessarily limit the number or order.

[0016] The positions, sizes, shapes, and ranges of each configuration shown in the drawings and the like may not represent the actual positions, sizes, shapes, and ranges in order to facilitate understanding of the invention. Therefore, in the present invention, it is not limited to the positions, sizes, shapes, and ranges disclosed in the drawings and the like. [Examples]

[0017] Figure 1 shows an example of the system configuration of Embodiment 1. The system consists of a computer 100 and a robot 101. The computer 100 and the robot 101 are connected directly or via a network.

[0018] The computer 100 comprises an arithmetic unit 110, a storage device 111, a communication device 112, an input device 113, and an output device 114. Each hardware element is connected, for example, via an internal bus, but is not limited to this method.

[0019] The input device 113 is a device for inputting data and commands into the computer 100. Examples of input devices 113 include a keyboard and a mouse.

[0020] The output device 114 is a device that outputs the results of control value changes, etc. The output device 114 may be, for example, a display, a printer, etc.

[0021] The storage device 111 is a device that stores programs and information executed by the arithmetic unit 110. The storage device 111 is, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc. The storage device 111 also stores information input via the input device 113 and the calculation results of the programs. Furthermore, the storage device 111 is also used as a work area.

[0022] The storage device 111 stores equipment management information 130, track management information 131, model management information 132, conversion formula management information 133, and work management information 134.

[0023] The equipment management information 130 stores equipment configuration information related to the configuration of the equipment on which the robot 101 performs its work. The data structure of the equipment management information 130 will be described later using Figures 2A and 2B.

[0024] The trajectory management information 131 stores trajectory information, including control values ​​for controlling the robot's trajectory during the operation. The data structure of the trajectory management information 131 will be described later with reference to Figure 3.

[0025] Model management information 132 stores models that output data values ​​used for controlling the robot. The data structure of model management information 132 will be described later with reference to Figure 4.

[0026] The model is, for example, a neural network, and is generated by reinforcement learning, deep learning, and deep reinforcement learning. The model in Example 1 accepts trajectory information, robot operating status information, and measurement data as input, and outputs a correction amount for the control values ​​included in the trajectory information. Note that the present invention is not limited to the structure of the model, the output of the model, and the learning method of the model.

[0027] The conversion formula management information 133 stores the conversion formula for the measurement data. The data structure of the conversion formula management information 133 will be described later using Figure 5.

[0028] Work management information 134 stores work information related to a work. The data structure of work management information 134 will be described later using Figure 5.

[0029] The arithmetic unit 110 is a device that controls the computer 100, and is, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array). The arithmetic unit 110 executes a program stored in the storage device 111. By executing processing according to the program, the arithmetic unit 110 operates as a functional unit (module) that realizes a specific function. In the following description, when a functional unit is described as the subject of processing, it indicates that the arithmetic unit 110 is executing a program that realizes that functional unit. In Embodiment 1, the arithmetic unit 110 functions as an equipment position and attitude calculation unit 120, an equipment analysis unit 121, a conversion formula generation unit 122, a measurement data conversion unit 123, a correction amount determination unit 124, and a control value calculation unit 125.

[0030] The equipment position and orientation calculation unit 120 calculates the position and orientation of the equipment and workpieces that constitute the equipment based on the equipment configuration information and trajectory information.

[0031] The equipment analysis unit 121 analyzes the differences between environments based on the equipment configuration information of each learning environment and control environment. For example, the equipment analysis unit 121 calculates the difference in position and orientation of sensors attached to the tip of the robot 101 in each environment, and also determines whether or not there are positioning pins for the workpiece in each environment.

[0032] The conversion formula generation unit 122 generates a conversion formula for converting measurement data so that differences between environments are eliminated.

[0033] The measurement data conversion unit 123 converts the measurement data acquired from the measurement device 140, which will be described later, using the conversion formula generated by the conversion formula generation unit 122.

[0034] The correction amount determination unit 124 calculates the correction amount by inputting the converted measurement data into the model.

[0035] The control value calculation unit 125 calculates a control value to be transmitted to the robot 101 based on the trajectory information and the correction amount calculated by the correction amount determination unit 124.

[0036] Furthermore, regarding the functional units of the computer 100, multiple functional units may be combined into a single functional unit, or a single functional unit may be divided into multiple functional units according to its function.

[0037] The communication device 112 is a device for communicating with an external device, such as a NIC (Network Interface Card).

[0038] The robot 101 performs operations, including gripping the workpiece and moving it from the starting point to the ending point, based on control information including control values ​​calculated by the computer 100.

[0039] The robot 101 includes a measuring device 140, a group of work devices 141, and a controller 142.

[0040] The measuring device 140 measures values ​​to understand the state of the workpiece as a result of the robot 101's work and generates measurement data. The measuring device 140 is a general sensing device such as an acceleration sensor, force sensor, camera, contact sensor, and current sensor.

[0041] The working device group 141 is a group of devices that enable gripping and moving of the workpiece, and is a group of general mechanical elements such as tools, links, and drive motors.

[0042] The controller 142 controls the work device group 141 based on control values ​​received from the computer 100. For example, the controller 142 moves the tool by controlling the drive motors that connect the links that function as joints, according to the control values. The controller 142 outputs operating status information to the computer 100, including the joint angle, angular velocity, and angular acceleration, as well as the torque and current values ​​of the drive motors.

[0043] Figures 2A and 2B show an example of the data structure of the equipment management information 130 in Example 1.

[0044] The equipment management information 130 stores entries that include ID 201, equipment name 202, classification 203, item 204, and content 205. Each entry corresponds to one piece of equipment configuration information. Each entry contains rows for multiple elements that make up the equipment.

[0045] ID201 is a field that stores identification information for equipment configuration. Equipment name202 is a field that stores the name of the equipment. Classification203 is a field that stores the classification of the elements that make up the equipment. Classification203 stores objects, robots, links, joints, etc. Item204 is a field that stores the management items of the elements. Content205 is a field that stores the content of the management items. Content205 stores files, numbers, strings, etc.

[0046] The "Assembly Cell Stand" element manages the name, mounting target, relative position, relative orientation, and shape file information. The "Force Sensor" element manages the name, mounting target, relative position, relative orientation, and shape file information. The "Hand" element manages the name, mounting target, relative position, relative orientation, and shape file information. The "Workpiece" element manages the name, workpiece type, presence or absence of positioning pins, weight, mounting target, relative position, relative orientation, and shape file information. The "Robo" element manages the name, mounting target, relative position, and relative orientation. The "Link" element manages shape file information. The "Joint" element stores the parent link, child link, joint type, lower joint angle limit, upper joint angle limit, and upper joint angle velocity limit.

[0047] The relative position and orientation of an element refer to its position and orientation relative to the mounting object. The assembly cell frame is the root of the equipment, and an absolute coordinate system is established with any point on the assembly cell frame as the origin.

[0048] Figure 3 shows an example of the data structure of the orbital management information 131 in Example 1.

[0049] Orbit management information 131 stores entries including ID 301, orbit name 302, attitude 303, and time 304. Each entry corresponds to one piece of orbital information.

[0050] ID301 is a field that stores identification information for the orbit. Orbit name302 is a field that stores the name of the orbit. Attitude303 is a group of fields that stores the angles of the joint axes at the waypoints. Time304 is a field that stores the time from when movement starts from the starting waypoint to when it arrives at the waypoint. The Time304 for the starting waypoint is "0.0".

[0051] Figure 4 shows an example of the data structure of the model management information 132 in Example 1.

[0052] Model management information 132 stores entries including ID 401, model 402, equipment name 403, and track name 404. Each entry corresponds to one model.

[0053] ID401 is a field that stores the model's identification information. Model402 is a field that stores the model. Equipment name403 is a field that stores the name of the equipment used when training the model. Equipment name403 corresponds to equipment name202. Trajectory name404 is a field that stores the name of the trajectory used when training the model. Trajectory name404 corresponds to trajectory name302.

[0054] Figure 5 shows an example of the data structure of the conversion formula management information 133 in Example 1.

[0055] The conversion formula management information 133 stores entries including ID 501, model 502, equipment name 503, track name 504, conversion formula (A) 505, and conversion formula (B) 506. Each entry corresponds to one operation.

[0056] ID501 is a field that stores identification information for the operation. Model502 is a field that stores the model applied to control the operation. Equipment name503 is a field that stores the name of the equipment in the operation. Equipment name503 corresponds to equipment name202. Track name504 is a field that stores the name of the track in the operation. Track name504 corresponds to track name302. Conversion formula (A)505 and conversion formula (B)506 are fields that store conversion formulas for converting measurement data. The conversion formula stored in conversion formula (A)505 is a formula for aligning the coordinate systems of sensors in two environments. The conversion formula stored in conversion formula (B)506 is a formula for aligning the dynamical systems of two environments. Here, the dynamical system of the environment means the dynamical system related to the gripping and movement of the workpiece. The dynamical system of the environment is described using parameters (force parameters) related to the forces and moments acting on the workpiece.

[0057] Figure 6 shows an example of the data structure of the work management information 134 in Example 1.

[0058] Work management information 134 stores entries including ID 601, equipment name 602, work type 603, moment of inertia parameter 604, and positioning pin parameter 605. Each entry corresponds to one piece of work information.

[0059] ID601 is a field that stores identification information for the workpiece. Equipment name602 is a field that stores the name of the equipment that handles the workpiece. Equipment name602 corresponds to equipment name202. Workpiece type603 is a field that stores the type of workpiece. Moment of inertia parameters604 is a group of fields that stores parameters for determining the moment of inertia of the workpiece. Positioning pin parameters605 is a group of fields that stores parameters related to the position of the workpiece's positioning pins.

[0060] The inertia moment parameter 604 and the positioning pin parameter 605 store parameters for calculating force parameters. Note that the inertia moment parameter 604 and the positioning pin parameter 605 are examples and are not limited to them. For example, fields for storing parameters related to material, stiffness, etc., may also be included.

[0061] Figure 7 is a flowchart illustrating an example of the robot control process performed by the computer 100 in Example 1. Figures 8 and 9 show examples of screens displayed by the computer 100 in Example 1.

[0062] The computer 100 displays the screen 800 shown in Figure 8 via the output device 114 and accepts user input regarding the model, equipment, and track in the operation (step S101).

[0063] The screen 800 includes an input area 801, an environment display area 802, a conversion formula display area 803, and an operation button area 804.

[0064] Input area 801 is an area for inputting models, equipment, tracks, etc., and includes selection fields 811, 813, 815, load buttons 812, 814, 816, and display field 817.

[0065] Selection field 811 is for selecting a model. When the load button 812 is pressed, the model specified in selection field 811 is read from the model management information 132. Selection field 813 is for selecting equipment. When the load button 814 is pressed, the equipment configuration information of the equipment specified in selection field 813 is read from the equipment management information 130. Selection field 815 is for selecting a track. When the load button 816 is pressed, the track information of the track specified in selection field 815 is read from the track management information 131. Display field 817 is for displaying the type of workpiece used in the specified equipment.

[0066] Furthermore, the system may accept model, equipment configuration information, and track information directly from the user.

[0067] The environment display area 802 is a section for displaying the learning environment and the control environment. The environment display area 802 includes display sections 821 and 822.

[0068] Display field 821 is a field for displaying information about the learning environment. Computer 100 refers to model management information 132 and searches for an entry corresponding to the model specified in selection field 811. Based on the equipment name 403 and track name 404 of the searched entry, computer 100 obtains equipment configuration information and track information from equipment management information 130 and track management information 131. Based on the obtained information, computer 100 generates an image representing the learning environment and displays it in display field 821.

[0069] Display field 822 is a field for displaying information about the control environment. Based on the information specified in selection fields 813 and 815, the computer 100 generates an image representing the actual work environment and displays it in display field 822.

[0070] In addition, display fields 821 and 822 may display parameters related to the shape of the workpiece, moment of inertia, position of the positioning pin, etc.

[0071] The conversion formula display area 803 is the field where the conversion formula is displayed.

[0072] The operation button area 804 is a field for displaying operation buttons to instruct the computer 100 to execute a process. The operation button area 804 includes a conversion formula button 841 and a control button 842. The conversion formula button 841 is an operation button for instructing the computer to execute the conversion formula calculation process. The control button 842 is an operation button for instructing the computer to execute the control process.

[0073] After the user enters the necessary information in the input area 801, they operate the conversion formula button 841. When the computer 100 receives this operation, it executes the conversion formula (A) calculation process and the conversion formula (B) calculation process (steps S102 and S103). Details of the conversion formula (A) calculation process and the conversion formula (B) calculation process will be described later.

[0074] The computer 100 displays the processing result in the conversion formula display area 803. After the user confirms the conversion formula display area 803, the user operates the control button 842. The computer 100 starts the control process (step S104). The control process is repeatedly executed until the operation, including gripping and moving the workpiece, is completed.

[0075] Computer 100 acquires measurement data from measuring device 140 and operational status information from controller 142 via communication device 112 (step S105).

[0076] Computer 100 converts the measurement data using conversion formula (A) and conversion formula (B) (step S106). Specifically, the measurement data conversion unit 123 converts the measurement data into first converted measurement data using conversion formula (A), and converts the first converted measurement data into second converted measurement data using conversion formula (B).

[0077] Computer 100 calculates correction amounts by inputting trajectory information, converted measurement data (second converted measurement data), and operating status information into the model (step S107). The model outputs correction amounts that match the operating cycle of robot 101. For example, if the operating cycle of robot 101 is Δt seconds and T seconds have elapsed since robot 101 started operating, the correction amounts for each joint angle of the robot at point T+Δt are calculated.

[0078] Computer 100 corrects the control values ​​included in the trajectory information based on the correction amount and generates control information including the corrected control values ​​(step S108). For example, if the operation period of robot 101 is Δt seconds and T seconds have elapsed since robot 101 started operation, the correction is performed by adding the correction amount to the angle (control value) of each joint at T+Δt in the trajectory information.

[0079] Computer 100 transmits control information to controller 142 via communication device 112 (step S109).

[0080] If the completion conditions for the task are not met, the computer 100 returns to step S105 and performs the same process. If the completion conditions for the task are met, the computer 100 terminates the control process.

[0081] The completion condition for the task is, for example, the time elapsed since the start of robot control based on trajectory information to the final waypoint of the trajectory (Time 304). Alternatively, the calculation of the control value at the final waypoint of the trajectory may also be used as the completion condition for the task.

[0082] The computer 100 presents the execution result via the output device 114 (step S110), and then terminates the robot control process.

[0083] For example, computer 100 displays screen 900. Screen 900 includes display fields 901 and 902. Display field 901 displays the model, equipment configuration information, trajectory information, work type, and conversion formula. Display field 902 displays the execution results, including a time-series graph 921 of the measurement data before conversion, a time-series graph 922 of the measurement data after conversion, and an image 923 showing the trajectory of the work.

[0084] In Image 923, the solid line represents the trajectory of robot 101 when controlled based on trajectory information, while the dotted line represents the trajectory of robot 101 when controlled based on corrected control values.

[0085] In the robot control process of Example 1, the operation of the robot 101 can be controlled under the control environment using an existing model. For example, even if a workpiece position shift occurs, the control value can be corrected according to the amount of shift by using the model.

[0086] Figure 10 is a flowchart illustrating an example of the conversion formula (A) calculation process performed by the computer 100 in Example 1. Figures 11A, 11B, and 11C show specific examples of the conversion formula (A) calculation process performed by the computer 100 in Example 1.

[0087] The equipment position and orientation calculation unit 120 acquires equipment configuration information and trajectory information of the learning environment (step S201). Specifically, the following processes are executed.

[0088] (S201-1) The equipment position and orientation calculation unit 120 refers to the model management information 132 and searches for the entry in which the model specified for model 402 is stored.

[0089] (S202-2) The equipment position and orientation calculation unit 120 refers to the equipment management information 130 and searches for an entry in which the value of the equipment name 202 matches the value of the equipment name 403 of the entry retrieved from the model management information 132. The equipment position and orientation calculation unit 120 obtains the equipment configuration information corresponding to the retrieved entry.

[0090] (S202-3) The equipment position and attitude calculation unit 120 refers to the track management information 131 and searches for an entry in which the value of the track name 302 matches the value of the track name 404 of the entry retrieved from the model management information 132. The equipment position and attitude calculation unit 120 obtains the track information corresponding to the retrieved entry.

[0091] The equipment position and orientation calculation unit 120 calculates the position and orientation of the sensor in the learning environment based on the equipment configuration information and trajectory information of the learning environment (step S202). Here, the position and orientation of the sensor in the learning environment refers to the position and orientation of the sensor at the start of the operation.

[0092] Figure 11A shows an example of the position and orientation of a sensor in a learning environment. In Figure 11A, the position and orientation of the sensor are shown relative to the equipment coordinate system 1100. The equipment coordinate system 1100 is an absolute coordinate system of the learning environment with an arbitrary point on the equipment as the origin. In Figure 11A, the equipment coordinate system 1100 is an absolute coordinate system with the installation position of the assembly cell frame A, which has the mounting target as the Root, as the origin. Coordinate system 1101 is a relative coordinate system with the sensor as the origin.

[0093] The equipment position and orientation calculation unit 120 calculates the position and orientation of the sensor based on the equipment coordinate system 1100 and equipment configuration information. First, RoboB is attached to the assembly cell stand A. The equipment position and orientation calculation unit 120 calculates the mounting position of RoboB based on the equipment coordinate system 1100 and the relative position and orientation of RoboB with respect to the assembly cell stand A. Link_ee is attached to RoboB. The equipment position and orientation calculation unit 120 calculates the position and orientation of Link_ee based on the mounting position of RoboB and the angles of each joint at the starting points of RoboB's trajectory information. A force sensor is installed on Link_ee. The equipment position and orientation calculation unit 120 calculates the position and orientation of the force sensor based on the position and orientation of Link_ee and the relative position and orientation of the force sensor with respect to Link_ee.

[0094] The equipment position and orientation calculation unit 120 calculates the position and orientation of the sensor in the control environment based on the equipment configuration information and trajectory information acquired in step S101 (step S203). Here, the position and orientation of the sensor in the control environment refers to the position and orientation of the sensor at the start of the operation. The calculation method is the same as in step S202.

[0095] Figure 11B shows an example of the position and orientation of a sensor in a control environment. In Figure 11B, the position and orientation of the sensor are shown relative to the equipment coordinate system 1110. The equipment coordinate system 1110 is an absolute coordinate system of the control environment with an arbitrary point on the equipment as the origin. In Figure 11B, the equipment coordinate system 1110 is an absolute coordinate system with the installation position of the assembly cell frame, which has the mounting target as the Root, as the origin. Coordinate system 1111 is a relative coordinate system with the sensor as the origin.

[0096] The equipment analysis unit 121 calculates the positional deviation and orientation deviation of the sensor based on the position and orientation of the sensor in the learning environment and the position and orientation of the sensor in the control environment (step S204).

[0097] The sensor's positional displacement is calculated as the displacement of the origin of coordinate systems 1101 and 1111. The sensor's attitude displacement is calculated as the rotation angle θ of coordinate systems 1101 and 1111, as shown in Figure 11C.

[0098] The conversion formula generation unit 122 generates a conversion formula (A) based on the positional and orientational misalignments of the sensors between the two environments (step S205). The conversion formula (A) is expressed in terms of a three-dimensional translation to correct the positional misalignment of the sensors and a three-dimensional rotation to correct the orientational misalignment. For example, the rotation with respect to the Z-axis is given by formula (1). The rotation angle θ is an angle representing the orientational misalignment of the sensors.

[0099]

number

[0100] The conversion formula generation unit 122 registers the conversion formula (A) in the conversion formula management information 133 (step S206), and terminates the conversion formula (A) calculation process.

[0101] Figure 12 is a flowchart illustrating an example of the conversion formula (B) calculation process performed by the computer 100 in Example 1. Figures 13A, 13B, 13C, 13D, 13E, and 13F are diagrams showing specific examples of the conversion formula (B) calculation process performed by the computer 100 in Example 1.

[0102] The equipment position and orientation calculation unit 120 calculates the position and orientation of the workpiece in the learning environment based on the equipment configuration information and trajectory information of the learning environment (step S301). Here, the position and orientation of the workpiece in the learning environment refers to the position and orientation of the workpiece at the start of the operation. It is assumed that the equipment configuration information and trajectory information during model learning are obtained in the conversion formula (A) calculation process.

[0103] Figure 13A shows an example of the position and orientation of a workpiece in a learning environment. Figure 13A shows the position and orientation of the workpiece relative to the sensor.

[0104] The equipment position and orientation calculation unit 120 calculates the position and orientation of the sensor by the same process as in step S202. Hand B is attached to the force sensor. The equipment position and orientation calculation unit 120 calculates the position and orientation of hand B based on the position and orientation of the force sensor and the relative position and orientation of hand B with respect to the force sensor. Workpiece B is gripped and moved by hand B. The equipment position and orientation calculation unit 120 calculates the position and orientation of workpiece B based on the position and orientation of hand B and the relative position and orientation of workpiece B with respect to hand B.

[0105] The equipment position and orientation calculation unit 120 calculates the position and orientation of the workpiece in the control environment based on the equipment configuration information and trajectory information acquired in step S101 (step S302). Here, the position and orientation of the workpiece in the control environment refers to the position and orientation of the workpiece at the start of the operation. The calculation method is the same as in step S301.

[0106] Figure 13B shows an example of the workpiece position and orientation in a control environment. Figure 13B shows the workpiece position and orientation relative to the sensor.

[0107] The equipment analysis unit 121 obtains the parameters of each workpiece in the learning environment and control environment from the workpiece management information 134 (step S303).

[0108] The conversion formula generation unit 122 calculates the difference in force parameters of the mechanical system for each environment based on the position and orientation of the sensors in each environment, the position and orientation of the workpiece, and the parameters of the workpiece (step S304). The method for calculating the difference in force parameters of the mechanical system for each environment will now be explained.

[0109] (Example 1) As shown in Figures 13C and 13D, the method for generating the conversion formula (B) when parameters a, b, c, d, e, f, and g related to the positioning pin are obtained will be explained. Parameters a, b, c, and d represent the distance from the edge of the workpiece in the learning environment to the positioning pin, and parameters e, f, and g represent the distance from the edge of the workpiece in the control environment to the positioning pin. The direction of rotation with respect to the central axis 1300 (1301) of the workpiece indicates the direction of the moment on the x-axis (x' axis) of the sensor coordinate system 1101 (1111). When the hand contacts the workpiece at the positioning pin, the direction and magnitude of the moment become force parameters. The ratio of the distance from the edge of the workpiece to the positioning pin, which is the fulcrum of the lever principle, becomes the magnitude of the moment of the workpiece. Therefore, in Figure 13C, the magnitude of the moment on the x-axis of coordinate system 1101 is b / a, and in Figure 13D, the magnitude of the moment on the x'-axis of coordinate system 1111 is g / h.

[0110] The conversion formula generation unit 122 calculates the magnitude and direction of the moment on the x-axis (x'-axis) for each environment, and calculates the difference in the magnitude and direction of the moment on the x-axis (x'-axis) for each environment as the change in the force parameter of the dynamical system for each environment. The same calculation is performed for the y-axis (y'-axis).

[0111] (Example 2) As shown in Figures 13E and 13F, the method for generating the conversion formula (B) when parameters a, b, c, and d related to the moment of inertia are obtained will be explained. Parameters a and b represent the length and width of the workpiece in the learning environment, and parameters c and d represent the length and width of the workpiece in the control environment. The direction of rotation with respect to the central axis 1300 (1301) of the workpiece indicates the direction of the moment on the x-axis (x' axis) of the sensor coordinate system 1101 (1111). For workpieces without positioning pins, the reciprocal of the moment of inertia represents the magnitude of the moment of the workpiece. In the case of a thin plate part as shown in Figure 13E, the moment of inertia on the x-axis is given by formula (2). e is the weight of the workpiece. In the case of Figure 13F, the moment of inertia on the x' axis is given by equation (3). f This is the weight of the workpiece.

[0112]

number

[0113]

number

[0114] The conversion formula generation unit 122 calculates the moment of inertia on the x-axis (x'-axis) for each environment, and calculates the ratio of the reciprocals of the moments of inertia on the x-axis (x'-axis) for each environment as the amount of change in the force parameters of the dynamical system for each environment. The same calculation is performed for the y-axis (y'-axis).

[0115] The above describes the method for calculating the difference in force parameters of the mechanical system in each environment. Note that the method described above is just one example and is not limited to it. The direction and magnitude of torsion may also be treated as force parameters.

[0116] The conversion equation generation unit 122 generates a conversion equation (B) based on the difference in force parameters of the dynamical systems in each environment (step S305). When the positioning pin parameters are used, a conversion equation (B) as shown in equation (4) is generated. When the moment of inertia parameters are used, a conversion equation (B) as shown in equation (5) is generated.

[0117]

number

[0118]

number

[0119] The conversion formula generation unit 122 registers the conversion formula (B) in the conversion formula management information 133 (step S306), and terminates the conversion formula (B) calculation process.

[0120] By using transformation equation (B), the dynamical systems of the two environments can be aligned. This allows the model to be used directly.

[0121] According to the present invention, measurement data can be transformed to eliminate differences in the dynamic systems of work between different environments. This makes it possible to achieve automatic control of robot work using existing models.

[0122] Note that conversion formula (A) is not a mandatory component. In this case, only conversion formula (B) is used.

[0123] It should be noted that the present invention is not limited to the embodiments described above, and various modifications are included. Furthermore, for example, the embodiments described above are detailed explanations of the configuration in order to clearly illustrate the present invention, and are not necessarily limited to those having all the configurations described. In addition, some of the configurations in each embodiment can be added to, deleted from, or replaced with other configurations.

[0124] Furthermore, each of the above-mentioned configurations, functions, processing units, processing means, etc., may be implemented in hardware, in whole or in part, for example, by designing them as integrated circuits. The present invention can also be implemented by software program code that realizes the functions of the embodiment. In this case, a storage medium on which the program code is recorded is provided to a computer, and the processor of that computer reads the program code stored in the storage medium. In this case, the program code read from the storage medium itself realizes the functions of the embodiment described above, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media used to supply such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, and the like.

[0125] Furthermore, the program code that implements the functions described in this embodiment can be implemented in a wide range of programming or scripting languages, such as assembler, C / C++, Perl, Shell, PHP, Python, and Java (registered trademark).

[0126] Furthermore, the program code for the software that implements the functions of the embodiment may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the computer's processor may read and execute the program code stored in the storage means or storage medium.

[0127] In the above-described embodiment, the control lines and information lines shown are those deemed necessary for explanation and do not necessarily represent all control lines and information lines in the actual product. All components may be interconnected. [Explanation of symbols]

[0128] 100 calculator 101 Robots 110 Arithmetic equipment 111 Storage device 112 Communication equipment 113 Input device 114 Output device 120 Equipment position / orientation calculation section 121 Equipment Analysis Department 122 Conversion Formula Generation Unit 123 Measurement Data Conversion Unit 124 Correction Amount Determination Unit 125 Control Value Calculation Unit 130 Equipment management information 131 Orbit management information 132 Model Management Information 133 Conversion Formula Management Information 134 Work Management Information 140 Measuring devices 141 Work equipment group 142 controllers 800, 900 screens

Claims

1. A computer system that controls a robot performing tasks including gripping and moving a workpiece using a model generated by machine learning, A computer comprising a computing device, a storage device connected to the computing device, and a network interface connected to the computing device, The system manages trajectory information, including control values ​​for controlling the robot's trajectory during the work, in both the learning environment and the control environment where the actual work is performed. Based on the difference between a first dynamical system representing the work in the learning environment and a second dynamical system representing the work in the control environment, a first transformation formula is generated for transforming the measurement data acquired by sensors installed on the robot so that the second dynamical system matches the first dynamical system. A computer system characterized by acquiring the aforementioned measurement data and operating status information indicating the operating status of the robot, converting the measurement data using the first conversion formula, inputting the trajectory information, the operating status information, and the converted measurement data into the model, and controlling the robot based on the output of the model.

2. A computer system according to claim 1, The learning environment and the control environment each manage equipment configuration information and work information related to the equipment and work, respectively. Using the trajectory information, the equipment configuration information, and the workpiece information, the first force parameter of the first dynamic system and the second force parameter of the second dynamic system are calculated. A computer system characterized by generating the first transformation formula based on the difference between the first force sensing parameter and the second force sensing parameter.

3. A computer system according to claim 2, A computer system characterized in that the first force-sensing parameter and the second force-sensing parameter are parameters related to the force and moment acting on the workpiece.

4. A computer system according to claim 1, Based on the position and orientation of the sensor in the learning environment and the position and orientation of the sensor in the control environment, a second transformation formula is generated to make the coordinate system of the sensor in the learning environment and the coordinate system of the sensor in the control environment consistent. A computer system characterized by converting the measurement data using the first conversion formula and the second conversion formula.

5. A method for controlling a robot that performs tasks including gripping and moving a workpiece, using a machine learning-generated model executed by a computer system, The aforementioned computer system, A computer comprising a computing device, a storage device connected to the computing device, and a network interface connected to the computing device, The system manages trajectory information, including control values ​​for controlling the robot's trajectory during the work, in both the learning environment and the control environment where the actual work is performed. The robot control method is as follows: The first step is for the computer to generate a first transformation formula for transforming measurement data acquired by sensors installed on the robot so that the second dynamic system matches the first dynamic system, based on the difference between a first dynamic system representing the work in the learning environment and a second dynamic system representing the work in the control environment. A robot control method characterized by comprising: a second step in which, when the computer acquires the measurement data and operating status information indicating the operating status of the robot, the computer converts the measurement data using the first conversion formula, inputs the trajectory information, the operating status information, and the converted measurement data into the model, and controls the robot based on the output of the model.

6. A robot control method according to claim 5, The aforementioned computer system manages equipment configuration information and work information for the learning environment and the control environment, respectively. The first step described above is: The steps include: the computer calculating a first force parameter of the first dynamic system and a second force parameter of the second dynamic system using the trajectory information, the equipment configuration information, and the workpiece information; A robot control method characterized by comprising the step of the computer generating the first transformation formula based on the difference between the first force sensing parameter and the second force sensing parameter.

7. A robot control method according to claim 6, A robot control method characterized in that the first force sensing parameter and the second force sensing parameter are parameters related to the force and moment acting on the workpiece.

8. A robot control method according to claim 5, The computer includes the step of generating a second transformation formula that matches the coordinate system of the sensor in the learning environment with the coordinate system of the sensor in the control environment, based on the position and orientation of the sensor in the learning environment and the position and orientation of the sensor in the control environment. The second step is a robot control method characterized in that the computer converts the measurement data using the first conversion formula and the second conversion formula.

Images