Information processing apparatus, method for controlling information processing apparatus, and storage medium

Abstract

To provide an information processing device and the like capable of performing simulation in consideration of an object existing in a work space of a robot while reducing a load of arithmetic processing.SOLUTION: The information processing device includes a storage unit that stores robot configuration information including a shape, a connection relationship, and a movable range of each arm constituting an articulated multi-axis robot, a reception unit that receives work space information regarding a shape of a workpiece to be subjected to teaching and a work space of the robot, and a work requirement set in advance for the workpiece, a generation unit that generates a three dimensional model of the work space including the robot and the workpiece in a virtual space based on the robot configuration information and the work space information, and a simulation unit that outputs a simulation result in the virtual space as control information of the robot in a real space. The generation unit voxelizes and generates a three dimensional object of an object present in a work space of a robot in a real space.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a control method for an information processing device, and a control program for an information processing device. [Background technology]

[0002] Traditionally, offline teaching of robots at production and manufacturing sites has been a time-consuming and labor-intensive process for operators. However, in recent years, progress has been made in the development of simulation devices capable of simulating robot movements in 3D. Using a simulation device, multiple robot movements relative to a workpiece can be simulated in a short amount of time, enabling machine learning for offline teaching.

[0003] For example, Patent Document 1 discloses that shape information of a robot, a workpiece, and peripheral equipment is used to construct a three-dimensional robot model, a workpiece model, and a peripheral equipment model, and the robot model is operated in a virtual space to simulate interference check between the robot, the workpiece, and the peripheral equipment. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-274148 Summary of the Invention [Problem to be solved by the invention]

[0005] In the simulation device described in Patent Document 1, a 3D peripheral device model is constructed based on shape information of the peripheral device. In other words, the simulation uses a 3D representation of the actual peripheral device. However, using 3D models of objects that are not directly related to the robot's operation, such as obstacles in the robot's workspace or adjacent robots, can place a burden on the simulation.

[0006] The present invention has been made in view of the above problems, and aims to provide an information processing device and the like that reduces the calculation load in a simulation that takes into account objects present in the workspace of a robot. [Means for solving the problem]

[0007] An information processing device according to one aspect of the present invention comprises: a memory unit that stores robot configuration information including the shape of each arm that constitutes a multi-joint, multi-axis robot, the connection relationships between the arms, and the range of motion of each arm; a reception unit that receives workspace information including the shape of a workpiece to be taught and information about the workspace of the robot that processes the workpiece, as well as pre-set work requirements for the workpiece; a generation unit that generates a three-dimensional model of the workspace including the robot and workpiece in a virtual space based on the robot configuration information and the workspace information; and a simulation unit that outputs the results of a simulation of the robot's operations in the virtual space that satisfy the work requirements as control information for the robot in real space, and the generation unit voxels and generates a three-dimensional object of an object that exists in the robot's workspace in real space.

[0008] An information processing device according to one aspect of the present invention may further include a learning unit that generates a learning model that is learned by simulating it in a virtual space using information about past work performed by a robot on a specified workpiece in real space.

[0009] In an information processing device according to one aspect of the present invention, the simulation unit may use a learning model to acquire control information that satisfies predetermined work requirements for the workpiece to be taught.

[0010] In an information processing device according to one aspect of the present invention, the learning unit may generate a learning model by machine learning using at least the shape of a predetermined workpiece, work requirements previously set for the predetermined workpiece, and past work spaces as input data, and the work performed by a robot on the predetermined workpiece as output data.

[0011] In the information processing device according to one aspect of the present invention, the robot may be a painting robot that paints a workpiece.

[0012] The information processing device according to an aspect of the present invention may further include a setting unit that sets the connection relationships between the arms that make up the robot.

[0013] A control method of an information processing device according to one aspect of the present invention includes the steps of: storing robot configuration information, including the shape of each arm constituting a multi-joint, multi-axis robot, the connection relationships between the arms, and the range of motion of each arm; receiving workspace information, including information on the shape of a workpiece to be taught and the workspace of the robot processing the workpiece, as well as pre-set work requirements for the workpiece; generating a three-dimensional model of the workspace including the robot and workpiece in a virtual space based on the robot configuration information and the workspace information; and outputting the results of a simulation of the robot's operation in the virtual space that satisfies the work requirements as control information for the robot in real space; the generating step generates a three-dimensional object of an object existing in the robot's workspace in real space by voxelizing it.

[0014] A control program for an information processing device according to one embodiment of the present invention enables the information processing device to perform the following functions: store robot configuration information including the shape of each arm constituting a multi-joint, multi-axis robot, the connection relationships between the arms, and the range of motion of each arm; receive workspace information including the shape of a workpiece to be taught and information about the workspace of the robot that processes the workpiece, as well as pre-set work requirements for the workpiece; generate a three-dimensional model of the workspace including the robot and workpiece in a virtual space based on the robot configuration information and workspace information; and output the results of a simulation of the robot's operations in the virtual space that satisfy the work requirements as control information for the robot in real space. The generating function generates a three-dimensional object of an object existing in the robot's workspace in real space by voxelizing it. [Effects of the Invention]

[0015] An information processing device according to one aspect of the present invention includes: a memory unit that stores robot configuration information, including the shapes of each arm constituting a multi-joint, multi-axis robot, the connection relationships between the arms, and the range of motion of each arm; a reception unit that receives workspace information, including information about the shape of a workpiece to be taught and information about the workspace of the robot processing the workpiece, as well as preset task requirements for the workpiece; a generation unit that generates a 3D model of the workspace including the robot and the workpiece in a virtual space based on the robot configuration information and the workspace information; and a simulation unit that outputs the results of a simulation of the robot's operations in the virtual space that satisfy the task requirements as control information for the robot in real space. The generation unit voxels the 3D objects of objects present in the robot's workspace in real space to generate them. This makes it possible to provide an information processing device and the like that reduces the computational load in simulations that take into account objects present in the robot's workspace. [Brief explanation of the drawings]

[0016] [Figure 1]FIG. 1 is a schematic configuration diagram of a simulation system according to one embodiment of the present invention and a block diagram of an information processing device. [Figure 2] Figure 2 shows an example of a screen used to set the connection relationships between the components of a robot. [Figure 3] FIG. 3 is a control flow of an information processing device according to an embodiment of the present invention. [Figure 4] FIG. 4 is a diagram illustrating an outline of a simulation system according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0017] An information processing device according to one embodiment of the present invention will be described in detail below with reference to the drawings. Note that the drawings are merely examples, and the information processing device of the present invention is not limited to those shown. Furthermore, the drawings are schematic, and the size ratios of the components of the robot, the number of workpieces, the workspace, and objects present around the robot, and their positional relationships are not strictly accurate.

[0018] <Embodiment> <System configuration> Fig. 1 is a schematic diagram of a simulation system including an information processing device 100 according to one embodiment of the present invention. The simulation system 500 includes the information processing device 100, an articulated, multi-axis robot 10, and a robot control device 200. The robot 10 is a painting robot that drives each arm constituting the robot under the control of the robot control device 200 to paint a workpiece 12. Note that Fig. 1 shows a painting robot with a spray gun attached to its tip as an example, but the robot 10 is not limited to this and may be a welding robot, a transport robot, a pick-up robot, or the like.

[0019] The information processing device 100 reproduces the robot 10, workpiece (work) 12, and object 14 contained in the robot 10's workspace in real space as three-dimensional models in a virtual space, and simulates the operation of the robot 10. Here, the object 14 is an object present around the robot 10, and may be another workpiece, another robot, control equipment, or the like. The virtual space in which the simulation is performed is displayed on a display unit 141. The display unit 141 is a monitor having a function of displaying various information output from the information processing device 100, and can be realized, for example, by an LCD (Liquid Crystal Display), an organic EL display, or the like. Note that the display unit 141 may also be, for example, a monitor of a mobile terminal or a tablet terminal. Note that, in one embodiment of the present invention, the display unit 141 is not essential for performing machine learning (reinforcement learning), which will be described later. An operator can input instructions to the information processing device 100 using an input device such as a keyboard 131 or a mouse (not shown).

[0020] The robot control device 200 drives the robot 10 based on robot control information output by a simulation in the information processing device 100. The control information may include, for example, values ​​such as the target angle and angular velocity of each axis (joint) constituting the robot 10, and the torque and drive power of the drive motor. Note that while FIG. 1 shows a configuration in which the robot control device 200 is connected to the information processing device 100, the present invention is not limited to this. In other words, the robot control device 200 may exist separately from the information processing device 100.

[0021] In the simulation system 500, the information processing device 100 reduces the load in simulating a plurality of movements of the robot 10 and provides an environment for efficient machine learning.

[0022] <Hardware configuration> The information processing device 100 includes a control unit 110, a communication I / F (interface) 120, an input I / F 130, an output I / F 140, and a storage unit 170 as hardware components.

[0023] The control unit 110 is typically a processor, and includes a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), etc., and may be realized by a logic circuit (hardware) formed in an integrated circuit (an integrated circuit (IC) chip, a large scale integration (LSI)), etc., or a dedicated circuit. The control unit 110 may execute the functions and methods described in each embodiment by reading a program stored in the storage unit 170 and executing code or instructions included in the read program.

[0024] The storage unit 170 stores various programs and various data required for the operation of the information processing device 100. The storage unit 170 may include, for example, a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. The storage unit 170 may also include memory (random access memory (RAM), read only memory (ROM), etc.) that provides a working area for the control unit 110. The information processing device 100 according to this embodiment functions as a reception unit 112, a generation unit 113, a simulation unit 114, a setting unit 115, and a learning unit 116 (described later) when the control unit 110 executes a program loaded into the memory of the storage unit 170. For example, the storage unit 170 may store motion analysis software for simulating the movement of the robot 10 in a virtual space.

[0025] The storage unit 170 stores robot configuration information including the shape of each arm constituting the robot 10, the connection relationships between the arms, and the range of motion of each arm. The robot configuration information may include the length, width, and length of the arms constituting the robot, the distance between joints (effective length), the connection relationships between the arms, and identification information of the connecting joints. For example, the robot configuration information may be extracted from CAD (Computer Aided Design) data of the robot 10. Furthermore, the robot configuration information may include soft limit values, the rotation direction of each joint, and the range of rotation angles as information indicating the range of motion of each arm. Note that this data may be received and stored by the receiving unit 112 (described later) via the input device (keyboard) 131 input by an operator. Although FIG. 1 illustrates the storage unit 170 as being integrated with the information processing device 100, the data stored in the storage unit 170 may be stored in a database server separate and independent from the information processing device 100.

[0026] The communication I / F 120 is implemented as hardware such as a network adapter, communication software, or a combination of these, and transmits and receives various data to and from an external device such as the robot control device 200, for example.

[0027] The input I / F 130 is a connection interface with an input device that inputs various operations to the information processing device 100. The input device includes, for example, a touch panel, hardware keys such as a keyboard 131, and a pointing device such as a mouse.

[0028] The output I / F 140 is a connection interface with an output device that outputs processing results such as simulation results by the information processing device 100. The output device includes, for example, a display unit (display) 141, a speaker, and the like.

[0029] <Functional configuration> The control unit 110 includes a receiving unit 112, a generating unit 113, a simulation unit 114, a setting unit 115, and a learning unit .

[0030] The receiving unit 112 receives workspace information, including information about the shape of the workpiece 12 to be taught and information about the workspace 300 of the robot 10 that processes the workpiece 12, as well as task requirements previously set for the workpiece 12. The workspace information and task requirements may be input by an operator via the keyboard 131. The workpiece 12 is an object that the robot 10 works on, and may be, for example, a part of a vehicle or other workpiece. The receiving unit 112 may receive information about the shape of the workpiece 12, such as the length, width, and height of the workpiece 12. Furthermore, the information about the workspace 300 may include information about an object 14 that exists in the workspace 300 of the robot 10 in real space. The object 14 is an object that exists in a position that may interfere with the robot 10's work, such as another workpiece, peripheral equipment, or another adjacent robot. The receiving unit 112 may receive information about the length, width, height, placement position, etc. of the object 14. The receiving unit 112 may acquire the above-mentioned workspace information from a design drawing (CAD data, etc.) of the workpiece 12 or the object 14, or from a moving image of the workpiece 12 or the object 14.

[0031] The work requirement is information indicating the work to be performed on the workpiece 12, and may be, for example, the locations to be painted on the workpiece 12 and the painting route (painting line) connecting the locations to be painted.

[0032] The generation unit 113 generates a 3D model of the workspace 300 including the robot 10 and workpiece 12 in the virtual space 400 based on the robot configuration information and workspace information. Hereinafter, the 3D models of the robot 10, workpiece 12, and object 14 will also be referred to as the robot model, workpiece model, and object model, respectively. Fig. 1 shows the display unit 141 on which the virtual space 400 including the robot model 20, workpiece model 22, and object model 24 is displayed.

[0033] Here, the generation unit 113 generates a 3D model by voxelizing (discretizing) the object model 24. Voxelization is a representation format in which the 3D model is divided into cubic regions of a predetermined size and information is reduced. Note that existing technologies may be used for voxelizing the object model 24 and generating the 3D model.

[0034] The simulation unit 114 uses a three-dimensional model of the robot 10 in the virtual space 400 to simulate an operation that satisfies the task requirements, and outputs the result as control information for the robot 10 in the real space 300. Specifically, the simulation unit 114 may output how each arm of the robot model 20 is driven when the robot model 20 is made to perform an operation that satisfies the task requirements.

[0035] The setting unit 115 sets the connection relationships between the arms that make up the robot 10. Fig. 2 shows an example of a setting screen for setting the connection relationships between the arms. The setting screen 40 is displayed on the display unit 141, and the operator can define the connection relationships between the arms on the setting screen 40.

[0036] The learning unit uses information about past work performed by the robot 10 on a predetermined workpiece in the real space to generate a learning model learned by simulating in the virtual space 400. The learning unit will be described later.

[0037] <Control flow> The above-mentioned processing by the information processing device 100 will be explained using the flowchart of FIG. (Step S11) The storage unit 170 stores robot configuration information including the shape of each arm constituting the robot 10, the connection relationship between the arms, and the range of motion of each arm.

[0038] (Step S12) The receiving unit 112 receives workspace information including information about the shape of the workpiece 12 to be taught and the workspace 300 of the robot 10 that processes the workpiece 12, as well as work requirements that have been set in advance for the workpiece 12.

[0039] (Step S13) When generating a three-dimensional model of the workspace including the robot and the workpiece in the virtual space 400 based on the robot configuration information and workspace information, the generation unit 113 generates a three-dimensional object of the object 14 that exists in the workspace 300 of the robot 10 in the real space by voxelizing it.

[0040] (Step S14) The simulation unit 114 outputs the results of simulating the operation of the robot 10 that satisfies the task requirements in the virtual space 400 as control information for the robot 10 in the real space.

[0041] As described above, one aspect of the present invention enables simulation using 3D models of a robot, a workpiece, and objects that exist as obstacles in a virtual space that reproduces a real-world workspace. In this case, the objects are converted into voxels, thereby reducing the computational load.

[0042] <Other embodiments> Here, the learning unit 116 generates a learning model that is learned by simulating in the virtual space 400 using information about past work performed by the robot 10 on a predetermined workpiece in the real space. More specifically, the learning unit 116 may perform learning by referring to a program by offline teaching that was used for the work in the real space. In this case, the learning unit 116 may generate the learning model by machine learning using at least the shape of the predetermined workpiece in the past work, the work requirements previously set for the predetermined workpiece, and the past work space as input data, and the work performed by the robot 10 on the predetermined workpiece as output data.

[0043] Learning by the information processing device 100 according to one aspect of the present invention will be described with reference to FIG. 4. The storage unit (database) 170 stores information about work previously performed by the robot 10. The database 170 may include, for example, the painting posture, entry / exit posture, intermediate posture, arm type and its soft limit value, vehicle type as the workpiece, the work environment including objects present in the work space, and the painting route of the robot 10. Note that a separate database 170 may exist for each robot. Specifically, the learning unit 116 uses the information contained in the database 170 to learn a model for generating an optimal painting route. The display unit 141 displays the three-dimensional environment (virtual space) constructed by the generation unit 113 of the information processing device 100 and a simulation performed by the simulation unit 114, in which the arm of the robot model 20 is operated (steps T11 and T12). The learning unit 116 performs learning to generate an optimal painting route using a reinforcement learning algorithm, such as Q-learning, ε-greedy algorithm, or policy gradient method (step T13). The generated learning model may be stored in the database 171, and control data for controlling the robot 10 may be output (step T14).

[0044] While the present invention has been described based on the drawings and examples, it should be noted that those skilled in the art would readily be able to make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of the present invention. For example, the functions included in each component, step, etc. may be rearranged so as not to cause logical inconsistencies, and multiple components, steps, etc. may be combined or separated into one. Furthermore, the configurations described in the above embodiments may be appropriately combined. For example, each component described as being included in the information processing device 100 may be realized in a distributed manner across multiple information processing devices.

[0045] Although a painting robot has been described above as an example of a robot, the present invention is not limited to painting robots and may be applied to other robots as long as they operate to perform some task. For example, the robot may be a welding robot or a transport robot that grasps a workpiece and moves it to a predetermined location.

[0046] Furthermore, in the above description, the method by which the information processing device automatically executes processing is described as the processors functioning as each functional unit constituting the information processing device 100 executing a robot simulation program (processing program) or the like to execute predetermined processing, but this may also be realized by a logic circuit (hardware) or dedicated circuit formed in an integrated circuit (IC (Integrated Circuit) chip, LSI (Large Scale Integration)) or the like in the device.

[0047] The processing program may be recorded on a processor-readable recording medium, and the recording medium may be a "non-transitory tangible medium" such as a tape, disk, card, semiconductor memory, or programmable logic circuit. The processing program may also be supplied to the processor via any transmission medium capable of transmitting the processing program (such as a communications network or broadcast waves). The present invention may also be realized in the form of a data signal embedded in a carrier wave, in which the processing program is embodied by electronic transmission.

[0048] The processing program can be implemented using, for example, a scripting language such as ActionScript or JavaScript (registered trademark), an object-oriented programming language such as Python (registered trademark), Objective-C or Java (registered trademark), or a markup language such as HTML5.

[0049] Although the present invention has been described based on the drawings and examples, it should be noted that various modifications and alterations can be easily made by those skilled in the art based on the present disclosure, and therefore, it should be noted that these modifications and alterations are included within the scope of the present invention. [Explanation of symbols]

[0050] 10. Robot 12 Workpiece 14 Object 20 Robot Models 22 Workpiece model 24 Object Models 40 Settings screen 100 Information processing device 110 control section 112 Reception 113 Generation part 114 Simulation Department 115 Setting section 116 Learning Department 120 Communication I / F (Interface) 130 Input I / F 131 Keyboard (input section) 140 Output I / F 141 Display section 170 Storage section 200 Robot control device 300 workspace 400 Virtual Space 500 Simulation System

Claims

1. a storage unit that stores robot configuration information including the shape of each arm that constitutes a multi-joint, multi-axis robot, the connection relationship between the arms, and the range of motion of each arm; a receiving unit that receives workspace information including information about the shape of a workpiece to be taught and a workspace of the robot that processes the workpiece, as well as preset work requirements for the workpiece; a generation unit that generates a three-dimensional model of the workspace including the robot and the workpiece in a virtual space based on the robot configuration information and the workspace information; a simulation unit that outputs a result of simulating an operation of the robot that satisfies the task requirements in the virtual space as control information for the robot in the real space; Equipped with The generation unit generates a three-dimensional object of an object existing in a workspace of the robot in real space by voxelizing the object. Information processing device.

2. a learning unit that generates a learning model by simulating in the virtual space using information on past work performed by the robot on a predetermined workpiece in the real space; The information processing device according to claim 1 .

3. the simulation unit uses the learning model to obtain control information that satisfies the work requirements that are preset for the workpiece on which the teaching is performed. The information processing device according to claim 2 .

4. the learning unit generates the learning model by machine learning using at least the shape of the predetermined workpiece, the work requirements preset for the predetermined workpiece, and the past work space as input data, and the work performed by the robot on the predetermined workpiece as output data. The information processing device according to claim 2 .

5. The robot is a painting robot that paints the workpiece. The information processing device according to claim 1 .

6. a setting unit that sets a connection relationship between the arms that configure the robot, The information processing device according to claim 1 .

7. The information processing device Storing robot configuration information including the shape of each arm constituting the articulated, multi-axis robot, the connection relationship between the arms, and the range of motion of each arm; receiving workspace information including information about a shape of a workpiece to be taught and a workspace of the robot for processing the workpiece, as well as predetermined work requirements for the workpiece; generating a three-dimensional model of the workspace including the robot and the workpiece in a virtual space based on the robot configuration information and the workspace information; outputting a result of simulating an operation of the robot that satisfies the task requirements in the virtual space as control information for the robot in the real space; Run the generating step generates a three-dimensional object of an object existing in a workspace of the robot in real space by voxelizing the object; A method for controlling an information processing device.

8. In the information processing device, a function of storing robot configuration information including the shape of each arm constituting a multi-joint, multi-axis robot, the connection relationship between the arms, and the range of motion of each arm; a function of receiving workspace information including information about the shape of a workpiece to be taught and the workspace of the robot that processes the workpiece, as well as pre-set work requirements for the workpiece; a function of generating a three-dimensional model of the workspace including the robot and the workpiece in a virtual space based on the robot configuration information and the workspace information; a function of outputting a result of simulating, in the virtual space, an operation of the robot that satisfies the task requirements as control information for the robot in the real space; Realize this, The generating function generates a three-dimensional object of an object existing in the workspace of the robot in real space by voxelizing it. A control program for an information processing device.

Images