Robot simulator

Abstract

A robot simulation device capable of facilitating parameter setting of force control is provided. A simulation device (50) simulates a force control action performed while a tool part mounted on a robot manipulator is brought into contact with a target workpiece. The simulation device (50) comprises: a storage unit (51) storing an action program and force control parameters as setting parameters related to the force control action; and a force control simulation execution unit (52) executing a simulation of the force control action based on the action program and the force control parameters. The force control simulation execution unit (52) comprises a virtual force generation unit (54) generating a virtual force that the tool part receives from the target workpiece when the tool part is in contact with the target workpiece based on position information of the tool part obtained from a simulation result of the force control action. The force control simulation execution unit (52) executes the simulation of the force control action based on the virtual force and a target force set as the force control parameter.

Description

Technical Field

[0001] The present invention relates to a robot simulation device for simulating the actions of a robot. Background Art

[0002] Various simulation devices are provided for simulating the movements of industrial robots. For example, Patent Document 1 describes the following: "A simulation device simulates a virtual robot obtained by virtualizing a robot, the simulation device comprising: a receiving unit that receives a command instructing a virtual holding unit of the virtual robot to at least hold or release a virtual object; and a control unit that, based on the command received by the receiving unit, causes the virtual holding unit to at least hold or release the virtual object." (Patent Document 1, Claim 1).

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2019-63879 Summary of the Invention

[0006] Problems to be solved by the invention

[0007] A robot manipulator is equipped with a force sensor, which allows it to detect the force applied to a workpiece and control the force while performing advanced tasks such as exploration, embedding, and grinding. However, proper force control requires skilled parameter adjustment. Generally, such parameter adjustments require operators to repeatedly fail / succeed in force control to master the technical secrets of parameter setting. A robot simulator is desired that can facilitate the parameter setting of force control.

[0008] Solutions for solving problems

[0009] One embodiment of the present disclosure is a robot simulation device, which is a simulation device that simulates a force control action performed while a tool part mounted on a robot manipulator is brought into contact with an object workpiece, the simulation device comprising: a storage unit that stores an action program and force control parameters as setting parameters related to the force control action; and a force control simulation execution unit that performs a simulation of the force control action based on the action program and the force control parameters, wherein the force control simulation execution unit comprises a virtual force generation unit that generates a virtual force that the tool part receives from the object workpiece when the tool part is in contact with the object workpiece based on position information of the tool part obtained according to a simulation result of the force control action, and the force control simulation execution unit performs a simulation of the force control action based on the virtual force and a target force set as the force control parameter.

[0010] Effects of the Invention

[0011] According to the above configuration, the operator can intuitively grasp the state of generation of the pressing force, and can facilitate parameter setting of the force control.

[0012] These and other objects, features, and advantages of the present invention will become more apparent from the detailed description of typical embodiments of the present invention shown in the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a system configuration diagram of a robot system including a control device as a simulation device according to one embodiment.

[0014] Figure 2 A configuration example of a robot system is shown.

[0015] Figure 3 Another configuration example of the robot system is shown.

[0016] Figure 4 This is a functional block diagram of the control device, external computer, and display device.

[0017] Figure 5 This is a block diagram of the force control in the robot motion control unit.

[0018] Figure 6 is a diagram illustrating the exploration task.

[0019] Figure 7 This is a flowchart showing the simulation operation of the exploration task.

[0020] Figure 8A This is a diagram explaining the operating states of the tool part in the exploration operation and the embedding operation.

[0021] Figure 8BThis is a diagram explaining the operating states of the tool part in the exploration operation and the embedding operation.

[0022] Figure 8C This is a diagram explaining the operating states of the tool part in the exploration operation and the embedding operation.

[0023] Figure 8D This is a diagram explaining the operating states of the tool part in the exploration operation and the embedding operation.

[0024] Figure 9 is a diagram illustrating an embedding operation.

[0025] Figure 10 This is a flowchart showing the simulation operation of the embedding operation.

[0026] Figure 11 It is a diagram showing an example of display of virtual force.

[0027] Figure 12 This is a diagram showing another example of displaying virtual force.

[0028] Figure 13 Shown in Figure 12 The display screen also displays a status of a message image for suggesting adjustment of the control parameters.

[0029] Figure 14 Shown in Figure 12 The display screen also displays an image that provides guidance for transitioning to the setting screen.

[0030] Figure 15 FIG. 1 is a diagram showing an example of an interface image that guides setting and inputting force control parameters. DETAILED DESCRIPTION

[0031] Next, embodiments of the present disclosure will be described with reference to the accompanying drawings. In the accompanying drawings, identical structural components or functional components are denoted by the same reference numerals. For ease of understanding, the scales in these drawings have been appropriately altered. The embodiments shown in the accompanying drawings are merely examples for implementing the present invention, and the present invention is not limited to the illustrated embodiments.

[0032] Figure 1 FIG. 1 is a system configuration diagram of a robot system 100 including a control device 50 of a robot simulation device according to one embodiment. Figure 1As shown, the control device 50 is connected to a robot manipulator 10 (hereinafter referred to as the manipulator 10) equipped with a tool at the front end of the wrist and a force sensor 3 as a force detector for detecting the external force applied to the tool. The force sensor 3 is installed between the front end of the wrist of the manipulator 10 and the tool. Alternatively, the force sensor 3 can also be set on the workpiece. The control device 50 has a force control function, which enables the manipulator 10 to perform various operations such as exploration operations, precision embedding operations, grinding (imitation actions) as advanced operations while detecting the force applied to the workpiece. The control device 50 can also have a structure as a general computer, which has a CPU, ROM, RAM, a storage device, an operating unit, a display unit, an input and output interface, a network interface, etc.

[0033] Furthermore, the control device 50 is connected to an external computer 90 and a display device 70. The external computer 90 is responsible for performing a physical simulation based on the motion model of the manipulator 10 when the control device 50 simulates a force control operation (hereinafter referred to as a force control simulation). The display device 70 displays the results of the force control simulation. In this specification, the term "simulation" encompasses not only the calculation of the position of the manipulator, etc., through numerical simulation, but also the simulation of the shape model of the manipulator, etc., in accordance with teaching data.

[0034] exist Figure 2 and Figure 3 The configuration example of the robot system 100 is shown in FIG. Figure 2 and Figure 3 , only the manipulator 10 (including the force sensor 3 and the tool portion 11 ) and the target workpiece are shown. Figure 2 The configuration example is shown in which a grinder 8 for performing a grinding operation on a workpiece W1 is mounted on the tool unit 11. In the case of such a configuration example, the control device 50 performs the grinding operation on the target workpiece based on force control.

[0035] Figure 3 FIG. 1 shows a configuration example in which a robot 9 for performing embedding work on a workpiece W2 is mounted on a tool unit 11. Figure 3 In the embodiment, a robot 9 for holding a workpiece 7 such as a cylinder is mounted on the tool portion 11. Figure 3 In the configuration example, the control device 50 performs an operation of inserting the workpiece 7 gripped by the robot 9 into a hole formed in the target workpiece W2 through force control.

[0036] The control device 50 also has the function of performing a force control simulation according to the teaching data (action program) and force control parameters provided by the operator, thereby visualizing the magnitude and location of the force applied to the tool portion 11 from the target workpiece (i.e., the pressing force applied to the target workpiece) and displaying it as an AR (augmented reality) image or a VR (virtual reality) image on the display device 70. This allows the operator to understand, for example, the degree of pressing force applied to which part of the target workpiece before performing the actual force control operation, and to adjust the action program and force control parameters. In addition, in this specification, the force control parameters are defined as including at least one of the target pressing force, pressing direction velocity, force control gain, exploration area, velocity gain, and teaching position.

[0037] Figure 4 FIG is a functional block diagram of the control device 50, the external computer 90, and the display device 70. Figure 4 As shown, the control device 50 includes a storage unit 51 that stores various information, a force control simulation execution unit 52 responsible for executing force control simulation, a robot motion control unit 53 that controls the motion of the robot manipulator 10, a virtual force generator (virtual force generation unit) 54, a virtual force learning unit 55, and an adjustment content generation unit 56. The storage unit 51 stores the motion program of the manipulator 10, 3D model data of the manipulator 10, the tool, and the workpiece, force control parameters, and other various data used to control the manipulator 10. The virtual force generator 54 generates a virtual force that the tool 11 receives from the target workpiece when the tool 11 is in contact with the target workpiece based on the position information of the tool 11 obtained from the force control simulation results. In this specification, a force that is virtually calculated as a force acting on an object may be referred to as a virtual force, and in the case of a pressing force, it may also be referred to as a virtual pressing force.

[0038] The external computer 90 includes a physical simulation unit 91 that executes a physical simulation of the manipulator 10 based on a motion model (motion equation) of the manipulator 10 .

[0039] In this embodiment, the display device 70 is configured as a head-mounted display. In addition, the display device 70 can also be configured by other information processing devices such as a tablet terminal equipped with a camera. The operator wears the display device 70 configured as a head-mounted display. The display device 70 includes a camera 71, an AR / VR image processing unit 72 that performs image processing for displaying augmented reality (AR) images or virtual reality (VR) images, a display 73, and a sound output unit 74. The camera 71 is arranged on the display device 70 in a manner such that the optical axis of the camera lens faces the front of the wearer, and the camera 71 captures an image of the real workspace including the manipulator 10. The AR / VR image processing unit 72 can use the information of the virtual pressing force acting on the target workpiece and its acting position obtained as a result of the force control simulation to perform augmented reality image processing that superimposes an image representing the virtual pressing force on the real image, or virtual reality image processing that superimposes an image representing the virtual pressing force on an image (moving image animation) of a virtual reality space in which a model of each object such as the manipulator 10 is configured. The display 73 is positioned in front of the wearer's eyes and displays images (videos) generated by the AR / VR image processing unit 72. Furthermore, the display device 70 is provided with an acceleration sensor (gyro sensor) and a position sensor (optical sensor, laser sensor, magnetic sensor) for acquiring the position of the display device 70 within the workspace. This allows the relative positional relationship between the coordinate system (camera coordinate system) fixed to the display device and the world coordinate system fixed to the workspace to be determined.

[0040] Figure 5 This is a block diagram of force control in the robot motion control unit 53. In this embodiment, the direction in which "force control + position control" should be performed (the pressing direction of the workpiece by the tool) and the direction in which only position control should be performed are divided, and the speed (angular velocity) command calculated for the direction in which "force control + position control" should be performed is synthesized with the speed (angular velocity) command calculated for the direction in which only position control should be performed to control the manipulator 10. Figure 5 For the sake of convenience, the position control is omitted here, but the position control is based on the position control side (for example, PD control) generally known in the art, which performs position control by feeding back the position detection value through the position sensor provided on each axis of the manipulator 10. Figure 5In the force control shown, the command speed (angular velocity) is calculated by multiplying the difference between the target force (force + torque) in the pressing direction and the force (torque) acting on the workpiece detected by the force sensor 3 by a control parameter called the force control gain. The force control gain represents the performance of the force control, with the property that the larger the force control gain, the faster the position / posture correction. Force (torque) detection and the calculation of the corresponding speed (angular velocity) command are performed every control cycle. The force control rule (the calculation formula for the speed (angular velocity) command) in this case can be expressed as follows.

[0041] Δx=Kf(F-Fd)

[0042] Where, Kf: force control gain

[0043] Fd: target force (force + torque, force: Fx, Fy, Fz, torque: Mx, My, Mz)

[0044] F: Detected force

[0045] Δx: Target movement amount (speed) per control cycle

[0046] Next, the force control simulation executed under the control of the control device 50 (force control simulation execution unit 52) ​​will be described. In this embodiment, when the operation program is executed, a virtual pressing force acting on the target workpiece is detected or generated using the method described below, thereby realizing the force control simulation.

[0047] (Virtual force generation method 1): Set the motion model (motion equation) of the robot manipulator 10 and execute it through physical simulation Figure 5 The action of the force control block diagram shown in FIG. Based on the position of the tool tip obtained by physical simulation, the virtual pressing force acting on the target workpiece is obtained by using a calculation model. That is, in the case of the virtual force generation method 1, the structure is as follows: Figure 5 As shown, a motion model is set for the manipulator 10, and a virtual pressing force is calculated by the virtual force generator 54. That is, the virtual force generator 54 functions as a force sensor in the force control simulation.

[0048] (Virtual Force Generation Method 2): A virtual force (virtual pressing force) is obtained using log data containing the force (torque) detected by the force sensor 3 and the position information of the robot (manipulator 10) when performing a force-controlled task in the same operating environment as before, or log data obtained by detecting and recording the force (torque) acting on the workpiece while the robot is actually moving relative to the workpiece using an operating program while stopping the tool (e.g., the rotation of a grinding stone). In Virtual Force Generation Method 2, the distance between the tool and the workpiece can be determined based on the taught trajectory. If log data exists that shows the distance between the robot's operating trajectory and the workpiece to be approximately the same, the pressing force recorded in this log data is used as the virtual force (virtual pressing force).

[0049] (Virtual force generation method 3): In actual operations related to a specific workpiece, training data representing the correspondence between the relative position and speed between the robot (tool) and the workpiece and the force (torque) detected by the force sensor is collected, and a learning model is constructed through the learning function to calculate the virtual force (virtual pressing force).

[0050] The virtual force generation method 1 is described in detail. In the virtual force generation method 1, the motion equation (motion model) of the robot manipulator 10 is set, and the virtual force generation method 1 is used to generate the virtual force. Figure 5 The force control block shown is operated to determine the position of the robot manipulator 10 (the position of the tool tip). The equation of motion of the robot manipulator 10 is generally expressed by the following equation.

[0051] [Number 1]

[0052]

[0053] In the above formula, θ represents the angle of each joint, M is the matrix related to the moment of inertia, h is the matrix related to the Coriolis force and centrifugal force, g is the term representing the influence of gravity, τ is the torque, and τ L is the load torque.

[0054] The motion instructions based on the teaching track (in Figure 5 In this example, a command given to the manipulator 10 is used as input data to the equation of motion to calculate the robot's behavior (the position of the tool tip). Based on the tool tip position calculated from the equation of motion, the virtual force (virtual pressing force) F applied to the workpiece when the tool tip contacts the workpiece is calculated. An example calculation of the virtual force F is shown below.

[0055] The first calculation example of the virtual force (virtual pressing force) F is for a case where the rigidity of the workpiece is relatively low relative to the tool. In this example, the virtual force F can be calculated by multiplying the amount by which the tool tip moves toward the workpiece beyond the contact point with the workpiece by a coefficient Kd related to the workpiece rigidity using the following equation: δ.

[0056] F=Kd·δ…(1a)

[0057] In this case, the position of the target workpiece is assumed to be fixed within the workspace. Alternatively, a method can be employed in which the force applied to the workpiece when the tool tip contacts the target workpiece is defined as F, the velocity when the tool tip exceeds the contact position with the workpiece is defined as Vc, and the force F applied to the workpiece is calculated using the following formula.

[0058] F=Kd·δ+Kc·Vc…(1b)

[0059] These coefficients Kd and Kc can be set according to the rigidity, shape, etc. of the target workpiece.

[0060] The second calculation example of the virtual force (virtual pressing force) F is based on the tool's deflection when the tool's rigidity is relatively low relative to the workpiece. The amount δ by which the tool's tip moves toward the workpiece beyond its contact point is considered the tool's deflection. Using the tool's rigidity coefficient (virtual spring coefficient), the virtual force F is calculated using the following formula.

[0061] F = (virtual spring constant of the tool) × δ…(2a)

[0062] In addition, in the case where the tool is a so-called floating tool having a mechanism (spring mechanism) that extends and retracts in the pressing direction, the extension length of the tool tip can be calculated based on the position of the tool tip and the position of the target workpiece, and the virtual force F can be obtained by the following formula.

[0063] F = (spring constant of tool) × extension length (2b)

[0064] The third calculation example of the virtual force (virtual pressing force) F is based on the distance the robot (tool tip) moves in response to a speed command in the pressing direction, when the tool has relatively high rigidity. In this example, the movement position based on the speed command is defined as Tx, and the position of the robot (tool tip) after the actual movement in response to the speed command is defined as d. The virtual force F is calculated using the following formula.

[0065] F=k×(Tx-d)…(3)

[0066] Here, k is a coefficient. The coefficient k may be set as a value obtained by experiment or experience.

[0067] Furthermore, the virtual force F may be calculated by substituting teaching data (teaching position, teaching speed) into the above-mentioned calculation formula instead of the position and speed of the tool tip obtained by physical simulation.

[0068] Next, the virtual force generation method 3 is described in detail. The generation of virtual pressing force based on the virtual force generation method 3 is performed by the virtual force learning unit 55. The virtual force learning unit 55 has the following functions: extracting useful rules, knowledge expressions, judgment criteria, etc. present in the set of input data through analysis and outputting its judgment results, and performing knowledge learning (machine learning). There are various methods of machine learning, but if roughly divided, they can be divided into "supervised learning", "unsupervised learning" and "reinforcement learning", for example. In addition, on the basis of implementing these methods, there is also a method called "deep learning" that learns the extracted feature value itself. In this embodiment, it is assumed that "supervised learning" is applied to the machine learning of the virtual force learning unit 55.

[0069] As described in "Virtual Force Generation Method 2" above, when the tool tip is in contact with the workpiece, it is believed that the relative distance between the tool tip and the workpiece, the relative speed, the coefficient related to the workpiece's rigidity or dynamic friction, and the coefficient related to the tool's rigidity are correlated with the magnitude of the pressing force. Therefore, the virtual force learning unit 55 performs learning using learning data that uses these values ​​correlated with the magnitude of the pressing force as input data and the pressing force detected by the force sensor in this case as response data.

[0070] As a specific example of learning model construction, there is an example of constructing learning models corresponding to the first to third calculation examples of the virtual force F described above. When constructing a learning model corresponding to the first calculation example of the virtual force F described above, learning data is collected using the relative distance (δ) between the tool tip position and the target workpiece, the relative velocity (Vc), and the values ​​related to the rigidity of the target workpiece (Kd, Kc) (or at least the relative distance (δ) between the tool tip position and the target workpiece and the value related to the rigidity of the workpiece (Kd)) as input data, and the pressing force detected by the force sensor in this case as response data. Then, learning is performed using this learning data to construct the learning model.

[0071] In the case of constructing a learning model corresponding to the second calculation example of the above-mentioned virtual force F, learning data is collected with the movement amount (δ) of the tool front end position and the "virtual spring coefficient of the tool" as input data, and the pressing force detected by the force sensor in this case as response data. Then, learning is performed using the learning data to construct a learning model. In addition, learning data (training data) consisting of input data and response data as the pressing force detected by the force sensor in this case can also be collected, and learning is performed using the learning data to construct a learning model, wherein the input data includes at least one of a coefficient related to the rigidity of the target workpiece and a coefficient related to the rigidity of the tool part (tool), and the distance (δ) of the tool part relative to the target workpiece when the tool part is in contact with the target workpiece.

[0072] When constructing a learning model corresponding to the third calculation example of the virtual force F, learning data is collected using the movement position (Tx) based on the speed command and the position (d) of the tool tip after actual movement in response to the speed command as input data, and the pressing force detected by the force sensor in this case as response data. Learning is then performed using this learning data to construct a learning model. This learning process corresponds to the operation of learning the coefficient k.

[0073] The learning described above can be achieved using a neural network (e.g., a three-layer neural network). The action modes of a neural network include a learning mode and a prediction mode. In the learning mode, the above-mentioned learning data (input data) is assigned as an input variable to the neural network, and the weights applied to the inputs of each neuron are learned. The error between the output value when the input data is assigned to the neural network and the correct value (answer data) is taken, and the error is backpropagated (backpropagation) to each layer of the neural network, and the weights of each layer are adjusted so that the output value is close to the correct value, thereby performing weight learning. When a learning model is constructed through such learning, the above-mentioned input data can be assigned as an input variable to the virtual force learning unit 55 to predict the virtual pressing force. That is, the force control simulation execution unit 52 (virtual force generator 54) assigns the above-mentioned input data to the virtual force learning unit 5 that has performed learning, and obtains a virtual force as its output.

[0074] The sound output unit 74 outputs a sound whose volume indicates the magnitude of the virtual force generated by the virtual force generator 54. For example, during the execution of a force control simulation, a sound corresponding to the magnitude of the virtual force generated by the virtual force generator 54 is output in real time, thereby enabling the operator to more intuitively grasp the magnitude of the virtual force.

[0075] Next, the simulation of specific operations based on the force control simulation function performed by the control device 50 described above will be described using two examples, namely, the exploration operation and the embedding operation. Figure 3 As shown, a manipulator 10 has a structure in which a manipulator 9 is attached to the front end of its wrist to grip the mating components. The control device 50 (force control simulation execution unit 52) ​​executes these simulations using teaching data (operation program) provided by the operator, model data for objects such as the manipulator and workpiece, and information on the positions of each object within the workspace. For ease of explanation, the following description of tool models, workpiece models, and the like may be simply referred to as "tool," "workpiece," and the like.

[0076] like Figure 6 As shown, the searching operation is an operation of bringing a certain component (fitting component W10) into contact with another component (fitted component W11) to perform fitting while searching for the position of a hole. Figure 7 This is a flowchart showing the simulation operation of a task performed by the exploration function. Figure 7 The simulation of the search operation is executed under the control of the CPU (force control simulation execution unit 52) ​​of the control device 50. First, the control device 50 transports the fitting component W10 to the teaching point P1 on the fitted component W11 as the target workpiece (step S11). Figure 8A The positional relationship between the tool part 11 and the target workpiece W11 in step S11 is shown. Figure 8A As shown in FIG. 1 , the front end portion of the fitting part W10 is positioned at the teaching point P1. Figure 8A In the stage, the force detected by the virtual force generator 54 (virtual force sensor) is only the weight of the tool part 11 (the robot hand 9 and the fitting part W10).

[0077] Next, the control device 50 moves the tool unit 11 toward the target workpiece (fitted member W11 ) at the taught speed (step S12 ). Figure 8B At this stage, the virtual force generator 54 detects the inertial force generated in the direction opposite to the moving direction according to the amount of movement of the tool part 11 in the moving direction.

[0078] Next, the control device 50 detects the impact force generated by the collision between the fitting component W10 and the fitting component W11, the target workpiece, based on the position / velocity information of the tool portion 11 and the position information of the fitting component W11. Furthermore, upon detecting the impact force generated by the collision, the control device 50 performs force control in the direction of the target pressing force, moving the fitting component W10 parallel to and rotationally relative to the contact plane with the fitting component W11 while searching for a position where the force applied to the target workpiece disappears (step S13). In this case, if a search area on the fitting component W11 is specified as a force control parameter, the search can be performed according to that search area. Figure 8C and Figure 8D FIG. 1 shows a situation in which the fitting part W10 collides with the target workpiece (the fitted part W11) and comes into contact with it. Figure 8C ), an impact force is applied to the tool 11. The virtual force generator 54 calculates the force generated when the tool 11 collides with the target workpiece (engaged component W11) as follows. When the target workpiece stops, let the velocity of the tool 11 be v and its mass be m. If the impulse is fully preserved, the following equation for the impulse holds.

[0079] (a1) Assuming a constant force per unit time

[0080] mv=F·Δt

[0081] Where Δt is the time and F is the generated force.

[0082] (a2) When the generated force is treated as an impulse (assuming that the force f changes with time)

[0083] mv = (integral of f(t))

[0084] (a3) When there is a loss of momentum during a collision

[0085] mv=e·F·Δt

[0086] Where e is the coefficient of restitution.

[0087] The control device 50 (virtual force generator 54 ) uses any one of the above equations (a1) to (a3) ​​to determine the force generated when the tool portion 11 collides with the target workpiece (fitted member W11 ).

[0088] After the collision between the tip of the tool portion 11 and the target workpiece (engaged member W11) is detected, the following Figure 8DAs shown, the tool portion 11 is in contact with the target workpiece (engaged component W11). In this contact state, the control device 50 can use any of the aforementioned virtual force generation methods 1 to 3 to determine the virtual pressing force. Here, the virtual force generation method 1 is used to determine the virtual pressing force.

[0089] As described above, when the impact force is detected, the control device 50 controls the force in the target pressing force direction while causing the tool portion 11 (fitting part W10) to move parallel to and rotate on the contact surface with the target workpiece (fitted part W11), and searches for a position where the force applied to the tool portion 11 by the target workpiece disappears (i.e., a position where a hole exists) (steps S13, S14). Figure 8D In such a search, if the load exceeds the allowable value when inserting into the portion considered to be a hole or if the search time exceeds a predetermined threshold (S14: Yes), the search is deemed to have failed and the process ends (step S18).

[0090] When the position where the force (virtual force) received from the target workpiece disappears is found during the exploration, the control device 50 presses the fitting component W10 into the hole (step S15). When the fitting component W10 is pressed into the hole, if the fitting component W10 receives a torque from the inner wall of the hole, etc., the posture of the fitting component W10 (tool part 11) is corrected to offset the torque (step S16). For example, when the center line of the fitting component W10 is tilted relative to the center axis of the fitting hole of the fitted component W11, a torque around the axis perpendicular to the fitting direction may be generated. Based on the contact position between the fitting component W10 (tool part 11) and the fitted component W11, for example, the above-mentioned virtual force generation method 1 can be used to calculate the virtual force generated at the contact point between the fitting component W10 (tool part 11) and the fitted component W11. Then, based on the virtual force, the torque around a specified point (axis) on the fitting component W10 can be calculated. Furthermore, when determining the moment, for example, when a portion of the fitting component W10 contacts the inner peripheral surface of the hole of the fitted component W11, the moment acting on the fitting component W can be determined by considering the force acting in the normal direction of the contact surface on the inner peripheral surface of the hole. It is assumed that when the amount of press-fitting into the hole reaches the target depth, the control device 50 ends this search operation (step S17).

[0091] Next, the simulation of the embedding operation is described. Figure 9 As shown in FIG. 1 , a certain component (fitting component W10 ) is inserted into a hole of another component (fitted component W13 ). Figure 10This is a flowchart showing the simulated action of the embedding operation. First, the control device 10 moves the front end of the tool part 11 (i.e., the embedding part W10) to the top (teaching point) of the hole of the target workpiece (the embedded part W13) (step S21). Then, the control device 50 moves the embedded part W10 toward the hole (step S22). The force detected by the virtual force generator 54 in steps S21 and S22 is compared with the reference Figures 8A to 8D The same situation as described above.

[0092] Next, upon detecting the force applied to the fitting component W10 during insertion into the hole of the engaged component W13, the control device 50 moves the fitting component W10 in the insertion direction. If the fitting component W10 experiences a moment from the inner surface of the hole, etc., the control device 50 corrects the fitting component W10's posture to offset the moment while continuing insertion (step S23). Alternatively, the control device 50 can calculate the force (force in the direction opposite to the insertion direction) applied to the virtual force generating unit during insertion based on the outer shape of the fitting component W10, the inner diameter of the hole on the engaged component W13 side, the friction coefficient of the inner circumference of the hole, and other factors. During this insertion operation, if the load (virtual force) exceeds an allowable value or the operation time exceeds a predetermined threshold (S24: "Yes"), the search is deemed to have failed and the process ends (step S25). When the amount of movement in the insertion direction reaches the target depth, the control device 50 terminates the insertion operation (step S26). In step S26 , the insertion operation may be terminated when the amount of movement in the insertion direction reaches the target depth and the pressing force (virtual pressing force) reaches the target pressing force.

[0093] Next, the virtual force display function of the control device 50 is explained. As described above, the control device 50 can generate the force (virtual force) exerted on the tool part 11 by executing a force control simulation. The control device 50 provides information including the virtual force, the location where the virtual force is generated, and the position of the tool part as a result of the simulation to the display device 70. This can be displayed as an augmented reality image in which an image representing the magnitude and the location where the virtual force is generated is superimposed on a real image of the workspace, or as an image representing the magnitude and the location where the virtual force is generated is superimposed on a virtual reality image represented by model data of each object. In the case of generating a virtual reality image, for example, it can be configured to provide the model data and configuration position information of each object in the workspace including the manipulator 10 from the control device 50 to the display device 70.

[0094] Generally speaking, teaching operations based on force control and setting force control parameters are difficult. Before an operator, having created teaching data, actually operates a robot to perform a force-controlled operation such as grinding, the location and magnitude of the force applied to the workpiece are visualized and provided as augmented reality or virtual reality images. This allows the operator to instantly identify locations where excessive force is being applied to the workpiece, allowing them to accurately adjust the teaching point, motion speed, and force control parameters.

[0095] Figure 11 Here, the figure shows an example of displaying virtual force. The figure shows a situation where the operator performs teaching work in the work space while displaying the result of force control simulation as an augmented reality image on the display 73 of the display device 70. Figure 11 As shown, the actual tool portion does not contact the target workpiece, and the operator performs a force control simulation using the teach point, teach speed, and force control parameters that they have input. The control device 50 displays an image of the location 123 where the tool portion contacts the target workpiece, generating a virtual force, and indicates the magnitude of the virtual force, for example, through the size of an arrow 122. The virtual force can also be represented by a color corresponding to the magnitude or as a numerical value. Furthermore, the control device 50 displays, for example, an image 121 representing the tool portion in contact with the target workpiece as the result of the force control simulation, as a semi-transparent image superimposed on the real-world image.

[0096] Figure 12 This is an example of a case where an image 121 of a tool portion in contact with a target workpiece, an arrow 122 indicating the magnitude of the force, and an image of a generated portion 123 are displayed as an augmented reality image alongside an image of the real object. For example, there is also the case where the amount of information to be provided as an augmented reality image is large, such as Figure 12 Displaying the augmented reality image next to the real object is more convenient for the operator (wearer).

[0097] The control device 50 may further include an adjustment content generating unit 56 that generates adjustment content for the control parameter for suppressing the virtual force to be within the predetermined reference value based on the result of comparing the virtual force with the predetermined reference value. Figure 13 Shown in Figure 12 The augmented reality image of the force control simulation result shown in FIG. 1 also displays a message image 131 suggesting that the target pressing force be reduced. Figure 14 As shown, an image 141 for guiding the transition to the target pressing force setting screen may be displayed on the left side of the screen in the hierarchical setting menu. Figure 13In addition to such adjustment contents, there may be examples such as recommendations to move the teaching track away from the workpiece or reduce the motion speed when the pressing force is too strong, or adjustments such as reducing the force control gain.

[0098] exist Figure 13 and Figure 14 In the image of the force control simulation result shown in FIG, a selection icon 132 for transitioning to a screen for guiding the setting of force control parameters is also displayed. In this state, when the operator selects the selection icon 132 using, for example, an operating device externally connected to the display device 70, an interface image 152 for guiding the setting input of the force control parameters is displayed on the left side of the screen ( Figure 15 ). The operator can adjust and input the force control parameters according to the interface image 152. Figure 15 In the example shown in Figure 1, the operator can follow the guidance to sequentially set or adjust the force control position (teach point), target pressing force, and force control gain. This interface is particularly useful for operators unfamiliar with force control settings. Furthermore, if various parameter settings are changed via this interface, the force control simulation can be rerun and the results displayed on the interface.

[0099] As described above, according to the present embodiment, the operator can intuitively grasp the state of generation of the pressing force, and can facilitate parameter setting for force control.

[0100] The present invention has been described above using typical embodiments. However, those skilled in the art will appreciate that changes to the above embodiments and various other changes, omissions, and additions may be made without departing from the scope of the present invention.

[0101] The distribution of functions among the control device 50, display device 70, and external computer 90 in the above embodiment is merely an example, and the arrangement of these functional blocks can be changed. The imaging device may be arranged at a fixed position in the workspace as a separate device from the display device.

[0102] The functional blocks of the control device and the display device may be realized by the CPU of these devices executing various software stored in a storage device, or may be realized by a hardware-based structure such as an ASIC (Application Specific Integrated IC).

[0103] Programs for executing the various simulation processes in the above-described embodiments can be recorded on various computer-readable recording media (eg, semiconductor memories such as ROM, EEPROM, and flash memory, magnetic recording media, and optical disks such as CD-ROM and DVD-ROM).

[0104] Description of Reference Numerals

[0105] 3: Force sensor; 10: Robot manipulator; 11: Tool unit; 50: Control device; 51: Storage unit; 52: Force control simulation execution unit; 53: Robot motion control unit; 54: Virtual force generator; 55: Virtual force learning unit; 70: Display device; 71: Camera device; 72: AR / VR image processing unit; 73: Display; 74: Sound output unit; 90: External computer; 91: Physical simulation unit; 100: Robot system.

Claims

1. A robot simulator that simulates a force control operation performed while a tool mounted on a robot manipulator is brought into contact with a target workpiece, the robot simulator comprising: a storage unit storing an operation program and force control parameters as setting parameters related to the force control operation; and a force control simulation execution unit that executes a simulation of the force control operation based on the operation program and the force control parameters, in, The force control simulation execution unit is configured to include a virtual force generation unit that generates a virtual force that the tool unit receives from the target workpiece when the tool unit is in contact with the target workpiece based on position information of the tool unit obtained from a simulation result of the force control action. The force control simulation execution unit performs a simulation of the force control action based on the virtual force and a target force set as the force control parameter. The virtual force generation unit obtains the virtual force based on any one of a coefficient related to the rigidity of the target workpiece, a coefficient related to the rigidity of the tool part, and a spring constant of the tool part, and position information of the tool part.

2. The robot simulation device according to claim 1, wherein: further comprising a physical simulation unit that uses a motion equation representing the robot manipulator and performs a physical simulation of the motion of the robot manipulator based on the motion program and the force control parameters, The virtual force generation unit obtains the virtual force based on the position information of the tool obtained by the physical simulation in a state where the tool is in contact with the target workpiece.

3. A robot simulator that simulates a force control operation performed while a tool portion mounted on a robot manipulator contacts a target workpiece, the robot simulator comprising: a storage unit storing an operation program and force control parameters as setting parameters related to the force control operation; and a force control simulation execution unit that executes a simulation of the force control operation based on the operation program and the force control parameters, in, The force control simulation execution unit is configured to include a virtual force generation unit that generates a virtual force that the tool unit receives from the target workpiece when the tool unit is in contact with the target workpiece based on position information of the tool unit obtained from a simulation result of the force control action. The force control simulation execution unit performs a simulation of the force control action based on the virtual force and a target force set as the force control parameter. The robot simulation device further includes a virtual force learning unit that performs machine learning based on training data consisting of input data including at least one of a coefficient related to the rigidity of the target workpiece and a coefficient related to the rigidity of the tool part, and a distance of the tool part relative to the target workpiece when the tool part is in contact with the target workpiece, and answer data corresponding to the input data as an actually measured pressing force. The virtual force generation unit acquires the virtual force using a learning model constructed by the machine learning performed by the virtual force learning unit.

4. The robot simulation device according to any one of claims 1 to 3, further comprising: an imaging device that captures an image of a real work space including the robot manipulator and the target workpiece; and The display device superimposes an image indicating the magnitude of the virtual force and a location where the virtual force is generated on the target workpiece as an augmented reality image on an image of the work space.

5. The robot simulation device according to any one of claims 1 to 3, wherein: The storage unit further stores model data representing the shapes and arrangement position information of the robot manipulator, the tool unit, and the target workpiece. The robot simulation device also has a display device, which uses the model data and the configuration position information to superimpose an image representing the magnitude of the virtual force and the location where the virtual force is generated on the object workpiece on a virtual reality image configured in a virtual working space including the tool part and the object workpiece for display.

6. The robot simulation device according to claim 4, wherein: further comprising an adjustment content generating unit for generating, based on a result obtained by comparing the virtual force with a predetermined reference value, an adjustment content for suppressing the virtual force within the predetermined reference value for the force control parameter; The adjustment content generating unit further superimposes an image indicating the adjustment content on the video and displays the image.

7. The robot simulation device according to claim 5, wherein: further comprising an adjustment content generating unit for generating, based on a result obtained by comparing the virtual force with a predetermined reference value, an adjustment content for suppressing the virtual force within the predetermined reference value for the force control parameter; The adjustment content generating unit further superimposes an image representing the adjustment content on the virtual reality image for display.

8. The robot simulation device according to any one of claims 1 to 3, wherein: The device further includes a sound output unit that outputs a sound indicating the magnitude of the virtual force in terms of volume.

9. The robot simulation device according to any one of claims 1 to 3, wherein: The force control action is an exploratory operation, an embedding operation or an imitation action.

Images