Elastic modulus estimation method

Abstract

To provide a modulus-of-elasticity estimation method which can efficiently estimate a modulus of elasticity of a workpiece with high accuracy even in the case of the workpiece being a flexible object.SOLUTION: In the modulus-of-elasticity estimation method for an object in work, a working device comprises a sensor for acquiring at least a shape of a target object and a stress sensor for detecting a stress at the time of contact with the target object, and a modulus-of-elasticity estimation device acquires a shape of the target object before deformation and a shape of the target object after the deformation, acquires a stress at the time of the deformation of the target object by the stress sensor, calculates bending energy from a displacement due to the deformation, and estimates a modulus of elasticity from the bending energy and a value of the stress.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a method for estimating elastic modulus. [Background technology]

[0002] Robots are being asked to perform a variety of tasks. One of these tasks is the ability to handle flexible objects such as cables (see, for example, Patent Document 1). It is difficult for a robot to handle such flexible objects with its hands. In order for a robot to handle flexible objects such as cables, it is necessary to generate a grasping method and procedure that takes into account the object's bendability. Thus, when handling flexible objects, it is necessary to take into account the step factor. In order to have a robot perform an appropriate task, it is necessary to appropriately estimate the elastic modulus of the object being worked on. In response to this, it has been proposed to estimate the elastic modulus by bringing a probe into contact with the object being measured (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6474179 [Patent Document 2] Japanese Patent Application Laid-Open No. 2009-034705 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology does not model the ease of deformation of soft objects, and the position and shape of the object change significantly due to internal forces, etc., so the hand trajectory and object position must be limited, making it impossible to apply to general environments. Also, with conventional technology that involves contacting a probe with the object, it takes time to fine-tune the spatial resolution of the measurement points.

[0005] The present invention has been made in consideration of the above-mentioned problems, and aims to provide an elastic modulus estimation method that can efficiently and accurately estimate the elastic modulus of a work object, even if it is a flexible object. [Means for solving the problem]

[0006] (1) In order to achieve the above object, one aspect of the present invention provides an elastic modulus estimation method for an object during work, in which a work device includes a sensor that acquires at least the shape of a target object and a stress sensor that detects stress when the target object is touched, and an elastic modulus estimation device acquires the shape of the target object before deformation and the shape of the target object after deformation, acquires the stress of the target object during deformation using the stress sensor, calculates bending energy from the amount of displacement before and after deformation, and estimates the elastic modulus from the values ​​of the bending energy and the stress.

[0007] (2) In addition, in an elastic modulus estimation method according to one aspect of the present invention, the elastic modulus estimation device may associate and store the three-dimensional shape of the target object before deformation with control points, associate and store the three-dimensional shape of the target object after deformation with control points and the stress, estimate warp fields of the target object before and after deformation, and calculate the bending energy from the integral value of the displacement of landmarks before and after deformation.

[0008] (3) In addition, in an elastic modulus estimation method according to one aspect of the present invention, the elastic modulus estimation device may align the deformed shape of the target object with the position of the shape before deformation in the estimated warp field, integrate the aligned three-dimensional shapes of the target object before and after deformation, and retain the estimated elastic modulus in the edge portions that make up the integrated three-dimensional shape.

[0009] (4) In addition, in an elastic modulus estimation method according to one aspect of the present invention, the elastic modulus estimation device may associate shape information of a target object and the elastic modulus with a target object class indicating a target class of the object and store the associated information in a database, acquire the target object class during work, refer to the database using the target object class acquired during work to acquire shape information and the elastic modulus of the object, and use the acquired shape information and elastic modulus to generate a work command to be performed by the work device.

[0010] (5) Furthermore, in an elastic modulus estimation method according to one aspect of the present invention, the elastic modulus estimation device may generate an initial target trajectory sequence for a task without taking dynamics into consideration, simulate an operation at time t, obtain a state of the target object and the task device when a force is applied to the target object for each time t, calculate a residual between the initial target trajectory sequence and the state of the target object and the task device for each time t, correct and determine a target trajectory sequence until the residual is within a predetermined value, and generate the task command using the determined target trajectory sequence.

[0011] (6) In addition, in the elastic modulus estimation method according to one aspect of the present invention, the elastic modulus estimation device may simultaneously estimate the elastic modulus around the contact point by calculating the displacement of a point associated with an image feature obtained from an image captured by the sensor. [Effects of the Invention]

[0012] According to (1) to (6), the elastic modulus of the work object can be estimated efficiently and accurately, even if the object is flexible. As a result, according to (1) to (6), the posture of the grasped object can be estimated accurately, even if the object is flexible. Furthermore, according to (1) to (6), information for grasping and operation planning can be increased. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 10 is a diagram illustrating an example of a state in which the robot handles a cable. [Figure 2] 1 is a diagram illustrating an example of the configuration of an elastic modulus estimation device according to an embodiment. [Figure 3] 10 is a flowchart of an example of a procedure for estimating an elastic modulus according to the embodiment. [Figure 4] 4A and 4B are diagrams showing an image of region division in the processing of FIG. 3 and an image of association with image features. [Figure 5] This is an image diagram of an example of n1 viewpoint before and at the current time, and an example of n2 viewpoint before and at the current time. [Figure 6] FIG. 1 is an image diagram for explaining the meaning of equation (20). [Figure 7] FIG. 1 is an image diagram for explaining a method according to an embodiment. [Figure 8] FIG. 10 is a diagram for explaining strain energy. [Figure 9] FIG. 10 is a diagram showing an example in which a robot is performing a task of bending a cable. [Figure 10] 10 is a flowchart of an example of a processing procedure during work according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings used in the following description, the scale of each component is appropriately changed so that each component can be recognized.

[0015] (overview) FIG. 1 is a diagram showing an example of a state in which a robot (working device) is handling a cable. In the example of FIG. 1, the robot 1 (working device) is performing a task of bending a cable, which is an object Obj, with its right hand 11R, in accordance with the control of an elastic modulus estimation device 100 provided inside the robot 1. In such a task, the robot 1 needs guidance because it does not know how to bend the cable. For this reason, in this embodiment, the elastic modulus is estimated and a target trajectory sequence is corrected based on the deformation of the object. Note that the cable is an example of fabric softener, and is not limited to this.

[0016] (Elastic modulus estimation device) An example of the configuration of the elastic modulus estimation device 100 will be described. Fig. 2 is a diagram showing an example of the configuration of an elastic modulus estimation device according to this embodiment. As shown in Fig. 2, the elastic modulus estimation device 100 includes a first sensor 101, a second sensor 102, an acquisition unit 103, an elastic modulus estimation unit 104, a DB 105, a dynamics simulation unit 106, and a motion generation unit 107. The elastic modulus estimation device 100 may also include a model storage unit 111 and a target object class estimation unit 112. The elastic modulus estimation device 100 is connected to the robot 1 by wire or wirelessly.

[0017] The robot 1 includes an internal sensor 108, a control device 109, and an actuator 110. The robot 1 also includes a sound pickup unit, a camera unit, a power supply unit, and the like. The configuration shown in FIG. 2 is an example and is not limiting. For example, the elastic modulus estimation device 100 may be provided in the robot 1 or may be an external device. Furthermore, the elastic modulus estimation device 100 may be realized on the cloud. Furthermore, in the embodiment, the robot 1 is described as an example of a working device, but the working device is not limited to a robot. For example, the working device may be a working device equipped with an arm or a hand.

[0018] The first sensor 101 is, for example, an RGB-D sensor. The first sensor 101 captures a color image (RGB image) and a depth image (depth image). The first sensor 101 outputs the captured color image and depth image to the elastic modulus estimation unit 104. The first sensor 101 may be, for example, an imaging device and a distance sensor.

[0019] The second sensor 102 is, for example, a force sensor. The second sensor 102 detects stress and outputs stress information indicating the detected stress to the elastic modulus estimation unit 104.

[0020] The acquisition unit 103 acquires target object class information indicating the target object class from the instructor or the target object class estimation unit 112. During learning, the acquisition unit 103 outputs the acquired target object class information to the elastic coefficient estimation unit 104. During work, the acquisition unit 103 outputs the acquired target object class information to the action generation unit 107. Note that the target object class is information indicating what the target object is, such as the name of the target object (e.g., processed cheese, French bread, etc.). Note that during work, the worker does not need to input the object class. For example, the work object may be determined from a captured image and the worker's line of sight, and the object class of the determined work object may be determined by pattern matching, etc.

[0021] The elastic coefficient estimation unit 104 estimates an object model including elastic coefficients and object shape based on the color image, depth image, stress information, and target object class information. Note that a method for estimating elastic coefficients will be described later. The elastic coefficient estimation unit 104 stores the object model in DB 105. Note that the object model includes, for example, information indicating the object shape and elastic coefficients.

[0022] DB105 is a database that stores an object model identifier, an object model including an object shape and elastic coefficient, and a target object class in association with the object model identifier. DB105 stores the relationship between force and deformation when the target object is a rigid body and the dynamics are not taken into consideration.

[0023] The dynamics simulation unit 106 performs, for example, generation of a sequence of target trajectories in a task, estimation of deformation of a target object, and correction of the target trajectories.

[0024] The motion generation unit 107 acquires object model information from the acquisition unit 103. Using the acquired object model information, the motion generation unit 107 refers to the DB 105 and acquires an object model associated with the acquired object model information. Using the object model, the motion generation unit 107 generates a speed direction command, which is work command information. Furthermore, based on the target trajectory sequence generated by the dynamics simulation unit 106, the motion generation unit 107 modifies the speed direction command so as to modify the target trajectory sequence.

[0025] The robot 1 includes at least one arm and one hand, and may be, for example, a bipedal robot.

[0026] The internal sensor 108 is, for example, an angle sensor, an angular velocity sensor, an acceleration sensor, a six-axis sensor, a gyro sensor, etc. The internal sensor 108 detects, for example, the angle, angular velocity, acceleration, etc. of the arms 12 (12L, 12R) (FIG. 1) and hands 11 (11L, 11R) of the robot 1.

[0027] The control device 109 controls the driving of the actuator 110 based on the speed direction command generated by the motion generation unit 107 and the detection result detected by the internal sensor 108. The control device 109 controls the motion according to the speed direction command generated by the motion generation unit 107, for example, and corrects and controls the motion using the detection value acquired by the internal sensor 108.

[0028] The actuators 110 include, for example, arm actuators provided at each joint (e.g., shoulder joint, elbow joint, wrist joint) of the robot 1, and hand joint actuators provided at the hand and finger parts. The actuators 110 operate under the control of the control device 109. Note that the drive circuit of the actuators 110 is omitted in FIG. 2.

[0029] (Method for estimating elastic modulus) Next, a method for estimating the elastic modulus will be described. Fig. 3 is a flowchart of an example of a processing procedure for estimating elastic modulus according to this embodiment. Fig. 4 is a diagram showing an image of region division in the processing of Fig. 3 and an image of association with image features. Note that the processing of Fig. 3 is an example of a processing procedure for storing an object model (shape, elastic modulus) in DB 105, i.e., during learning.

[0030] (Step S1) The first sensor 101 acquires an RGB-D image. The second sensor 102 acquires stress information. Note that the stress before the robot 1 comes into contact with the target object is 0.

[0031] (Step S2) The elastic modulus estimation unit 104 divides the RBG-D image output by the first sensor 101 into regions for each object label as shown in image g11 in Fig. 4. The elastic modulus estimation unit 104 performs the region division using, for example, a trained CNN (Convolutional Neural Network) classifier.

[0032] (Step S3) The elastic modulus estimation unit 104 determines whether or not a work object is present in the captured image. If the elastic modulus estimation unit 104 determines that a work object is present in the captured image (Step S3; YES), the process proceeds to Step S4. If the elastic modulus estimation unit 104 determines that a work object is not present in the captured image (Step S3; NO), the process proceeds to Step S17.

[0033] (Step S4) The elastic modulus estimation unit 104 generates a 3D image by constructing only the target object in 3D. The elastic modulus estimation unit 104 performs image processing (e.g., binarization, edge detection, clustering, etc.) on the 3D image of the target object using a well-known method to extract image feature points (e.g., vertices, edges).

[0034] (Step S5) The elastic modulus estimation unit 104 extracts control points for the task (for example, gripping) based on the image features as shown in image g12 in FIG.

[0035] (Step S6) The elastic modulus estimation unit 104 calculates the three-dimensional positions of the control points.

[0036] (Step S7) The elastic modulus estimation unit 104 determines whether the hand 11 of the robot 1 has not yet come into contact with the target object based on the stress information. If the elastic modulus estimation unit 104 determines that the hand 11 has not yet come into contact with the target object (step S7; YES), the process proceeds to step S8. If the elastic modulus estimation unit 104 determines that the hand 11 has already come into contact with the target object (step S7; NO), the process proceeds to step S11.

[0037] (Step S8) Since the target object has not yet been contacted and is not deformed, the elastic modulus estimation unit 104 holds the three-dimensional shape and control points of the target object before deformation.

[0038] (Step S9) The motion generator 107 generates a speed / direction command using the object model. The controller 109 controls the actuator 110 based on the generated speed / direction command to perform a motion to contact the target object.

[0039] (Step S10) The elastic modulus estimation unit 104 calculates the elastic modulus through the processes of steps S11 to S16.

[0040] (Step S11) After contact with the target object, the target object is deformed compared to before contact, so the elastic coefficient estimation unit 104 stores the three-dimensional shape, control points, and stress of the deformed target object. Note that when the hand 11 comes into contact with the target object, the amount of force with which the target object is being pressed can be determined from the detection value of the second sensor, which is a force sensor of the hand 11.

[0041] (Step S12) The elastic coefficient estimation unit 104 estimates the Warp Field before and after deformation of the target object. The Warp Field is a deformation space that handles rigid and non-rigid transformations. The elastic coefficient estimation unit 104 calculates bending energy from the integral value of landmark displacement before deformation. A landmark is a target point. The elastic coefficient estimation unit 104 determines a target value of stress when planning a contact motion. The actual value of stress is measured and obtained, for example, by a six-axis sensor on the fingertip of the robot 1. As a result, the elastic coefficient estimation unit 104 calculates the elastic coefficient from the bending energy and stress.

[0042] (Step S13) The elastic modulus estimation unit 104 aligns the post-deformation shape with the pre-deformation shape using the estimated WarpField.

[0043] (Step S14) The elastic modulus estimation unit 104 integrates the three-dimensional shape of the target object before deformation and the three-dimensional shape of the target object after deformation.

[0044] (Step S15) The elastic modulus estimation unit 104 stores the estimated elastic modulus in the edge portion that constitutes the integrated three-dimensional shape.

[0045] (Step S16) The elastic modulus estimation unit 104 stores the results of the estimation and processing in steps S11 to S16 in the DB 105.

[0046] When working with flexible objects, it is necessary to estimate the softness of the object, i.e., the elastic modulus of the target object. In such tasks, it is necessary to generate a task procedure that takes into account the bending of the target object. However, in previous robotic tasks, the target object was not treated as a flexible object, but as a rigid body that does not deform. For this reason, in this embodiment, the target object is modeled as a flexible object, and control is performed by solving the control equations. For this reason, in this embodiment, a model is created that maintains the flexibility between edge vertices in the mesh of the surface shape of the target object. As a result, according to this embodiment, it is possible to control work involving flexible objects.

[0047] The learning of the elastic coefficient starts before the target object is grasped, and is performed while the target object is being grasped.

[0048] (Background technology) In this embodiment, it is necessary to estimate the Warp Field before and after deformation of a target object. For this purpose, we will first explain a technique related to a well-known non-rigid deformation model (see Reference 1). This model is expressed using vertices and edges. In Reference 1, the problem is posed as determining how the vertex shapes deform, for example.

[0049] Reference 1; Mingsong Dou, Sameh Khamis, et al, “Fusion4D: Real-time Performance Capture of Challenging Scenes”, Microsoft Research, ACM Transactions on Graphics, Volume 35, Issue 4, July 2016, Article No. 114, pp 1-13

[0050] ED (Embedded Deformation) node g uniformly extracted from RefTSDF (Truncated Signed Distance Function) k The set is given by the following equation (1):

[0051]

number

[0052] A set of ED node indices S m is expressed as the following equation (2).

[0053]

number

[0054] The reference frame mesh V extracted from the reference TSDF is expressed by the following equation (3): where R is the set of all real numbers.

[0055]

number

[0056] Also, the skinning weight w k m is expressed as the following equation (4).

[0057]

number

[0058] In equation (4), Z is a value for normalization, σ is, for example, 0.5d, and d is the average distance between adjacent ED nodes after uniform sampling. The parameter G is given by the following equation (5).

[0059]

number

[0060] In equation (4), the global rotation R, the global translation T, and the affine transformation A k , local translation t k is expressed as the following equation (6).

[0061]

number

[0062] The transformation formula for the vertices of the key frame is expressed as the following formula (7).

[0063]

number

[0064] The transformation formula for the normal of the key frame is expressed as the following formula (8).

[0065]

number

[0066] To obtain the WarpField, the energy function E(G) is formulated, the type of deformation that is allowed is regularized by giving a penalty to the discrepancy between the model and the observed data, and constraints, etc. are encoded. As a result, the energy function R(G) is expressed as in the following equation (9). In this case, the parameters are found by finding the term that minimizes this energy function.

[0067]

number

[0068] In equation (9), E data (G) is the current data, hull is a term to keep the deformation of the model within the field of view, and E rot (G) is a term that makes the affine transformation closer to a rigid transformation (rotation transformation), and E smooth (G) is a term for approximating the affine transformation between neighboring ED nodes, and λ is a weight. The constraint for approximating the deformation of the model to the current data is given by the following equation (10).

[0069]

number

[0070] Equation (10) can be approximated as follows: n (G) is the set of vertices visible at the nth viewpoint.

[0071]

number

[0072] Note that when updating parameters or when the projection crosses pixel boundaries, P n (Π n (v)) may vary discretely and prevent differentiability, which may lead to slow or no convergence.

[0073] FIG. 5 is an image diagram of an example of n1 viewpoint before the start (Prev) and the current (Curr) and an example of n2 viewpoint before the start and the current (Curr). Also, as a method for finding parameters that minimize this energy function, for example, the correspondence method is known. In the correspondence method, the constraint for making the model deformation closer to the previous parameters is the following equation (12). In the correspondence method, the cost function is such that the correspondence before and after deformation matches.

[0074]

number

[0075] In equation (12), q nf is expressed as the following equation (13).

[0076]

number

[0077] Moreover, the corresponding set of viewpoint n is given by the following equation (14).

[0078]

number

[0079] In Reference 1, corresponding points are extracted using a learning-based method called Global Patch Collider. However, in the technique described in Reference 1, the number of corresponding points is overwhelmingly smaller than that of GPC (Gel Permeation Chromatography), which is thought to be the cause of the deterioration of the convergence of the energy function (sparse nonlinear).

[0080] Returning to equation (9), the explanation continues. Term E to keep the deformation of the model within the field of view hull (G) is expressed as the following equation (15).

[0081]

number

[0082] In addition, the term E rot (G) can be expressed as the following equation (16).

[0083]

number

[0084] In addition, a term E is used to approximate the affine transformation between neighboring ED nodes. smooth (G) can be expressed as the following equation (17).

[0085]

number

[0086] In equation (17), ρ(|| ∥) is a robustifier that takes into account discontinuities in the deformation field, and represents the residual term of the robust loss function. The loss function uses the Cauchy loss as shown in equation (18).

[0087]

number

[0088] (Method of estimating both object flexibility and 3D reconstruction in accordance with the present invention) Next, we describe our method for estimating both the flexibility and 3D reconstruction of an object. In this embodiment, the deformation model of DynamicFusion is used, and in the energy function R(G) of Equation (9) of Reference 1, λ smooth E smoorh (G) term is expressed as λ TPS E TPS In this embodiment, the other terms are found by finding the replaced TPS term.

[0089]

number

[0090] In addition, E TPS (Thin Plate Spline) (G) is expressed by the following equation (20): The TPS method is a technique for image processing in which an image is elastically deformed.

[0091]

number

[0092] The meaning of equation (20) is a heat map g23 of bending energy of energy g22 after deformation relative to energy g21 before deformation, as shown in Fig. 6. Fig. 6 is an image diagram for explaining the meaning of equation (20). In this embodiment, a combination of voxels (nodes) and forces is used, as shown in Fig. 7. Fig. 7 is an image diagram for explaining the method of this embodiment. When a force F is applied from the left and right to the pre-deformation g31 in Fig. 7, it deforms as shown in g32 after deformation. Fig. 7 is a diagram showing an example of modeling the elasticity of an object between nodes.

[0093] Next, we explain thin plate splines and deformation decomposition (see Reference 2).

[0094] Reference 2;Fred L, Bookstein, “Principal warps: thin-plate splines and the decomposition of deformations”, IEEE, TRANSACTIONS ON PATTEN ANALYSIS AND MACHINE INTELLIGENCE, Vol,11 No.6, 1989, p567-585

[0095] First, let us explain the symbols used in the formula. U(r) is a function. P i is the coordinate (x i ,y i ) a1,a x ,a y are the coefficients, and the matrix W is (w1,…,w n ) The matrix K is given by the following equation (21), the matrix P is given by the following equation (22), and the block matrix L is given by the following equation (23). Also, FIG. 8 is a diagram for explaining the distortion energy.

[0096]

number

[0097]

number

[0098]

number

[0099] The last three rows of L act as coefficients w i The sum of these is zero, and the point P i This is to ensure that the cross product of the x and y coordinates of is approximately zero. Also, the distance r ij is the distance between positions i and j, as shown in the following equation (24).

[0100]

number

[0101] Also, in equation (23), the matrix P T represents the transpose of matrix P. Matrix O represents a 3-by-3 matrix whose elements are all 0. Also, let some n vectors be V=(v1,…,v n ), Y is (V|0 0 0) T and is a vector of (n+3) columns. Also, the vector W, coefficients a and L -1 The relationship with Y is given by the following equation (25).

[0102]

number

[0103] This L -1 Using the elements of Y, a function f(x, y) is defined at an arbitrary position on the plane as in the following equation (26).

[0104]

number

[0105] Then, as described in Reference 2, we use a function f that minimizes a non-negative quantity to integrate only the displacement in the definition of strain energy, I f can be expressed as the following equation (27).

[0106]

number

[0107] The meaning of this equation (27) is that it is the integration of only the displacement in the definition of strain energy. That is, the energy E in equation (20) TPS (G) means that it is equivalent to the definition of strain energy in equation (27) obtained by integrating only the displacement.

[0108] As described in Reference 2, Equation (27) can be obtained as the following Equation (28).

[0109]

number

[0110] In addition, in equation (28), L n -1 L -1 This integer is zero if and only if all components of W are zero. In this case, the calculated spline is f(x, y) = a1 + a x x+a y y, resulting in a flat surface. In the technique described in Reference 2, the point (x i ,y i ) is a landmark, and V has n rows and 2 columns as shown in the following equation (28).

[0111]

number

[0112] Also, in equation (29), (x i ',y i ') each of which is R 2 In another copy of (x i ,y i ) is a homologous landmark.

[0113] The bending energy should be proportional to the displacement in equation (27) multiplied by the ratio of stress to elastic modulus. For this reason, the robot 1 knows the stress applied to the object through command values ​​or sensor values, and can therefore calculate the elastic modulus. What this means is that when considering a cylinder with an infinitesimally small circular area, if we assume a linear elastic body when stress is applied perpendicular to the circle, the simple Hooke's law (displacement = stress / elastic modulus) holds. For this reason, stress and displacement can be observed, and the elastic modulus can be calculated. Also, when considering a system with an area (assuming a homogeneous material), the total amount of stress and the resulting displacement becomes strain energy (in the case of tension). The relationship between stress and elastic modulus is U = ∫(stress 2 / 2 elastic modulus) dx. In this equation, the elastic modulus and stress are constants, so they can be taken out of the integral. Therefore, a value proportional to the bending energy can be obtained from the integral of stress and displacement.

[0114] In this manner, in this embodiment, the bending energy is calculated by minimizing the energy function E(G). Then, in this embodiment, the stress is obtained from the command value or the detected value of the sensor. Furthermore, in this embodiment, the bending energy and stress are used to calculate the elastic modulus from equation (28).

[0115] (Motion planning and dynamics simulation) Next, an example of processing performed by the motion generation unit 107 and the dynamics simulation unit 106 will be described. FIG. 9 is a diagram showing an example of a robot bending a cable. FIG. 10 is a flowchart showing an example of a processing procedure during work according to this embodiment. The following work example is, for example, winding a cable. FIG. 9 is an image of the robot performing the winding work with its right hand, using a guide g54 attached to a glass surface g53, for example.

[0116] (Step S101) The motion generation unit 107 or the dynamics simulation unit 106 generates an initial target trajectory sequence for how the cable g11 is to be wound, without considering dynamics, etc., using the information stored in the DB 105. The initial target trajectory sequence is the trajectory of the motion of the tip of the hand 11.

[0117] (Step S102) The dynamics simulation unit 106 repeats the processes of steps S102 to S107 for each time t until the condition of step S106 is satisfied.

[0118] (Step S103) The dynamics simulation unit 106 executes the operation at time t in the simulation.

[0119] (Step S104) Here, the dynamics of the target object, such as its flexibility, are generated in advance. Therefore, the dynamics simulation unit 106 can predict how the target object will deform if a force is applied to the target object where and in what direction (for example, the dashed line g52 in FIG. 9). In this way, the dynamics simulation unit 106 observes the states of the robot and the target object through simulation.

[0120] (Step S105) The dynamics simulation unit 106 calculates the residual (deviation, for example, the double-headed arrow g55 in FIG. 9) between the result of the actual dynamics simulation and the initial target trajectory sequence.

[0121] (Step S106) The dynamics simulation unit 106 determines whether the residual is within a predetermined value. If the dynamics simulation unit 106 determines that the residual is within the predetermined value (step S106; YES), it ends the repeated processing of steps S102 to S107. If the dynamics simulation unit 106 determines that the residual is not within the predetermined value (step S106; NO), it proceeds to step S107.

[0122] (Step S107) The dynamics simulation unit 106 corrects the target trajectory and returns to the processing of step S103.

[0123] The work may start before grasping the target object, or may start after grasping the target object.

[0124] In this manner, in this embodiment, the dynamics simulation unit 106 simulates the deformation of the target object for each time t, and corrects the target trajectory based on the deviation of the deformation when the object is a rigid body. The motion generation unit 107, for example, calculates the amount of movement required for the transition from the original predicted value and the simulated value. The motion generation unit 107 corrects the trajectory at time t so as to cancel out the linear motion calculated in this manner, and performs the simulation again. In this manner, the dynamics simulation unit 106 repeats the simulation for each time until the residual error falls within a predetermined value.

[0125] After the simulation of FIG. 10 is completed, the motion generation unit 107 generates a control command and controls the robot 1.

[0126] (Variation) Here, an example of a method for determining the elastic modulus according to the prior art will be described. Generally, elastic modulus measurements are performed by a contact method, for example, by pressing a probe against the object, which takes a long time to measure. If the resolution of the elastic modulus measurement is increased using such a contact method, it is not possible to measure it in a realistic time frame. In contrast, in this embodiment, by calculating the displacement of points associated by features, the softness (elastic modulus) around the contact point can also be estimated at the same time, which significantly reduces the time required to obtain the elastic modulus compared to conventional techniques. Furthermore, according to this embodiment, these processes make it possible to obtain the elastic modulus at high resolution in a shorter time than conventional techniques.

[0127] As described above, according to this embodiment, the elastic modulus can be efficiently estimated even if the target object is flexible. Furthermore, according to this embodiment, the assumption that the target object is a rigid body can be eliminated, so the posture of the grasped object can be accurately estimated even if the object is flexible. Note that this embodiment makes it possible to handle various objects. Furthermore, according to this embodiment, hard or soft parts of the object can be presented, so more information can be provided for grasping and operation planning.

[0128] A program for implementing all or part of the functions of the elastic modulus estimation device 100 of the present invention may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be loaded into a computer system and executed to perform all or part of the processing performed by the elastic modulus estimation device 100. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. The term "computer system" also includes a WWW system equipped with a website provision environment (or display environment). The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. The term "computer-readable recording medium" also includes devices that retain a program for a certain period of time, such as volatile memory (RAM) within a computer system that acts as a server or client when the program is transmitted via a network such as the Internet or a communication line such as a telephone line.

[0129] The program may also be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may also be a program that realizes part of the above-mentioned functions. Furthermore, the program may be a so-called differential file (differential program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0130] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0131] 1...robot, 100...elastic coefficient estimation device, 101...first sensor, 102...second sensor 102, 103...acquisition unit, 104...elastic coefficient estimation unit, 105...DB, 106...dynamics simulation unit, 107...motion generation unit, 108...internal sensor, 109...control device, 110...actuator

Claims

1. A method for estimating an elastic modulus of an object in a task, comprising: the working device includes at least a sensor that acquires the shape of the target object and a stress sensor that detects stress when the working device touches the target object; The elastic modulus estimation device acquiring a shape of the target object before deformation and a shape of the target object after deformation; Obtaining stress during deformation of the target object by the stress sensor; The bending energy is calculated from the amount of displacement before and after deformation. An elastic modulus estimation method for estimating an elastic modulus from the bending energy and stress values.

2. The elastic modulus estimation device The three-dimensional shape of the target object before deformation is associated with the control points and stored; The three-dimensional shape of the target object after deformation, the control points, and the stresses are associated and stored; a warp field of the target object before and after deformation is estimated, and the bending energy is calculated from an integral value of displacement of the landmark before and after deformation; The elastic modulus estimation method according to claim 1 .

3. The elastic modulus estimation device In the estimated warp field, aligning the deformed shape of the target object to a position of the pre-deformed shape; Integrating the three-dimensional shapes of the aligned target object before and after deformation; retaining the estimated elastic coefficient at an edge portion constituting the integrated three-dimensional shape; The elastic modulus estimation method according to claim 2 .

4. The elastic modulus estimation device storing the shape information of the target object and the elastic coefficient in association with a target object class indicating a target class of the object in a database; Obtaining the target object class during operation; Using the target object class acquired during the operation, the database is referenced to acquire shape information and the elastic coefficient of the object; generating a work command to be performed by the work device using the acquired shape information and the elastic coefficient; The elastic modulus estimation method according to any one of claims 1 to 3.

5. The elastic modulus estimation device Generate an initial target trajectory sequence for the task without considering dynamics, The operation at time t is executed in a simulation. acquiring states of the target object and the working device when a force is applied to the target object for each time t; calculating a residual between the state of the target object and the working device for each time t and the initial target trajectory sequence; correcting and determining the target trajectory sequence until the residual error is within a predetermined value; generating the work command using the determined target trajectory sequence; The elastic modulus estimation method according to claim 4.

6. The elastic modulus estimation device and simultaneously estimating the elastic modulus around the contact point by calculating the displacement of the point associated with the image feature acquired from the image captured by the sensor. The elastic modulus estimation method according to any one of claims 1 to 5.

Images