Manipulator system, management device for managing operation state of manipulator, and management method therefor

Abstract

To enable the detection of a gripping state in addition to a contact state with respect to a target object in a manipulator.SOLUTION: A manipulator system provided with drive means for driving a manipulator and measurement means for measuring a position of the manipulator. The manipulator system includes: a storage unit that stores a mass of the manipulator before operation of the manipulator and a spring coefficient of the manipulator before the operation of the manipulator; a reception unit that receives an input signal that drives the manipulator, and an output signal of the manipulator when operated by the input signal; a calculation unit that calculates a mass of the manipulator during an operation of the manipulator and a spring coefficient of the manipulator during the operation of the manipulator using the input signal and the output signal; and an identification unit that identifies an operation state of the manipulator by comparing respectively the mass of the manipulator before the operation of the manipulator and the spring coefficient of the manipulator before the operation of manipulator both of which the storage unit stores, with the mass of the manipulator during the operation of the manipulator and the spring coefficient of the manipulator both of which the calculation unit calculates.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a manipulator system for operating an object, a management device for managing the operating status of the manipulator, and a management method thereof.

Background Art

[0002] Remote manipulators are used to perform operations such as grasping or moving various objects. For example, there is a micromanipulator for handling minute objects such as cells that are difficult for humans to directly operate. In order to realize complex operations, it is conceivable to operate an object by a human remotely operating a manipulator. In such a manipulator that performs such remote operation, in order to perform more precise and delicate operations and to prevent damage to the object or mechanism, when the manipulator contacts the object or when operating the object, the manipulator presents a reaction force to the user, etc. It is required to notify the user that the manipulator is in contact with the object.

[0003] On the other hand, particularly when targeting small objects, it becomes difficult to directly install a sensor for detecting contact or a sensor for measuring the reaction force when gripping an object at the tip of the manipulator. Therefore, it is required to detect a state such as contact by some method and notify the user to enhance safety.

[0004] As a method for notifying the contact between the manipulator and the object, in Patent Document 1, a vibrator and a vibration detection sensor are attached to the manipulator, and when the deviation between the output frequency of the vibrator and the vibration frequency detected by the vibration detection sensor becomes a certain value or more, it is determined that contact has been made with the object, and it is possible to notify the user of the contact. In addition, as a means for notifying the user of the contact, methods using sound or light are disclosed.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] According to the technique disclosed in Patent Document 1, a user operating a manipulator can confirm whether the manipulator has come into contact with an object through sound or light.

[0007] On the other hand, there is a problem that the cost increases by mounting sensors for contact detection or actuators such as sensors that vibrate the manipulator or vibration detection sensors. In addition, in order to easily realize complex operations, it is required to provide more information to the operator, such as presenting that the manipulator is grasping and moving the object. However, in the method described in Patent Document 1, the information presented is limited to contact with the object, and complex operations cannot be easily realized.

[0008] An object of the present invention is to enable a manipulator to detect not only the contact state but also the gripping state with respect to an object.

Means for Solving the Problems

[0009] The present invention relates to a manipulator system including drive means for driving a manipulator and measurement means for measuring the position of the manipulator. The system comprises a storage unit for storing the mass of the manipulator before manipulator operation and the spring constant of the manipulator before manipulator operation, an input signal for driving the manipulator, a reception unit for receiving an output signal of the manipulator operated by the input signal, a calculation unit for calculating the mass of the manipulator during manipulator operation and the spring constant of the manipulator during manipulator operation using the input signal and the output signal, and a specification unit for comparing the mass of the manipulator before manipulator operation and the spring constant of the manipulator before manipulator operation stored in the storage unit with the mass of the manipulator during manipulator operation and the spring constant of the manipulator during manipulator operation calculated by the calculation unit, respectively, and specifying the operation state of the manipulator.

[0010] Alternatively, the present invention relates to a management device for managing the operation state of a manipulator. The device comprises a storage unit for storing the mass of the manipulator before manipulator operation and the spring constant of the manipulator before manipulator operation, an input signal for driving the manipulator, a reception unit for receiving an output signal of the manipulator operated by the input signal, a calculation unit for calculating the mass of the manipulator during manipulator operation and the spring constant of the manipulator during manipulator operation using the input signal and the output signal, and a specification unit for comparing the mass of the manipulator before manipulator operation and the spring constant of the manipulator before manipulator operation stored in the storage unit with the mass of the manipulator during manipulator operation and the spring constant of the manipulator during manipulator operation calculated by the calculation unit, respectively, and specifying the operation state of the manipulator.

[0011] Alternatively, the present invention relates to a management method for managing the operating status of a manipulator. The method includes storing the mass of the manipulator before the manipulator operates and the spring coefficient of the manipulator before the manipulator operates, receiving an input signal for driving the manipulator and an output signal of the manipulator operated by the input signal, using the input signal and the output signal to calculate the mass of the manipulator during the manipulator operation and the spring coefficient of the manipulator during the manipulator operation, comparing the stored mass of the manipulator before the manipulator operates and the stored spring coefficient of the manipulator before the manipulator operates with the calculated mass of the manipulator during the manipulator operation and the calculated spring coefficient of the manipulator during the manipulator operation respectively, and identifying the operating status of the manipulator.

Advantages of the Invention

[0012] According to the present invention, in a manipulator, it is possible to detect a gripping state in addition to a contact state with respect to an object.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Embodiments for Carrying Out the Invention

[0014] Hereinafter, embodiments of a manipulator system according to the present disclosure will be described with reference to the drawings and the like. The following description shows specific examples of the content of the present invention, and the present invention is not limited to these descriptions, and various changes and modifications can be made by those skilled in the art within the scope of the technical idea disclosed in this specification.

Examples

[0015] FIG. 1 shows a configuration example of a manipulator system according to an embodiment of the present invention. The manipulator system includes a drive unit 1, an end effector 2, and a computer 4, and is a device for operating an object to be operated 3. The portion surrounded by the dotted line is shown enlarged. The drive unit 1 includes a drive unit output angle measurement unit 11 that measures the angle of the drive unit 1 and can measure the output angle.

[0016] The manipulator system is a device for operating a minute object that is difficult to directly perform delicate operations with a human hand. Examples of the object to be operated 3 include a container for containing cells.

[0017] The manipulator system is driven using remote operation by a human, and includes a remote operation device drive unit 5 and a remote operation device operation unit 6 for receiving remote operation, and observation means 7 for observing the object to be operated, such as an optical microscope or an electron microscope. The remote operation device also includes a remote operation device drive unit angle measurement unit 51 that measures the output angle of the remote operation device drive unit 5 and can measure the output angle. The user manually operates the remote operation device operation unit 6 while confirming the image of the object to be operated 3 observed by the observation means 7. In response to the operation, the remote operation device drive unit 5 operates, and the output angle is acquired by the remote operation device drive unit angle measurement unit 51. The drive unit 1 of the manipulator system operates according to the operation amount acquired by the remote operation device drive unit angle measurement unit 51.

[0018] The manipulator system may not only receive operation commands by remote control, but also automatically perform determined operations, or automatically generate operation commands from information such as images acquired by the observation means 7 and perform operations according to the operation commands.

[0019] The computer 4 is a management device that manages the operating status of the manipulator. The computer 4 includes a reception unit 41 that receives an input signal to the drive unit 1 and the output angle of the drive unit 1, a calculation unit 42 that calculates the mass and spring coefficient during the operation of the manipulator system based on the signal received by the reception unit, a storage unit 43 that stores and holds the mass and spring coefficient of the manipulator system, a specification unit 44 that compares the mass and spring coefficient estimated by the calculation unit 42 with the mass and spring coefficient held by the storage unit 43 to specify the operating state, a calculation signal generation unit 45 that adds a signal for assisting the parameter estimation of the calculation unit to the input signal to the drive unit, an output unit 46 that presents the state to the user based on the operating state specified by the specification unit 44, and a control unit 47 that controls the output of the manipulator system and controls the remote operation device of the user based on the signal of the generated operation command by the remote operation of the user.

[0020] FIG. 2 is a flowchart showing the flow of state determination in the manipulator system to which the present invention is applied. The flow of processing based on FIG. 2 is as follows.

[0021] In step 201, hardware modeling of the manipulator system is performed. For example, the transfer characteristics from the torque τ to the output angle θ of the drive unit 1 of the manipulator system are modeled as in Equation 1.

[0022]

Equation

[0023] In Equation 1, s represents the Laplace operator, G represents the gain, M represents the mass, D represents the damping, and K represents the spring coefficient.

[0024] The torque is the torque output by the drive unit 1 in proportion to the signal input to the drive unit 1. The model of Equation 1 is merely an example, and it is desirable to use a model that is considered suitable for expressing the characteristics of the system to which the present invention is applied. For example, the entire control system may be modeled, and the modeling may be performed from the command values of the angle and force to the response values. The modeling in Step 201 is performed only once as a preliminary preparation for performing the work in the manipulator system.

[0025] In Step 202, next, before actually operating the object to be operated 3 using the manipulator system, a method for parameter identification is determined in order to identify each parameter of Equation 1. The process of Step 202 is performed only once after the modeling in Step 201 is completed. An example of a method for estimating the parameters in the calculation unit is shown below.

[0026] When the model of Equation 1 is converted into a model of a discrete system that can be processed by a computer using bilinear transformation, it becomes as shown in Equation 2.

[0027]

Equation

[0028] In Equation 2, z is the z-operator, k is the discretized time, A0 to A2 and B0 to B2 are coefficients, and each coefficient is represented by the following Equation 3 to Equation 8.

[0029]

Equation

[0030]

Equation

[0031]

Equation

[0032]

Number

[0033]

Number

[0034]

Number

[0035] From number 3 to number 8, Ts represents the sampling period of the control system.

[0036] Next, a method for estimating the parameters shown from number 3 to number 8 in real time will be described. Define the parameter vector P and the data vector D as in numbers 9 and 10.

[0037]

Number

[0038]

Number

[0039] At this time, the estimated value Pe of the parameter at a certain time k can be expressed as in number 11.

[0040]

Number

[0041] In number 11, V[k] is a covariance matrix and is defined in number 12.

[0042]

Number

[0043] The initial value of the estimated value Pe of the parameter is set to zero. Also, it is effective to set the initial value of the covariance matrix to the identity matrix I and the coefficient to γI with the coefficient being γ. At this time, γ is preferably determined empirically according to the characteristics of the manipulator system. A method of sequentially estimating parameters such as Equation 11 and Equation 12 is known as the sequential least squares method.

[0044] From Equation 11, the parameter represented by Equation 9 is estimated. Using the parameter estimated from Equation 11 and the relational expressions of Equations 3 to 8, each coefficient shown in Equations 3 to 8 can be calculated.

[0045] In Equations 3 to 5, the mass M, the damping D, and the spring constant K exist as variables, and since the number of unknown variables is the same as the number of equations, it is possible to calculate the mass M, the damping D, and the spring constant K using a system of simultaneous equations. Also, the gain G can be calculated using any one of Equations 6 to 8. By the method described so far, it is possible to calculate the mass M and the spring constant K of the manipulator system from each parameter.

[0046] Although the method of estimating the parameter has been described so far, the method of estimating the parameter in the present invention is not limited to the above description, and it is preferable to select an appropriate method according to the modeling method.

[0047] In step 203, based on the model created in the above step, the manipulator system is operated to estimate the parameters of the manipulator itself and hold them in the storage unit 43. At this time, the operation of the object to be operated is not performed, and the manipulator system is operated without touching the object. In the above operation, a signal having a wide range of frequency components, for example, an M-sequence signal, is used as an input to the manipulator.

[0048] The manipulator is driven using the M series signal as an input signal, and the reception unit receives the input signal and the output angle of the manipulator. Based on the signal received by the reception unit, the calculation unit sequentially estimates the parameters by the method described in step 202.

[0049] In step 203, the manipulator system is operated for a predetermined time, and the average values of the estimated values of the mass M, damping D, spring constant K, and gain G estimated from the parameter estimation values at each time are held in the storage unit 43 as parameters in the non-contact state of the manipulator system. The parameter identification in step 203 is performed only once before performing work with the manipulator system.

[0050] In step 204, the manipulator system operates on the object to be targeted. As an example of operating the object, a method of driving the manipulator system by remote operation by a human will be described. Hereinafter, steps 204 to 206 are repeatedly processed when performing remote operation.

[0051] Fig. 7 shows the signal flow inside the computer 4 in steps 204 to 206. Hereinafter, the description will be made using Fig. 7 in combination. A human uses a remote operation device to position the manipulator system and operate the object to be operated. This operation is performed by the control unit 47 determining the input signal 401 to the drive unit 1 using the output angle 402 of the drive unit 1. Next, the calculation unit 42 sequentially performs parameter identification from the input signal to the drive unit 1 acquired by the reception unit 41 and the output angle 403 of the drive unit 1. By calculating the spring constant and mass in real time sequentially while the manipulator system is operating, the operating state of the manipulator can be grasped in a timely manner. Also, in the control unit 47, based on the measured position of the manipulator, an input signal for operating to a desired position is calculated. It is provided with a remote operation unit that receives an operation by a human, and based on the operation of the remote operation unit, the control unit 47 can calculate a command for the position or force output by the manipulator.

[0052] At this time, when the frequency components included in the signal input from the control unit 47 to the actuator are limited, it is known that parameter identification may not be performed stably. Therefore, it is preferable that the calculation signal generation unit 45 adds an M-sequence signal 407 having a signal magnitude that does not interfere with the remote operation by a human to the input obtained by the remote operation to drive the manipulator system. By adding such a signal that does not interfere with the remote operation, parameter identification can be stably performed.

[0053] In step 205, according to the operation in step 204, the specific unit 44 determines the operation state. The specific unit 44 determines the operation state of the manipulator system according to the parameter estimation result 404 in the calculation unit 42. The specific unit 44 discriminates three states: non-contact, contact, and gripping.

[0054] FIG. 3 shows an example of a manipulator system in a non-contact state. The non-contact state means that the manipulator and the object to be operated do not touch each other, and the operation of the manipulator does not affect the object to be operated.

[0055] FIG. 4 shows an example of a manipulator system in a contact state. The contact state means that the manipulator and the object to be operated are in contact, but they do not operate integrally. In FIG. 4, although the end effector is operating in the X direction, the object to be operated is not operating.

[0056] FIG. 5 shows an example of a manipulator system in a gripping state. The gripping state means that the manipulator grips the object to be operated, and the object to be operated also operates integrally according to the operation of the manipulator. In FIG. 5, the manipulator grips the object to be operated, and the end effector moves in the X direction and the object to be operated also moves in the X direction integrally.

[0057] Next, a method for determining the operating state using the parameters estimated in step 204 will be described. When in the non-contact state, the mass and spring constant estimated in step 204 are almost the same as the value 405 stored in the storage unit 43. Therefore, when the difference between the estimated mass and spring constant and the mass and spring constant stored in the storage unit 43 is smaller than certain thresholds Kth and Mth, the manipulator system determines that it is in a non-contact state with the object 3 to be operated on.

[0058] When in the contact state, since the output angle for the same force becomes smaller, the spring constant K of the manipulator system increases. When the difference between the spring constant K estimated in step 204 and the spring constant K of the manipulator system stored in the storage unit 43 is greater than a certain threshold Kth, the manipulator system determines that it is in contact with the object 3 to be operated on.

[0059] When in the gripping state, since the object 3 to be operated on moves integrally with the manipulator system, the mass M increases. When the difference between the mass estimated in step 204 and the mass of the manipulator system stored in the storage unit 43 is greater than a certain threshold Mth, the manipulator system determines that it is gripping the object 3 to be operated on.

[0060] As a method for determining the spring constant threshold Kth and the mass threshold Mth, a method determined in advance according to the type of the object 3 to be operated on or a method determined by performing a test in advance is effective. As described above, the specific unit 44 can distinguish the three states of non-contact, contact, and gripping.

[0061] In addition to the above discrimination method, it is also effective to use the gain G and the attenuation D for discrimination as an auxiliary method. For example, in the contact state, the output of the manipulator system becomes small, so the gain G becomes smaller compared to the non-contact state. Therefore, when the difference from the gain of the manipulator system stored in the memory unit becomes smaller than a certain threshold Gth, it can be determined that the state is the contact state. In order to improve the accuracy and reliability of the state determination, it is effective to perform the determination in combination with the change in the spring constant.

[0062] Also, in the gripping state, the manipulator system and the object to be manipulated operate integrally, and since the viscous resistance during operation increases, the attenuation D increases. Therefore, when the difference from the gain of the manipulator system stored in the memory unit becomes larger than a certain threshold Dth, it can be determined that the state is the gripping state. Also in this case, in order to increase the reliability of the state determination, it is effective to perform the determination in combination with the change in mass.

[0063] The method for specifying the operating state in the specific unit 44 is not limited to the above, and it is preferably determined according to the type of the manipulator system and the object to be manipulated to which the present invention is applied.

[0064] In step 206, according to the operation state 406 specified in step 205, the presentation information 408 to the user is switched. There are mainly two types of presentation information to the user. One is the force sensation information presented to the user via the remote operation device, and the other is the visual information presented via the image.

[0065] First, the presentation of the force sensation information will be described. The force sensation information is presented to the user by controlling the remote operation device drive unit 5. In the non-contact state, the device for performing the remote operation operates as the user operates, and is controlled so as not to interfere with the user's operation without presenting a reaction force or the like in particular.

[0066] In the contact state, a reaction force is presented via a device for remote operation so that an operation that may damage the object to be operated after contact is not performed. At this time, it is effective to present a reaction force having a magnitude proportional to the difference between the determined contact position and the current position.

[0067] In the gripping state, while presenting the reaction force presented in the contact state, it is displayed that the object is in the gripping state on the screen for observing the object. This display will be described later. By this display, it is possible to recognize that the object is being gripped while applying an appropriate force to the device for remote operation, enabling a safe operation.

[0068] Next, the presentation of visual information will be described. FIG. 3 shows an example of a screen for displaying information presented to the user. The screen of the information presentation unit 100 for presenting information to the user is composed of an object display unit 101 that displays an image of the object to be operated and the manipulator system acquired by the observation means 7, and a state display unit 102 that displays the current state of the manipulator system.

[0069] The object display unit 101 always displays the image obtained by the observation means 7 to present the situation around the object to be operated to the user. In the state display unit 102, it is possible to visually determine the state of the manipulator system by displaying, by visual means such as characters, whether the manipulator system is in a non-contact, contact, or gripping state. It is also effective to improve visibility by using not only characters but also colors. It is also effective to display the time change of physical quantities such as the position of the manipulator system as a graph.

[0070] When step 206 is completed, if the operation of the object is to be continued, the process returns to step 204 and the repetitive process is performed until the operation of the manipulator system is completed. When the operation of the object is terminated, the series of processes is also terminated. From step 204 to step 206, it is preferable to perform calculations at the earliest possible cycle in order to present information to the user quickly.

[0071] So far, the switching of the presentation information to the user by the control unit 47 has been described. However, it is also effective to switch the control mode in the control unit 47 according to the specified operating state. For example, in the non-contact state, it is preferable that the manipulator system follows the user's operation and performs accurate positioning. On the other hand, in the contact or gripping state, since the manipulator system is in contact with the object to be operated, it is also effective to switch to force control in which the force output by the manipulator system is controlled by the specifying unit so as not to damage the object to be operated. At this time, it is preferable to confirm in advance whether there is an influence on the parameter estimation in the calculation unit due to the switching of the control unit 47.

[0072] As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and even if there are design changes and the like within the scope not departing from the gist of the present invention, they are included in the present invention. Note that the manipulator system of the present disclosure is not limited to the manipulator of this embodiment.

Explanation of Signs

[0073] 1 ··· Driving unit 2 ··· End effector 3 ··· Object to be operated 4 ··· Computer 5 ··· Remote operation device driving unit 6 ··· Remote operation device operation unit 7 ··· Observation means 11 ··· Driving unit output angle measurement unit 41 ··· Reception unit 42 ··· Calculation unit 43 ··· Storage unit 44 ··· Specifying unit 45 ··· Calculation signal generation unit 46 ··· Output unit 47 ··· Control unit 51 ··· Remote operation device driving unit angle measurement unit 100 ··· Information presentation unit 101 ··· Object to be operated display unit 102 ··· Status display unit

Claims

1. In a manipulator system comprising drive means for driving a manipulator and measurement means for measuring the position of the manipulator, a storage unit that stores the mass of the manipulator before manipulator operation and the spring constant of the manipulator before manipulator operation; a reception unit that receives an input signal for driving the manipulator and an output signal of the manipulator operated by the input signal; a calculation unit that calculates the mass of the manipulator during manipulator operation and the spring constant of the manipulator during manipulator operation using the input signal and the output signal; a specifying unit that compares the mass of the manipulator before manipulator operation and the spring constant of the manipulator before manipulator operation stored in the storage unit with the mass of the manipulator during manipulator operation and the spring constant of the manipulator during manipulator operation calculated by the calculation unit, respectively, and specifies the operation status of the manipulator. A manipulator system characterized by comprising:

2. In the manipulator system according to Claim 1, a control unit that determines a control mode of the manipulator using the operation status of the manipulator specified by the specifying unit; A manipulator system characterized by comprising an output unit that outputs the control mode of the manipulator to the manipulator according to the operation status of the manipulator specified by the specifying unit.

3. In the manipulator system according to Claim 1, a control unit that determines presentation information to the user using the operation status of the manipulator specified by the specifying unit; A manipulator system characterized by comprising an output unit that outputs the presentation information to the information presentation unit of the user according to the operation status of the manipulator specified by the specifying unit.

4. In the manipulator system according to Claim 1, The calculation unit calculates parameters of a physical model of the manipulator system modeled in advance using the input signal for driving the manipulator and the output signal of the manipulator. A manipulator system characterized by this.

5. In the manipulator system according to Claim 1, A manipulator system, comprising a calculation signal generation unit that outputs a signal for stably calculating the spring coefficient and mass in the calculation unit. **Claim 6** In the manipulator system according to claim 1, the calculation unit is characterized in that it calculates the spring coefficient and mass in real time sequentially while the manipulator system is operating. **Claim 7** In the manipulator system according to claim 1, a control unit that calculates the input signal for operating to a desired position based on the measured position of the manipulator, characterized in that the manipulator system is provided. **Claim 8** In the manipulator system according to claim 1, a control unit that calculates the input signal for operating to a desired position based on the measured position of the manipulator, and a remote operation unit that receives a human operation, and based on the operation of the remote operation unit, the control unit calculates a position or force command output by the manipulator. **Claim 9** In the manipulator system according to claim 1, an observation unit that observes an object to be manipulated by the manipulator, and a presentation unit that can present an image or video of the object to be manipulated acquired by the observation unit to the user, characterized in that the manipulator system is provided. **Claim 10** In a management device that manages the operation status of a manipulator, a storage unit that stores the mass of the manipulator before the manipulator operation and the spring coefficient of the manipulator before the manipulator operation, a reception unit that receives an input signal for driving the manipulator and an output signal of the manipulator operated by the input signal, a calculation unit that calculates the mass of the manipulator during the manipulator operation and the spring coefficient of the manipulator during the manipulator operation using the input signal and the output signal, a specific unit that compares the mass of the manipulator before the manipulator operation and the spring coefficient of the manipulator before the manipulator operation stored in the storage unit with the mass of the manipulator during the manipulator operation and the spring coefficient of the manipulator during the manipulator operation calculated by the calculation unit, respectively, and specifies the operation status of the manipulator. **Claim 11** In a management device for managing the operating status of the manipulator according to claim 10, A management device comprising a control unit that determines a control mode of the manipulator or presentation information to a user using the operating status of the manipulator specified by the specifying unit.

12. In a management method for managing the operating status of a manipulator, Storing the mass of the manipulator before the manipulator operation and the spring constant of the manipulator before the manipulator operation, Receiving an input signal for driving the manipulator and an output signal of the manipulator operated by the input signal, Using the input signal and the output signal to calculate the mass of the manipulator during the manipulator operation and the spring constant of the manipulator during the manipulator operation, Comparing the stored mass of the manipulator before the manipulator operation and the spring constant of the manipulator before the manipulator operation with the calculated mass of the manipulator during the manipulator operation and the spring constant of the manipulator during the manipulator operation, respectively, and specifying the operating status of the manipulator.

Images