Micromanipulator system, and micromanipulator system control method

Abstract

To provide a micromanipulator system and micromanipulator system control method that can efficiently make a position adjustment of a micro tool.SOLUTION: A micromanipulator system 1 is a micromanipulator system that is used together with a microscope 10 capable of observing an observation target object S, and that comprises: manipulators 40 that each include a micro tool 41, and that can move the micro tool 41; an acquisition unit 20 that can acquire a microscope photograph at the microscope 10; and a control unit 50 that can control the manipulators 40. The control unit 50 is configured to acquire a tool image being the microscope photograph of the one or more micro tools 41 the acquisition unit 20 acquires, and change a position of each micro tool 41 on the basis of the tool image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a micromanipulator system and a method for controlling the micromanipulator system.

Background Art

[0002] Conventionally, in the medical field, foreign object analysis, etc., various processes may be performed on a fine sample using a micro tool. Examples of processes using a micro tool include, for example, intracytoplasmic sperm injection. In intracytoplasmic sperm injection, sperm held at the tip of a micro tool are directly injected into an egg.

[0003] When performing a fine operation such as intracytoplasmic sperm injection, an operator places a petri dish containing a sample such as an egg on a microscope and performs a fine operation while checking the sample and the micro tool with the microscope. Further, in such a fine operation, a micromanipulator device capable of moving the micro tool is used (for example, Patent Document 1). The operator operates the micromanipulator device while checking the sample and the micro tool with the microscope to perform a fine operation on the sample.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In a fine operation using a microscope, an operator needs to focus the microscope on the sample and the micro tool. For example, the operator first focuses the microscope on the sample. Next, the operator moves the position of the micro tool so that the micro tool can be confirmed with the microscope focused on the sample.

[0006] However, the operation of adjusting the position of the micro tool requires moving the tip of the micro tool within the field of view of the microscope. Also, since the size of the tip of the micro tool is extremely small, on the order of several micrometers, it is a difficult operation. Therefore, there was a concern that the operator would require a great deal of labor for the operation of adjusting the position of the micro tool.

[0007] In view of the above circumstances, an object of the present invention is to provide a micromanipulator system and a control method for a micromanipulator system that can efficiently adjust the position of a micro tool.

Means for Solving the Problem

[0008] In order to solve the above problems, the present invention proposes the following means. The micromanipulator system of the present invention is a micromanipulator system used together with a microscope capable of observing an object to be observed, having a micro tool, a manipulator capable of moving the micro tool, an acquisition unit capable of acquiring a micrograph in the microscope, and a control unit capable of controlling the manipulator. The control unit acquires a tool image, which is the micrograph of the micro tool acquired by the acquisition unit, and changes the position of the micro tool based on the tool image.

[0009] The control method for a micromanipulator system of the present invention is a control method for a micromanipulator system used together with a microscope capable of observing an object to be observed, including a tool position changing step of acquiring a tool image, which is a micrograph of a micro tool in the microscope, and changing the position of the micro tool based on the tool image.

Effect of the Invention

[0010] According to the micro manipulator system and the control method of the micro manipulator system of the present invention, it is possible to provide a micro manipulator system and a control method of the micro manipulator system that can efficiently adjust the position of the micro tool.

Brief Description of Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Embodiments for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram schematically showing the configuration of the micro manipulator system 1 according to the present embodiment. FIG. 2 is a block diagram schematically showing the micro manipulator system 1 according to the present embodiment.

[0013] The micro manipulator system 1 includes a microscope 10, an acquisition unit (imaging unit) 20, a focus drive unit 30, a manipulator 40, a control unit 50, and an input unit 60.

[0014] The microscope 10 is a microscope capable of observing an object to be observed S, such as cells or eggs, at a predetermined magnification. As the microscope 10, a known microscope can be adopted. The microscope 10 has a table on which a container such as a petri dish containing a sample (object to be observed S) can be placed. An operator can observe the object to be observed S in the container placed on the table of the microscope 10 at a predetermined magnification by using the microscope 10.

[0015] The acquisition unit (imaging unit) 20 can acquire a micrograph in the microscope 10 and is, for example, an imaging device such as a camera capable of taking a micrograph. Here, the micrograph refers to an image obtained by photographing the actual field of view of the microscope 10. The acquisition unit 20 transmits the acquired micrograph to a control unit 50 described later.

[0016] The acquisition unit 20 may also take a video of the actual field of view of the microscope 10 and acquire a predetermined frame (image) extracted from the taken video as a micrograph.

[0017] When the microscope 10 is equipped with a camera, the acquisition unit 20 may be provided in the control unit 50 and configured to be able to acquire a micrograph from the camera provided in the microscope 10. In that case, the acquisition unit 20 may also acquire a video obtained by photographing the actual field of view of the microscope 10 from the camera provided in the microscope 10 and acquire a predetermined frame (image) extracted from the video obtained from the camera provided in the microscope 10 as a micrograph.

[0018] The focus drive unit 30 is a drive device capable of changing the focus of the microscope 10. For example, when the microscope 10 is equipped with a focusing device for focusing on an object to be observed S placed on the table, the focus drive unit 30 can operate the focusing device provided in the microscope 10 to change the focus of the microscope 10. Changing the focus of the microscope 10 means, for example, changing the distance from the focus of the objective lens of the microscope 10 to the object to be observed S.

[0019] By changing the focusing of the microscope 10 by the focus drive unit 30, for example, the distance from the objective lens of the microscope 10 to the observation object S can be changed, and the focal position of the microscope 10 can be adjusted to the observation object S. The focus drive unit 30 may change the focusing of the microscope 10 by switching the objective lens of the microscope 10 that observes the observation object S to an objective lens with a different focal length.

[0020] The manipulator 40 includes a micro tool 41, a movable part 42, and a pump 43. The manipulator 40 is an electric manipulator driven electrically.

[0021] The micro tool 41 is a micro tool that can be used for fine work. The micro tool 41 is, for example, a micro tool that can be used for intracytoplasmic sperm injection, and is a pipette with a tip diameter of about several μm.

[0022] The movable part 42 is connected to the micro tool 41, can move the micro tool 41 in any direction, and is a movable device driven electrically. The movable part 42 is, for example, an XYZ-axis table, a linear motion mechanism composed of an electric motor and a ball screw, etc. The movable part 42 only needs to be able to move the micro tool 41 in any direction such as the vertical direction or the horizontal direction, and may be a multi-joint robot arm.

[0023] The pump 43 is a pump device such as a syringe pump connected to the micro tool 41. When the micro tool 41 is a pipette, by driving the pump 43, the inside of the micro tool 41 can be made into a negative pressure or a positive pressure. For example, when the inside of the micro tool 41 is made into a negative pressure by the pump 43, the observation object S can be adsorbed to the tip of the micro tool 41, and the observation object S can be held by the micro tool 41. Also, when the inside of the micro tool 41 is made into a positive pressure by the pump 43, the observation object S held by adsorbing the observation object S can be released from the micro tool 41.

[0024] The micromanipulator system 1 includes two manipulators 40. Of the two manipulators 40, one is also referred to as the first manipulator 40A, and the other is also referred to as the second manipulator 40B.

[0025] Also, the micro tool 41 included in the first manipulator 40A is also referred to as the first micro tool 41A, and the micro tool 41 included in the second manipulator 40B is also referred to as the second micro tool 41B.

[0026] Also, the movable part 42 included in the first manipulator 40A is also referred to as the first movable part 42A, and the movable part 42 included in the second manipulator 40B is also referred to as the second movable part 42B.

[0027] Also, the pump 43 included in the first manipulator 40A is also referred to as the first pump 43A, and the pump 43 included in the second manipulator 40B is also referred to as the second pump 43B.

[0028] The first manipulator 40A and the second manipulator 40B are independent of each other. Therefore, the first movable part 42A and the second movable part 42B can move independently of each other, and the first movable part 42A and the second movable part 42B can move the first micro tool 41A and the second micro tool 41B in different directions from each other. Also, the first pump 43A and the second pump 43B can be driven independently of each other.

[0029] The control unit 50 is a control device capable of controlling part or all of the micromanipulator system 1. The control unit 50 is connected to the focus drive unit 30 and the manipulator 40 by wire or wirelessly, and can control the focus drive unit 30 and the manipulator 40.

[0030] The control unit 50 can control the movable part 42 and the pump 43 of the manipulator 40. Also, the control unit 50 can control the first manipulator 40A and the second manipulator 40B respectively.

[0031] The control unit 50 is connected to the acquisition unit 20 by wire or wirelessly, and can acquire the microscopic photograph acquired by the acquisition unit 20 from the acquisition unit 20. The control unit 50 can control the acquisition unit 20.

[0032] Further, the control unit 50 may include the acquisition unit 20. When the microscope 10 is provided with an imaging device such as a camera, the control unit 50 is connected to the microscope 10 by wire or wirelessly, and the acquisition unit 20 included in the control unit 50 acquires a microscopic photograph from the microscope 10.

[0033] The control unit 50 is, for example, a program-executable device (computer) including a processor, a memory, a storage unit, etc. Each function of the control unit 50 is realized, for example, by one or more processors such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) executing a program stored in a program memory. However, all or part of these functions may be realized by hardware (for example, a circuit unit) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a PLD (Programmable Logic Device). Further, all or part of the above functions may be realized by a combination of software and hardware. The storage unit is realized by a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), a ROM (Read-Only Memory), a RAM (Random Access Memory), or the like.

[0034] The input unit 60 is an input device such as a keyboard, mouse, lever, dial, knob, or joystick that is connected to the control unit 50 by wire or wirelessly. An operator can input various operations to the control unit 50 by operating the input unit 60.

[0035] For example, the operator can input various numerical values, symbols, or character strings, etc. used by the control unit 50 to control the micromanipulator system 1 to the control unit 50 via the input unit 60.

[0036] In addition, the operator can operate the manipulator 40 using the input unit 60. The control unit 50 controls the manipulator 40 based on the operation input to the input unit 60. For example, the control unit 50 controls the movable part 42 based on the operation input to the input unit 60, and moves the position of the micro tool 41 connected to the movable part 42.

[0037] The operator can operate the manipulator 40 using the input unit 60 and perform fine operations such as micro-insemination on the observation object S using the micro tool 41.

[0038] Next, a control method for the micromanipulator system 1 will be described. FIG. 3 is a flowchart showing a control method for the micromanipulator system 1.

[0039] (Step S100) When the micromanipulator system 1 is activated, the micromanipulator system 1 performs step S100 (initial value acquisition step). In step S100, the control unit 50 acquires initial values necessary for the operation of the micromanipulator system 1.

[0040] Examples of the initial values acquired by the control unit 50 in step S100 include the lens magnification and illumination information of the microscope 10, the diameter of the observation object S such as an egg, and the tip diameters of the micro tools 41 (the first micro tool 41A and the second micro tool 41B) such as a pipette.

[0041] In step S100, the control unit 50 may acquire an initial value pre-stored in a storage unit or the like of the control unit 50. Further, in step S100, the control unit 50 may receive an input of an initial value from the input unit 60.

[0042] (Step S200) Next, the micromanipulator system 1 performs step S200 (focus adjustment process). At this time, on the table of the microscope 10, a container such as a petri dish containing a plurality of observation objects S is placed, and a predetermined observation object S among the plurality of observation objects S is suction-held by the first micro tool 41A at the tip of the first manipulator 40A.

[0043] FIG. 4 is a flowchart showing the focus adjustment process S200 in the control method of the micromanipulator system 1. In step S200, the micromanipulator system 1 first performs step S210.

[0044] (Step S210) In step S210, the control unit 50 acquires a microscopic photograph from the acquisition unit 20. When the acquisition unit 20 is an imaging device such as a camera, the acquisition unit 20 acquires a microscopic photograph in the microscope 10 by taking a picture. The acquisition unit 20 transmits the taken microscopic photograph to the control unit 50.

[0045] The microscopic photograph acquired by the control unit 50 in step S210 is a microscopic photograph taken of the observation object S from above. Here, the microscopic photograph of the observation object S is referred to as an "observation object image". The observation object image is a microscopic photograph when the observation object S is included in the actual field of view of the microscope 10. Further, the observation object image acquired by the control unit 50 in step S210 is also referred to as a "first image".

[0046] (Step S220) Next, the micromanipulator system 1 performs step S220. In step S220, the control unit 50 performs image processing on the first image acquired in step S210 and extracts the contour of the observation object S in the first image.

[0047] Specifically, the control unit 50 first performs grayscale conversion on the first image. Next, the control unit 50 performs binarization on the first image. The control unit 50 extracts the contour of the observation object S from the first image that has undergone grayscale conversion and binarization, and based on the extracted contour of the observation object S, obtains the diameter of the observation object S in the first image.

[0048] (Step S230) Next, the micromanipulator system 1 performs step S230. In step S230, the control unit 50 changes the focusing of the microscope 10 based on the contour of the observation object S extracted from the first image. The control unit 50 controls the focus driving unit 30 to change the focusing of the microscope 10 so that the diameter of the observation object S within the actual field of view of the microscope 10 approaches the diameter of the observation object S that is one of the initial values acquired in step S100.

[0049] (Step S240) Next, the micromanipulator system 1 performs step S240. In step S240, the control unit 50 acquires an observation object image from the acquisition unit 20 in the same manner as in step S210. Here, the observation object image acquired by the control unit 50 in step S240 is also referred to as the "second image". The second image is an observation object image acquired after the control unit 50 changes the focusing of the microscope 10 based on the first image.

[0050] Also, the control unit 50 performs grayscale conversion and binarization on the second image in the same manner as in step S220, extracts the contour of the observation object S in the second image, and based on the extracted contour of the observation object S, obtains the diameter of the observation object S in the second image.

[0051] In step S240, the control unit 50 determines whether the diameter of the observation object S in the second image is close to the diameter of the observation object S obtained in step S100.

[0052] For example, the control unit 50 determines whether the diameter of the observation object S in the second image is close to the diameter of the observation object S obtained in step S100 by using a threshold value. At this time, the control unit 50 may use a value pre-stored in a storage unit or the like of the control unit 50 as the threshold value, or may use the value input from the input unit 60 in step S100 as the threshold value.

[0053] When the control unit 50 determines that the diameter of the observation object S in the second image is a value close to the diameter of the observation object S obtained in step S100, the micromanipulator system 1 proceeds to step S250.

[0054] (Step S250) In step S250, the control unit 50 calculates and acquires the area of the observation object S in the second image. In addition, the control unit 50 acquires the focal position of the microscope 10 when the second image is acquired. The control unit 50 stores the acquired area of the observation object S and the focal position of the microscope 10. After performing the process of step S250, the micromanipulator system 1 proceeds to step S270.

[0055] Also, when the control unit 50 determines in step S240 that the diameter of the observation object S in the second image is not a value close to the diameter of the observation object S obtained in step S100, the micromanipulator system 1 proceeds to step S260.

[0056] (Step S260) In step S260, the control unit 50 determines whether there is a candidate for the contour of the observation object S in the second image obtained in step S240.

[0057] When the control unit 50 determines that there is no candidate for the contour of the object S to be observed in the second image, the micromanipulator system 1 returns to step S210 and repeats the subsequent steps.

[0058] When the control unit 50 determines that there is a candidate for the contour of the object S to be observed in the second image, the micromanipulator system 1 proceeds to step S270.

[0059] (Step S270) In step S270, the control unit 50 determines whether the amount of change when changing the focus of the microscope 10 in step S230 is equal to or greater than the radius of the object S to be observed. Here, the amount of change when changing the focus of the microscope 10 is, for example, the amount of movement of the focal position of the microscope 10 changed by the focus drive unit 30.

[0060] When the control unit 50 determines that the amount of change when changing the focus of the microscope 10 is smaller than the radius of the object S to be observed, the micromanipulator system 1 returns to step S210 and repeats the subsequent steps.

[0061] When the control unit 50 determines that the amount of change when changing the focus of the microscope 10 is equal to or greater than the radius of the object S to be observed, the micromanipulator system 1 proceeds to step S280.

[0062] (Step S280) As described above, in step S270, when the control unit 50 determines that the amount of change when changing the focus of the microscope 10 is smaller than the radius of the object S to be observed, the micromanipulator system 1 returns to step S210.

[0063] Therefore, the micromanipulator system 1 repeatedly executes the processes of steps S210 to S270 until the control unit 50 determines that the amount of change when changing the focus of the microscope 10 in step S270 is equal to or greater than the radius of the object S to be observed.

[0064] When the micro-manipulator system 1 performs the process of step S250 multiple times, the control unit 50 stores the area information of the observation object S for the number of times the process of step S250 is repeated.

[0065] In step S280, the control unit 50 acquires the area information with the largest area of the observation object S among the plurality of area information stored by the control unit 50. Also, the contour of the observation object S (the contour at the maximum area) when the area of the observation object S is the largest is acquired.

[0066] In the above control flow, after the control unit 50 determines whether the diameter of the observation object S in the second image in step S240 is close to the diameter of the observation object S acquired in step S100 using a predetermined threshold value, the control unit 50 acquires the area information of the observation object S in step S250.

[0067] Also, in step S270, the control unit 50 determines whether the amount of change when changing the focus of the microscope 10 in step S230 is equal to or greater than the radius of the observation object S. If it is not equal to or greater than the radius, the processes of steps S210 to S270 are repeated, and if it is equal to or greater than the radius, the process proceeds to step S280.

[0068] When the observation object S is spherical, by acquiring the area information while scanning within a range equal to or greater than the radius of the observation object S, the area information of the center position of the observation object S can be acquired without omission. That is, in step S280, the maximum area contour acquired by the control unit 50 is mainly the contour at the center position of the observation object S.

[0069] (Step S290) Next, the micro-manipulator system 1 performs step S290. In step S290, the control unit 50 acquires and outputs the focal position of the microscope 10 when the contour of the observation object S is the maximum area contour acquired in step S280.

[0070] The focal position of the microscope 1 acquired by the control unit 50 in step S290 is the focal position focused on the observation object S. Here, the focal position focused on the observation object S is referred to as the "observation object focal position". When the maximum area contour acquired in step S280 is the contour at the center position of the observation object S, the observation object focal position acquired in step S290 is the focal position focused on the center position of the observation object S.

[0071] (Step S300) Next, the micromanipulator system 1 performs step S300 (tool position changing process).

[0072] FIG. 5 is a flowchart showing the tool position changing process S300 in the control method of the micromanipulator system 1. In step S300, the micromanipulator system 1 first performs step S310.

[0073] (Step S310) In step S310, the control unit 50 acquires a microscope photograph from the acquisition unit 20. When the acquisition unit 20 is an imaging device such as a camera, the acquisition unit 20 acquires a microscope photograph in the microscope 1 by taking a picture. The acquisition unit 20 transmits the taken microscope photograph to the control unit 50.

[0074] The microscope photograph acquired by the control unit 50 in step S310 is a microscope photograph of the tip of the micro tool 41. Here, the microscope photograph of the tip of the micro tool 41 is referred to as the "tool image". The tool image is a microscope photograph when the tip of the micro tool 41 is included in the actual field of view of the microscope 1. Also, the tool image acquired by the control unit 50 in step S310 is also referred to as the "third image".

[0075] (Step S320) Next, the micromanipulator system 1 performs step S320. In step S320, the control unit 50 performs image processing on the third image acquired in step S310 and extracts the contour of the tip of the micro tool 41 in the third image.

[0076] Specifically, the control unit 50 first performs grayscale conversion on the third image. Next, the control unit 50 performs binarization on the third image. The control unit 50 extracts the contour of the tip of the micro tool 41 from the third image subjected to grayscale conversion and binarization, and obtains the diameter of the tip of the micro tool 41 in the third image based on the extracted contour of the tip of the micro tool 41.

[0077] (Step S330) Next, the micromanipulator system 1 executes step S330. In step S330, the control unit 50 changes the position of the tip of the micro tool 41 based on the contour of the tip of the micro tool 41 extracted from the third image. The control unit 50 controls the movable part 42 and changes the position of the tip of the micro tool 41 so that the tip of the micro tool 41 is in focus with the microscope 10.

[0078] At this time, the focal position of the microscope 10 is the observation object focal position obtained in step S200. That is, the control unit 50 moves the tip of the micro tool 41 to the focal position of the microscope 10 that is in focus with the observation object S based on the third image (tool image). At this time, the control unit 50 mainly changes the vertical height of the tip of the micro tool 41.

[0079] (Step S340) Next, the micromanipulator system 1 executes step S340. In step S340, the control unit 50 acquires a tool image from the acquisition unit 20 in the same manner as in step S310. Here, the tool image acquired by the control unit 50 in step S340 is also referred to as the "fourth image". The fourth image is a tool image acquired after the control unit 50 moves the tip position of the micro tool 41 based on the third image.

[0080] Further, in the same manner as in step S320, the control unit 50 performs grayscale conversion and binarization on the fourth image, extracts the contour of the tip of the micro tool 41 in the fourth image, and obtains the diameter of the tip of the micro tool 41 in the fourth image based on the extracted contour of the tip of the micro tool 41.

[0081] In step S340, the control unit 50 determines whether the diameter of the tip of the micro tool 41 in the fourth image is close to the diameter of the tip of the micro tool 41 obtained in step S100.

[0082] For example, the control unit 50 determines whether the diameter of the tip of the micro tool 41 in the fourth image is close to the diameter of the tip of the micro tool 41 obtained in step S100 using a threshold value. At this time, the control unit 50 may use a value pre-stored in a storage unit or the like of the control unit 50 as the threshold value, or may use the value input from the input unit 60 in step S100 as the threshold value.

[0083] When the control unit 50 determines that the diameter of the tip of the micro tool 41 in the fourth image is a value close to the diameter of the tip of the micro tool 41 obtained in step S100, the micromanipulator system 1 proceeds to step S350.

[0084] (Step S350) In step S350, the control unit 50 calculates and obtains the area of the tip of the micro tool 41 in the fourth image. Further, the control unit 50 obtains the tip position of the micro tool 41 when the fourth image is obtained. The control unit 50 stores the obtained area and position of the tip of the micro tool 41. After performing the process of step S350, the micromanipulator system 1 proceeds to step S370.

[0085] Also, when the control unit 50 determines in step S340 that the diameter of the tip of the micro tool 41 in the fourth image is not a value close to the diameter of the tip of the micro tool 41 obtained in step S100, the micromanipulator system 1 proceeds to step S360.

[0086] (Step S360) In step S360, the control unit 50 determines whether there is a candidate for the contour of the tip of the micro tool 41 in the fourth image acquired in step S340.

[0087] When the control unit 50 determines that there is no candidate for the contour of the tip of the micro tool 41 in the fourth image, the micromanipulator system 1 returns to step S310 and performs the subsequent steps again.

[0088] When the control unit 50 determines that there is a candidate for the contour of the tip of the micro tool 41 in the fourth image, the micromanipulator system 1 proceeds to step S370.

[0089] (Step S370) In step S370, the control unit 50 determines whether the amount of movement when changing the tip position of the micro tool 41 in step S330 is equal to or greater than the radius of the tip of the micro tool 41.

[0090] When the control unit 50 determines that the amount of movement when changing the tip position of the micro tool 41 is smaller than the radius of the tip of the micro tool 41, the micromanipulator system 1 returns to step S310 and performs the subsequent steps again.

[0091] When the control unit 50 determines that the amount of movement when changing the tip position of the micro tool 41 is equal to or greater than the radius of the tip of the micro tool 41, the micromanipulator system 1 proceeds to step S380.

[0092] (Step S380) As described above, in step S370, when the control unit 50 determines that the amount of movement when changing the tip position of the micro tool 41 is smaller than the radius of the tip of the micro tool 41, the micromanipulator system 1 returns to step S310.

[0093] Therefore, the micro-manipulator system 1 repeatedly executes the processes of steps S310 to S370 until the control unit 50 determines that the amount of movement when changing the tip position of the micro-tool 41 in step S370 is equal to or greater than the radius of the tip of the micro-tool 41.

[0094] When the micro-manipulator system 1 executes the process of step S350 multiple times, the control unit 50 stores the area information of the tip of the micro-tool 41 for the number of times the process of step S350 is repeated.

[0095] In step S380, the control unit 50 acquires the area information with the largest tip area of the micro-tool 41 among the plurality of area information stored by the control unit 50. Further, the contour of the tip of the micro-tool 41 when the tip area of the micro-tool 41 is the largest (the contour at the maximum area) is acquired.

[0096] (Step S390) Next, the micro-manipulator system 1 executes step S390. In step S390, the control unit 50 acquires and outputs the tip position of the micro-tool 41 when the contour of the tip of the micro-tool 41 is the contour at the maximum area acquired in step S380.

[0097] The tip position of the micro-tool 41 acquired by the control unit 50 in step S390 is located at the observation object focus position where the observation object S is in focus. That is, when the micro-manipulator system 1 finishes the process of step S390, the microscope 10 is in focus on both the observation object S and the tip of the micro-tool 41.

[0098] An operator can perform fine operations such as micro-injection using the micro-manipulator system 1 in a state where the microscope 10 is in focus on both the observation object S and the tip of the micro-tool 41 by the above-described control method of the micro-manipulator system 1.

[0099] By inputting an operation into the input unit 60, the operator moves the tip of the micro tool 41 horizontally to perform a micro operation. At this time, if the tip of the micro tool 41 is moved in the vertical direction, the tip of the micro tool 41 will deviate from the focus of the microscope 10. Therefore, it is preferable for the operator to perform the micro operation without moving the tip of the micro tool 41 in the vertical direction.

[0100] For example, the operator can hold an egg with the first micro tool 41A and inject sperm from the second micro tool 41B into the egg to perform in vitro fertilization. At this time, since the focus of the microscope 10 is aligned with the egg (observation object S) and the tip of the micro tool 41, the operator can sufficiently observe the micro tool 41, the egg, and the sperm to perform in vitro fertilization.

[0101] By executing the above control flow, the control unit 50 can place the tip of the micro tool 41 at a focus position where the focus is aligned with the center position of the observation object S.

[0102] For example, in in vitro fertilization, by holding a position close to the center of the egg with the micro tool 41 and injecting sperm into a position close to the center of the egg, the moment force applied to the egg can be reduced, and the stress on the egg can be reduced.

[0103] Also, in the picking operation of holding the observation object S with a micro tool 41 such as a capillary, by holding a position close to the center of the observation object S, the risk of the held observation object S falling when moving the observation object S can be reduced.

[0104] In the control method of the above-described micromanipulator system 1, by the control unit 50 acquiring the lens magnification of the microscope 10 in step S100, when the control unit 50 performs image processing on the microscope photograph, it is possible to obtain information on the distance from the pixels of the microscope photograph. Further, by the control unit 50 acquiring the illumination information of the microscope 10 in step S100, when the control unit 50 performs image processing on the microscope photograph, it is possible to perform image processing considering the influence of the illumination of the microscope 10, and there is an effect that the image processing result can be stabilized.

[0105] According to the micromanipulator system 1 of the present embodiment, it includes a manipulator 40 capable of moving the micro tool 41, an acquisition unit 20 capable of acquiring a microscope photograph in the microscope 10, and a control unit 50 capable of controlling the manipulator 40. The control unit 50 acquires the microscope photograph (tool image) of the micro tool 41 acquired by the acquisition unit 20, and changes the position of the micro tool 41 based on the tool image. The control unit 50 moves the tip of the micro tool 41 to the focal position of the microscope 10 based on the tool image.

[0106] Furthermore, the micromanipulator system 1 includes a focus drive unit 30 capable of changing the focusing of the microscope 10. The control unit 50 acquires an observation object image which is a microscope photograph of the observation object S acquired by the acquisition unit 20, controls the focus drive unit 30 based on the observation object image, and changes the focusing of the microscope 10 to the observation object focus position where the observation object S is in focus. Also, the control unit 50 moves the tip of the micro tool 41 to the observation object focus position based on the tool image at the observation object focus position.

[0107] The control method of the micromanipulator system 1 of the present embodiment includes a tool position change step S300 of acquiring a microscope photograph (tool image) of the micro tool 41 in the microscope 10 and changing the position of the micro tool 41 based on the tool image. In the tool position change step S300, based on the tool image, the tip of the micro tool 41 is moved to the focal position of the microscope 10.

[0108] Furthermore, before the tool position changing step S300, the control method of the micromanipulator system 1 further includes a focusing changing step S200 of obtaining an image of an object to be observed S, which is a microphotograph of the object to be observed S in the microscope 10, and changing the focusing of the microscope 10 to the focal position of the object to be observed S where the object to be observed is in focus based on the image of the object to be observed. The tool position changing step S300 moves the tip of the micro tool 41 to the focal position of the object to be observed based on the tool image at the focal position of the object to be observed.

[0109] By using the micromanipulator system 1 and the control method of the micromanipulator system 1 of the present embodiment, the tip of the micro tool 41 can be automatically moved to the position where the microscope 10 is in focus on the object to be observed S.

[0110] As a result, it is possible to provide a micromanipulator system 1 and a control method of the micromanipulator system 1 that can efficiently adjust the position of the micro tool 41.

[0111] As described above, an embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and design changes and the like within the scope not departing from the gist of the present invention are also included. In addition, the components shown in the above-described embodiment and the modified examples shown below can be combined as appropriate.

[0112] (Modification Example 1) In the above embodiment, the micromanipulator system 1 includes the microscope 10, but the mode of the micromanipulator system is not limited to this. The micromanipulator system may be used together with a microscope provided outside the micromanipulator system without including a microscope.

[0113] (Modification Example 2) In the above embodiment, the manipulator 40 includes a pump 43, but the form of the manipulator is not limited thereto. The manipulator may not include a pump. For example, when the micro tool is not a pipette but a solid needle-like tool, it is not necessary to apply negative or positive pressure to the inside of the micro tool by a pump, so the pump may not be provided.

[0114] (Modification 3) In the above embodiment, in the focusing change step S200, the control unit 50 changes the focusing of the microscope 10 to the observation object focus position where the observation object S is in focus based on the observation object image, but the form of the control unit is not limited thereto.

[0115] In the focusing change step, the operator may manually adjust the focus of the microscope to the observation object. Also, when using a microscope having an autofocus device that automatically focuses on the observation object, the autofocus device of the microscope may be used to focus on the observation object.

[0116] In that case, the control unit may acquire the position where the observation object is in focus by manual operation or by the autofocus device of the microscope as the observation object focus position, and move the tip position of the micro tool based on the tool image at the observation object focus position determined by manual operation or by the autofocus device of the microscope.

[0117] (Modification 4) In the above embodiment, the control unit 50 performs image processing on the micrograph to extract the contours of the observation object S and the tip of the micro tool 41, but the form of the control unit is not limited thereto. The control unit may determine the focus position of the microscope and the tip position of the micro tool using an image processing method different from contour extraction.

Explanation of Reference Numerals

[0118] 1 Micromanipulator system 10 Microscope 20 Acquisition unit (imaging unit) 30 Focus drive unit 40 Manipulator 41 Micro Tool 42 Movable Part 43 Pump 50 Control Unit 60 Input Unit S Observation Object S100 Initial Value Acquisition Step S200 Focus Change Step S300 Tool Position Change Step

Claims

1. A micromanipulator system used together with a microscope capable of observing an object to be observed, comprising a manipulator having a micro tool and capable of moving the micro tool, an acquisition unit capable of acquiring a microphotograph in the microscope, and a control unit capable of controlling the manipulator, wherein the control unit acquires a tool image which is the microphotograph of the micro tool acquired by the acquisition unit, and changes the position of the micro tool based on the tool image, a micromanipulator system.

2. The micromanipulator system according to claim 1, further comprising two manipulators.

3. The micromanipulator system according to claim 1 or claim 2, wherein the control unit moves the tip of the micro tool to the focal position of the microscope based on the tool image.

4. The micromanipulator system according to claim 3, further comprising a focus drive unit capable of changing the focusing of the microscope, wherein the control unit acquires an object image which is the microphotograph of the object to be observed acquired by the acquisition unit, controls the focus drive unit based on the object image, and changes the focusing of the microscope to an object focus position focused on the object to be observed, and moves the tip of the micro tool to the object focus position based on the tool image at the object focus position.

5. A control method for a micromanipulator system used together with a microscope capable of observing an object to be observed, the method comprising a tool position changing step of acquiring a tool image which is a microphotograph of a micro tool in the microscope and changing the position of the micro tool based on the tool image.

6. The control method for a micromanipulator system according to claim 5, wherein in the tool position changing step, the tip of the micro tool is moved to the focal position of the microscope based on the tool image.

7. Before the tool position changing step, the method further comprises a focusing changing step of acquiring an object image which is a microphotograph of the object to be observed in the microscope and changing the focusing of the microscope to an object focus position focused on the object to be observed based on the object image. ​ ​ ​ ​ ​ The tool position changing step moves the tip of the micro tool to the observation object focus position based on the tool image at the observation object focus position. The control method of the micromanipulator system according to claim 6.

Images