Position estimation device, automatic injection device, and program

Abstract

To enable more highly accurate calculation of a three-dimensional position of an object.SOLUTION: A position estimation device comprises: first and second image acquisition means which acquire first and second images of an object from different directions; approximate expression calculation means which calculates first and second approximate expressions of the object based on the first and second images; and position calculation means which calculates a first intersection between a surface covering the object and a first surface passing through the first image acquisition means and the line of the first approximate expression, calculates a first bent line formed by bending the line passing through each first division point of the first intersection and the first image acquisition means at each first division point on the basis of the refractive index in the surface, calculates a second intersection between a surface covering the object and a second surface passing through the second image acquisition means and the line of the second approximate expression, calculates a second bent line formed by bending the line passing through each second division point of the second intersection and the second image acquisition means at each second division point on the basis of the refractive index of the surface, and calculates the position of the object on the basis of each point where the first bent line intersects with the second bent line.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a position estimation device, an automatic injection device, a position estimation method, and a program for estimating the three-dimensional position of an object. [Background technology]

[0002] A position estimation device is known that acquires first and second images of an object by photographing the object from different directions, and calculates the three-dimensional position of the object based on the acquired first and second images (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6532042 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above position estimation device does not take into account the optical characteristics of the surface covering the object, and therefore the estimation accuracy of the three-dimensional position of the object is reduced.

[0005] The present invention has been made to solve these problems, and its main object is to provide a position estimation device, an automatic injection device, a position estimation method, and a program that can calculate the three-dimensional position of an object with higher accuracy. [Means for solving the problem]

[0006] In order to achieve the above object, one aspect of the present invention is to First and second image acquisition means for acquiring first and second images of an object by photographing the object from different directions; an approximation equation calculation means for calculating first and second approximation equations of a line representing a characteristic shape of the object in the first and second images based on the first and second images acquired by the first and second image acquisition means, respectively; a position calculation means for calculating a position of the object, The position calculation means calculating a first plane passing through the position of the first image acquisition means and the line of the first approximation equation calculated by the approximation equation calculation means, calculating a first intersection line between the first plane and a surface covering the object, and calculating first refracted lines by refracting a line passing through each of the first division points obtained by dividing the first intersection line and the first image acquisition means at each of the first division points based on a refractive index within the surface; a second plane passing through the position of the second image acquisition means and the line of the second approximation equation calculated by the approximation equation calculation means is calculated, a second intersection line between the second plane and a surface covering the object is calculated, and a second refracted line is calculated by refracting a line passing through each second division point obtained by dividing the second intersection line and the second image acquisition means at each second division point based on a refractive index within the surface, calculating points at which the first refraction line and the second refraction line intersect, and calculating a three-dimensional position of the object based on the calculated points; position estimation device It is. In this embodiment, The approximation equation calculation means may calculate a linear linear equation or a curve equation of second or higher order as the approximation equation representing the characteristic shape of the object. In this embodiment, The first and second image acquisition means may be infrared cameras. In this embodiment, The object may be a blood vessel. In order to achieve the above object, one aspect of the present invention is to a puncture needle is inserted into the blood vessel based on the three-dimensional position of the blood vessel calculated by the position estimation device; An automatic injection device characterized by may be also possible. In this embodiment, A moving means for moving the puncture needle; a force detection means for detecting a force acting on the puncture needle; a determination means for determining whether or not the puncture needle has reached the blood vessel based on the force of the puncture needle detected by the force detection means; A control means for controlling the moving means based on a result of the determination by the determination means; The device may include: In this embodiment, The determination means may determine that the puncture needle has reached the blood vessel when the force of the puncture needle detected by the force detection means reaches a second peak. In this embodiment, The control means may control the moving means to withdraw the puncture needle from the blood vessel when the force of the puncture needle detected by the force detection means reaches or exceeds a threshold value. In order to achieve the above object, one aspect of the present invention is to A step of photographing an object from different directions by first and second image acquisition means and acquiring first and second images of the object; calculating, based on the acquired first and second images, first and second approximation equations of a line representing a characteristic shape of the object in the first and second images, respectively; a step of calculating a first plane passing through the position of the first image acquisition means and the calculated line of the first approximation equation, calculating a first intersection line between the first plane and a surface covering the object, and calculating first refracted lines obtained by refracting a line passing through each of the first division points obtained by dividing the first intersection line and the first image acquisition means at each of the first division points based on a refractive index within the surface; a step of calculating a second plane passing through the position of the second image acquisition means and the line of the calculated second approximation equation, calculating a second intersection line between the second plane and a surface covering the object, and calculating second refracted lines obtained by refracting a line passing through each second division point obtained by dividing the second intersection line and the second image acquisition means at each second division point based on a refractive index within the surface; calculating points where the first refraction line and the second refraction line intersect, and calculating a three-dimensional position of the object based on the calculated points; A position estimation method including: may be also possible. In order to achieve the above object, one aspect of the present invention is to A process of photographing an object from different directions using first and second image acquisition means to acquire first and second images of the object; a process of calculating first and second approximation equations of a line representing a characteristic shape of the object in the first and second images, respectively, based on the acquired first and second images; a process of calculating a first plane passing through the position of the first image acquisition means and the line of the calculated first approximation equation, calculating a first intersection line between the first plane and a surface covering the object, and calculating first refracted lines obtained by refracting a line passing through each of the first division points obtained by dividing the first intersection line and the first image acquisition means at each of the first division points based on a refractive index within the surface; a process of calculating a second plane passing through the position of the second image acquisition means and the line of the calculated second approximation equation, calculating a second intersection line between the second plane and a surface covering the object, and calculating second refracted lines obtained by refracting a line passing through each second division point obtained by dividing the second intersection line and the second image acquisition means at each second division point based on a refractive index within the surface; A process of calculating points where the first refraction line and the second refraction line intersect, and calculating a three-dimensional position of the object based on each of the calculated points; A program that causes a computer to execute may be also possible. Effect of the Invention

[0007] According to the present invention, it is possible to provide a position estimation device, an automatic injection device, a position estimation method, and a program capable of calculating the three-dimensional position of an object with high accuracy. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a schematic configuration of a position estimation device according to a first embodiment of the present invention. [Diagram 2] FIG. 4 is a diagram illustrating a specific method for calculating the first and second refraction lines. [Diagram 3]FIG. 2 is a flowchart showing the flow of a position estimation method according to the first embodiment. [Figure 4] FIG. 13 is a diagram showing an experimental puncture success rate. [Diagram 5] 1 is a diagram showing estimated positions of a blood vessel cross section according to a conventional position estimation method and the position estimation method according to the first embodiment. FIG. [Figure 6] FIG. 11 is a block diagram showing a schematic system configuration of an automatic injection device according to a second embodiment of the present invention. [Figure 7] FIG. 2 is a diagram showing a schematic configuration of a moving section. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] EMBODIMENT 1 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the drawings. A position estimation device according to a first embodiment of the present invention can estimate three-dimensional positions of blood vessels in a human body with high accuracy, for example, in a medical field.

[0010] However, conventional position estimation devices do not take into account the optical properties of human skin, such as the refractive index, which results in problems with the estimation accuracy when estimating the three-dimensional positions of blood vessels in the human body, such as the blood vessel depth being estimated as shallower than the actual depth.

[0011] In contrast, in the position estimation device 1 according to the first embodiment, the three-dimensional positions of the blood vessels in the human body are estimated taking into account the optical characteristics of the skin of the human body, as described below. This makes it possible to estimate the three-dimensional positions of the blood vessels in the human body with high accuracy.

[0012] 1 is a block diagram showing a schematic configuration of a position estimation device according to embodiment 1. The position estimation device 1 according to embodiment 1 includes first and second cameras 2 and 3 for acquiring images of blood vessels, an approximate equation calculation unit 4 for calculating an approximate equation of the blood vessels, and a position calculation unit 5 for calculating a three-dimensional position of the blood vessels.

[0013] The first and second cameras 2, 3 are specific examples of first and second image acquisition means. The first and second cameras 2, 3 respectively capture images of blood vessels from different directions and capture first and second images as stereo images of the blood vessels. The first and second cameras 2, 3 are preferably infrared cameras. This allows, for example, blood vessels under the skin to be seen through and captured. The first and second cameras 2, 3 output the captured first and second images to the approximation equation calculation unit 4.

[0014] The approximate expression calculation unit 4 is a specific example of an approximate expression calculation means. The approximate expression calculation unit 4 calculates approximate expressions that represent characteristic shapes of blood vessels based on first and second images of the blood vessels acquired by the first and second cameras 2 and 3, respectively.

[0015] The approximate equation calculation unit 4 extracts blood vessel portions from the first and second images acquired by the first and second cameras 2 and 3. For example, the approximate equation calculation unit 4 extracts, from the first and second images, regions having color characteristics specific to blood vessels and in which brightness and saturation are within specific ranges, as blood vessel portions.

[0016] The approximation calculation unit 4 calculates, for example, a linear equation as an approximation expression representing the characteristic shape of the extracted blood vessel portion. Normally, blood vessels (veins) in the arm, where blood is collected or transfused, extend in a straight line, so the characteristic shape of the blood vessel can be expressed by a linear equation. This makes it possible to easily express the characteristic shape of the blood vessel by an approximation expression.

[0017] The approximation equation calculation unit 4 binarizes the first image captured by the first camera 2. The approximation equation calculation unit 4 generates binary image data in which, for example, the area occupied by blood vessels is set to "1" and the other areas are set to "0". Then, the approximation equation calculation unit 4 extracts the blood vessel portion from the binarized first image. The approximation equation calculation unit 4 calculates, as the first approximation equation, a linear equation of the central axis line passing through the center of the extracted blood vessel portion (area of ​​"1"), for example, using the least squares method.

[0018] Similarly, the approximate equation calculation unit 4, for example, binarizes the second image captured by the second camera 3 and extracts a blood vessel portion from the binarized second image. The approximate equation calculation unit 4 calculates, as a second approximate equation, a linear equation of a central axis line passing through the center of the extracted blood vessel portion using the least squares method or the like.

[0019] In this way, the approximate equation indicating the characteristic shape of the blood vessel, which is the object, can be easily calculated without using complicated processing. The approximate equation calculation unit 4 outputs the calculated first and second approximate equations to the position calculation unit 5.

[0020] The position calculation unit 5 is a specific example of a position calculation means. The position calculation unit 5 calculates the three-dimensional position of the blood vessel with high accuracy based on the first and second approximate equations of the blood vessel calculated by the approximate equation calculation unit 4, taking into consideration the refractive index in the skin surface (for example, the skin model surface).

[0021] The position calculation unit 5 calculates an equation of a first plane that passes through the focal position of the first camera 2 and the line of the first approximation equation calculated by the approximation equation calculation unit 4. The position calculation unit 5 calculates a first intersection line between the calculated first plane and a plane representing the skin model surface. This first intersection line is a blood vessel image seen from the first camera 2. The skin model surface is a specific example of a surface covering an object.

[0022] As described above, light is refracted at the skin surface, but the optical path of the refracted light within the skin surface does not exist on the same plane. Therefore, in the present embodiment 1, the first intersection line calculated above is finely divided, and the refraction angle is obtained from the incident light at each point, and the line of sight from the first camera 2 is rotated at different angles. In this way, the three-dimensional position of the blood vessel is estimated with high accuracy, taking into account the difference in refraction angle depending on the line of sight direction of the camera.

[0023] Specifically, the position calculation unit 5 calculates first refraction lines by refracting a line passing through each point (hereinafter, first division point) obtained by dividing the first intersection line at a predetermined interval and the focal position of the first camera 2 at each first division point based on the refractive index within the skin model surface.

[0024] For example, the first refraction line can be calculated as follows: Fig. 2 is a diagram for explaining a specific method for calculating the first and second refraction lines.

[0025] 1st division point p 1,n The unit vector pointing from the incident light V to the focal position C1 of the first camera 2 is 1,n Let n be the number of divisions. The number of divisions is, for example, 100, but is not limited to this and may be any number. Increasing the number of divisions improves the estimation accuracy, but also increases the amount of calculation. Therefore, it is preferable to set the number of divisions to an optimal value experimentally obtained by taking both into consideration. Furthermore, the division points are spaced equally apart, but may be spaced unequal.

[0026] Incident light V 1,n and the plane L representing the skin model surface. s Unit normal vector N s By calculating the inner product of and, the distance from the focal position C1 of the first camera 2 to the first division point p 1,n Incident angle θ 1,n Then, from Snell's law, the refraction angle φ 1,n Derive the angle of incidence θ 1,n and the refraction angle φ 1,n is calculated by the following formula: θ 1,n =cos -1 (N s ·V 1,n ) φ 1,n =sin -1 {(n air / n skin )sinθ 1,n}

[0027] For example, the refractive index on the surface of the skin model is n skin = 1.56, the refractive index in air is n air = 1.00. The axis of rotation that refracts the incident light is ω 1,n is the incident light V 1,n and normal vector N s The incident light V 1,n The rotation angle γ 1,nis calculated from the following formula: ω 1,n =V 1,n ×N s Gamma 1,n =θ 1,n -φ 1,n

[0028] Incident light V 1,n The rotation axis ω 1,n Rotation angle γ 1,n By rotating it by , the refracted vector v 1,n is calculated as follows: v 1,n =R 1,n ω1,nγ1,n (-V 1,n )

[0029] R i,n ω1,nγ1,n is the Rodrigues rotation matrix. 1,n If the vector in the direction is the refracted ray, the equation of the first refracted line representing the light path is as follows. L 1,n =p 1,n +tv 1,n Let t be any real number.

[0030] The position calculation unit 5 calculates the second refraction line in the same manner as the first refraction line described above. The position calculation unit 5 calculates the second refraction line by refracting a line passing through each of the second division points obtained by dividing the second intersection line at a predetermined interval and the focal position C2 of the second camera 3 at each of the second division points based on the refractive index within the skin model surface.

[0031] Specifically, the second refraction line is calculated as follows. 2nd dividing point p 2,n The unit vector pointing from the focal position C2 of the second camera 3 to the incident light V 2,n The incident light V 2,n and the plane L representing the skin model surface. s Unit normal vector N s By calculating the inner product of and, the distance from the focal position C2 of the second camera 3 to the second division point p 2,n Incident angle θ2,n Then, from Snell's law, the refraction angle φ 2,n Derive the angle of incidence θ 2,n and the refraction angle φ 2,n is calculated by the following formula: θ 2,n =cos -1 (N s ·V 2,n ) φ 2,n =sin -1 {(n air / n skin )sinθ 2,n}

[0032] Rotation axis ω that refracts the incident light 2,n is the incident light V 2,n and normal vector N s The incident light V 2,n The rotation angle γ 2,n is calculated from the following formula: ω 2,n =V 2,n ×N s Gamma 2,n =θ 2,n -φ 2,n

[0033] Incident light V 2,n The rotation axis ω 2,n Rotation angle γ 2,n By rotating it by , the refracted vector v 2,n is calculated as follows: v 2,n =R 2,n ω2,nγ2,n (-V 2,n )

[0034] From a point on the blood vessel image 2,n If the vector in the direction is the refracted light, the equation of the second refracted line representing the light path is as follows. L 2,n =p 2,n +tv 2,n

[0035] The position calculation unit 5 calculates the points where the first refraction line and the second refraction line intersect with each other. 1,n =p 1,n +tv 1,n and the second refraction line L 2,n =p 2,n +tv 2,n The points where and intersect are calculated.

[0036] More specifically, the first refraction line L 1,n =p 1,n +tv 1,n and the second refraction line L 2,n =p 2,n +tv 2,n The points on the first and second refraction lines where the distance between and is the shortest are called t 1、n , t 2、n The position calculation unit 5 calculates the position of this point t 1、n , t 2、n Calculate.

number

[0037] The position calculation unit 5 calculates the shortest point t 1、n , t 2、n The position calculation unit 5 calculates the position of the blood vessel as a three-dimensional linear equation based on the calculated group of midpoints. The position calculation unit 5 calculates the position of the blood vessel as a three-dimensional linear equation by performing, for example, principal component analysis on the calculated group of midpoints.

[0038] FIG. 3 is a flowchart showing the flow of the position estimation method according to the first embodiment.

[0039] The first and second cameras 2, 3 respectively capture images of blood vessels from different directions, obtain first and second images of the blood vessels, and output the obtained first and second images to the approximate equation calculation unit 4 (step S101).

[0040] The approximate equation calculation unit 4 calculates first and second approximate equations representing the characteristic shape of the blood vessel based on the first and second images of the blood vessel output from the first and second cameras 2 and 3, respectively, and outputs the calculated first and second approximate equations to the position calculation unit 5 (step S102).

[0041] The position calculation unit 5 calculates an equation of a first plane that passes through the focal position of the first camera 2 and the line of the first approximation equation calculated by the approximation equation calculation unit (step S103). The position calculation unit 5 calculates a first intersection line between the calculated first plane and a plane representing the skin model surface (step S104).

[0042] The position calculation unit 5 calculates first refraction lines obtained by refracting a line passing through each of the first division points obtained by dividing the first intersection line at a predetermined interval and the focal position of the first camera 2 at each of the first division points based on the refractive index within the skin model surface (step S105).

[0043] The position calculation unit 5 calculates an equation of a second plane that passes through the focal position of the second camera 3 and the line of the second approximation equation calculated by the approximation equation calculation unit 4 (step S106). The position calculation unit 5 calculates a second intersection line between the calculated second plane and a plane representing the skin model surface (step S107).

[0044] The position calculation unit 5 calculates second refraction lines obtained by refracting a line passing through each second division point obtained by dividing the second intersection line at a predetermined interval and the focal position of the second camera 3 at each second division point based on the refractive index within the skin model surface (step S108).

[0045] The position calculation unit 5 calculates the points where the first and second refraction lines intersect (step S109). The position calculation unit 5 calculates the position of the blood vessel as a three-dimensional linear equation based on the calculated point group (step S110).

[0046] Next, the estimation accuracy of the position estimation method according to the first embodiment is confirmed by the following experiment. In this experiment, a robot arm automatically punctures a blood vessel based on the three-dimensional position of the blood vessel estimated by the position estimation method. The computer transmits the coordinate values ​​(X, Y, Z) and posture angles (α, β, γ) of the hand T to the controller, and the controller controls the multi-joint robot arm to that position.

[0047] Based on images captured simultaneously by two infrared cameras, the three-dimensional positions of blood vessels are estimated using the position estimation method according to the first embodiment and a conventional position estimation method. Note that the conventional position estimation method estimates the three-dimensional positions without considering the refraction at the skin surface as described above.

[0048] The skin model was created by passing a silicone tube with an outer diameter of 4 mm and an inner diameter of 3 mm as a simulated blood vessel through a case made of an acrylic board, and then filling it with super-soft urethane molding resin (human skin gel hardness 0, Exseal). India ink was placed inside the tube as a simulated blood vessel, and three markers were attached to the surface of the skin model to define the skin plane.

[0049] The skin model was placed in two areas, close and far from the camera. In each area, the blood vessel position was estimated and the automatic puncture operation was performed when the skin model was horizontal and when it was tilted ±10° around the blood vessel axis. The skin model was placed five times under each condition, at different positions and angles for each sample, and the success or failure of the puncture was compared. The success or failure of the puncture was judged by pulling the syringe after the puncture operation and confirming backflow of blood.

[0050] Fig. 4 is a diagram showing the puncture success rate in the above experiment. As shown in Fig. 4, in the conventional position estimation method, the puncture success rate for all distances (short distance and long distance) was 33%, but in the position estimation method according to the present embodiment 1, the puncture success rate for all distances is 76%, which is a significant improvement. In particular, in the position estimation method according to the present embodiment, the estimation accuracy is maintained even in a posture where the skin plane is inclined (±10°), which is a weakness of the conventional position estimation method, and the puncture success rate is improved.

[0051] In addition, the conventional position estimation method stops before reaching the simulated blood vessel because it estimates the position shallower than the actual blood vessel position, resulting in failure to puncture. In contrast, the position estimation method according to the first embodiment reaches the inside of the blood vessel and succeeds in puncturing. Such results are often seen when the puncture target is located in a far-away area from the camera.

[0052] Fig. 5 is a diagram showing the estimated positions of blood vessel cross sections by the conventional position estimation method and the position estimation method according to the present embodiment 1. As shown in Fig. 5, it can be seen that the position estimation method according to the present embodiment 1 estimates a location closer to the true blood vessel position than the conventional position estimation method.

[0053] As described above, the position estimation device 1 according to the first embodiment calculates a first plane passing through the position of the first camera 2 and the line of the first approximation formula calculated by the approximation formula calculation unit 4, calculates a first intersection line between the first plane and the skin surface covering the blood vessel, calculates a first refraction line by refracting a line passing through the first camera 2 and each of the first division points obtained by dividing the first intersection line at a predetermined interval, at each of the first division points based on the refractive index within the skin surface, calculates a second plane passing through the position of the second camera 3 and the line of the second approximation formula calculated by the approximation formula calculation unit 4, calculates a second intersection line between the second plane and the skin surface covering the blood vessel, calculates a second refraction line by refracting a line passing through the second camera 3 and each of the second division points obtained by dividing the second intersection line at a predetermined interval, at each of the second division points based on the refractive index within the skin surface, calculates points at which the first refraction line and the second refraction line intersect, and calculates the three-dimensional position of the blood vessel based on each of the calculated points.

[0054] This allows the three-dimensional position of the blood vessel to be calculated with high accuracy, taking into account the refractive index within the skin surface that covers the blood vessel. Also, the first and second intersection lines are divided, the refraction angle is obtained from the incident light for each of the first and second division points, and the lines of sight from the first and second cameras 2 and 3 are rotated at different angles. This allows the three-dimensional position of the blood vessel to be estimated with high accuracy, taking into account the difference in refraction angle depending on the line of sight direction of the camera.

[0055] In the above embodiment 1, the blood vessels of a person in a medical field are used as the object, but the object is not limited thereto and may be any object. As a result, the three-dimensional position of any object can be calculated with high accuracy by a simple method, similar to the case of blood vessels.

[0056] For example, when an object is covered with a surface (glass, liquid, etc.) and light is refracted on the surface, the position estimation device 1 according to the present embodiment 1 can estimate the three-dimensional position of the object with high accuracy. In particular, the position estimation device 1 according to the present embodiment 1 can estimate the three-dimensional position of the object with high accuracy not only when the surface is flat, but also when the surface has projections and recesses.

[0057] In the above-mentioned first embodiment, the approximate expression calculation unit 4 calculates linear expressions as the first and second approximate expressions, but is not limited thereto. The approximate expression calculation unit 4 may calculate curve expressions such as two-dimensional curves or three-dimensional curves as the first and second approximate expressions.

[0058] In this case, the position calculation unit 5 calculates a first surface equation of the first surface including the position of the first camera 2 and the first curve (first approximate equation) on the first image calculated by the approximate equation calculation unit 4.

[0059] The position calculation unit 5 calculates a first intersection line between the calculated first curved surface and a plane representing the skin model surface. The position calculation unit 5 calculates first refraction lines obtained by refracting a line passing through each of the first division points obtained by dividing the first intersection line at a predetermined interval and the focal position of the first camera 2 at each of the first division points based on the refractive index within the skin model surface.

[0060] Similarly, the position calculation unit 5 calculates a second surface equation of a second surface including the position of the second camera 3 and the second curve (second approximate equation) on the second image calculated by the approximate equation calculation unit 4.

[0061] The position calculation unit 5 calculates a second intersection line between the calculated second curved surface and a plane representing the skin model surface. The position calculation unit 5 calculates second refraction lines obtained by refracting a line passing through each of the second division points obtained by dividing the second intersection line at a predetermined interval and the focal position of the second camera 3 at each of the second division points based on the refractive index within the skin model surface.

[0062] The position calculation unit 5 calculates the points where the first refraction line and the second refraction line calculated above intersect with each other. The position calculation unit 5 calculates the three-dimensional position of the blood vessel based on the calculated point group.

[0063] EMBODIMENT 2 The automatic injection device according to the second embodiment of the present invention automatically punctures a puncture needle into a person's blood vessel in, for example, a medical facility, etc., to collect blood, perform blood transfusion, etc. The automatic injection device according to the second embodiment automates blood collection, transfusion, etc., performed in a medical facility, thereby preventing secondary infection among medical staff, eliminating waiting times at hospitals, and resolving problems associated with blood collection, transfusion, etc., such as puncture errors.

[0064] FIG. 6 is a block diagram showing a schematic system configuration of the automatic injection device according to the second embodiment.

[0065] The automatic injection device 10 according to the second embodiment inserts a puncture needle into a blood vessel based on the three-dimensional position of the blood vessel calculated by the position estimation device 1 according to the first embodiment. The blood vessel is, for example, a superficial venous vessel in the antecubital fossa of a person's forearm.

[0066] The automatic injection device 10 includes a position estimation device 1, a moving unit 11 that moves the puncture needle, a force detection unit 12 that detects the force applied to the puncture needle, a determination unit 13 that determines whether the puncture needle has reached a blood vessel, and a control unit 14 that controls the moving unit 11.

[0067] As described above, the position estimation device 1 calculates the three-dimensional position of the blood vessel, and outputs the calculated three-dimensional position of the blood vessel to the control unit 14. The position estimation device 1 has a hardware configuration of a normal computer including, for example, a processor 1a such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), an internal memory 1b such as a RAM (Random Access Memory) or a ROM (Read Only Memory), a storage device 1c such as an HDD (Hard Disk Drive) or an SDD (Solid State Drive), an input / output I / F 1d for connecting peripheral devices such as a display, and a communication I / F 1e for communicating with devices outside the device. The position estimation device 1, the determination unit 13, and the control unit 14 may be integrated into one unit.

[0068] The moving unit 11 is a specific example of a moving means. For example, as shown in FIG. 7, the moving unit 11 is configured as a manipulator that can move the puncture needle to an arbitrary position. The moving unit 11 has an arm unit 111, a slide mechanism 112 connected to the tip of the arm unit 111, and a linear actuator 113 that drives the slide mechanism 112. The arm unit 111 is a multi-joint arm that is configured of a plurality of joint units 114 and links 115. Each joint unit 114 is provided with a servo motor that rotates each joint unit 114, a rotation sensor that detects rotation information of each joint unit 114, and the like.

[0069] A syringe 116 having a puncture needle is connected to the slide mechanism 112. The linear actuator 113 drives the slide mechanism 112, thereby enabling the puncture needle of the syringe 116 to be linearly moved closer to or farther away from a blood vessel.

[0070] Force detection unit 12 is a specific example of a force detection means. Force detection unit 12 is configured with, for example, a six-axis force sensor. Force detection unit 12 is provided on slide mechanism 112, for example. Force detection unit 12 can detect the force acting on the puncture needle when slide mechanism 112 brings syringe 116 close to a blood vessel and punctures the puncture needle into the blood vessel.

[0071] The determination unit 13 is a specific example of a determination means. The determination unit 13 determines whether or not the puncture needle has reached (penetrated) a blood vessel based on the force of the puncture needle detected by the force detection unit 12. For example, the determination unit 13 determines that the puncture needle has reached a blood vessel when the force of the puncture needle detected by the force detection unit 12 reaches a second peak. This is because the first peak indicates that the puncture needle has penetrated the skin, and the second peak indicates that the puncture needle has penetrated the blood vessel wall. In this way, it is possible to easily and accurately determine whether the puncture needle has reached a blood vessel.

[0072] The control unit 14 is a specific example of a control means. The control unit 14 performs, for example, feedback control or robust control by transmitting control signals to the arm unit 111 and the linear actuator 113. In this way, the control unit 14 can move the puncture needle of the syringe 116 to any position, such as by moving the puncture needle of the syringe 116 closer to or farther away from a blood vessel.

[0073] The control unit 14 controls the arm unit 111 and the linear actuator 113 based on the three-dimensional position of the blood vessel output from the position estimation device 1, and causes the puncture needle of the syringe 116 to puncture the blood vessel.

[0074] For example, control unit 14 controls arm unit 111 to move syringe 116 to a predetermined distance from the blood vessel based on the three-dimensional position of the blood vessel output from position estimation device 1. Next, control unit 14 controls linear actuator 113 to linearly move the puncture needle of syringe 116 to puncture the blood vessel.

[0075] At this time, the control unit 14 moves the puncture needle to, for example, a position preset on the intersection line indicating the blood vessel as a target position based on the three-dimensional position of the blood vessel output from the position estimation device 1. For example, the center position of the intersection line, a position of a predetermined ratio, a position designated by the user, etc. are preset in the memory.

[0076] The control unit 14 controls the linear actuator 113 of the moving unit 11 based on the determination result by the determination unit 13. For example, based on the determination result by the determination unit 13 that the puncture needle has reached a blood vessel, the control unit 14 controls the linear actuator 113 to stop the linear movement of the puncture needle of the syringe 116. Then, the control unit 14 performs control to collect blood or transfuse blood from the puncture needle of the syringe 116. Note that the mechanism and control for collecting blood or transfusing blood are well known and are described in detail in, for example, JP 2014-239831 A, and therefore this can be used by reference.

[0077] When the control unit 14 determines that blood collection has been completed, it may control the linear actuator 113 to automatically remove the puncture needle of the syringe 116 from the blood vessel. This automates blood collection and transfusion, greatly reducing the burden on medical staff.

[0078] Incidentally, when the puncture needle of the syringe 116 is inserted into a blood vessel, it is preferable for safety reasons to monitor in real time whether any abnormal force is acting on the puncture needle, and if any abnormal force is acting, to immediately remove the puncture needle from the blood vessel.

[0079] In contrast, the automatic injection device 10 according to the second embodiment withdraws the puncture needle from the blood vessel when the force acting on the puncture needle exceeds a threshold value. In other words, when an abnormal force is acting on the puncture needle such that the force acting on the puncture needle exceeds a threshold value, the puncture needle is immediately withdrawn from the blood vessel.

[0080] For example, when the control unit 14 determines that the force of the puncture needle detected by the force detection unit 12 has exceeded a threshold value, it controls the linear actuator 113 of the moving unit 11 to withdraw the puncture needle from the blood vessel. This allows the puncture needle to be withdrawn from the blood vessel immediately, improving safety, even if an abnormal force is applied when the puncture needle is inserted into the blood vessel.

[0081] In the second embodiment, the automatic injection device 10 is configured to insert a puncture needle into a blood vessel and perform blood sampling, etc., based on the three-dimensional position of the blood vessel calculated by the position estimation device 1, but is not limited to this. In the second embodiment, the automatic injection device 10 may be configured to puncture a blood vessel (such as an artery in the foot or the base of the hand) based on the three-dimensional position of the blood vessel calculated by the position estimation device 1, and insert a catheter into the blood vessel.

[0082] Furthermore, the position estimation device 1 may calculate the three-dimensional position of a nerve under the skin, and the automatic injection device 10 may perform an anesthetic injection such as lumbar anesthesia or a nerve block injection based on the three-dimensional position of the nerve calculated by the position estimation device 1.

[0083] In the second embodiment, the position estimation device 1 may estimate the position of only a small predetermined area including the part to be punctured in the entire blood vessel image, instead of estimating the position of the entire blood vessel image. This can reduce the amount of calculation and shorten the time required for position estimation.

[0084] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims.

[0085] The present invention can also be realized by causing a CPU to execute a computer program to perform the processing shown in FIG.

[0086] The program can be stored and provided to a computer using various types of non-transitory computer readable media. The non-transitory computer readable media includes various types of tangible storage media. Examples of the non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (random access memories)).

[0087] The program may be provided to the computer by various types of transitory computer readable media. Examples of the transitory computer readable medium include an electric signal, an optical signal, and an electromagnetic wave. The transitory computer readable medium can provide the program to the computer via a wired communication path such as an electric wire and an optical fiber, or via a wireless communication path. [Explanation of symbols]

[0088] 1 Position estimation device 2. First Camera 3. Second Camera 4 Approximate formula calculation section 5 Position calculation section 10 Automatic injection device 11 Mobile section 12 Force detection unit 13 Judgment section 14 Control section 111 Arm section 112 Slide mechanism 113 Linear Actuator 114 Joints 115 Link 116 Syringe

Claims

1. a first image capturing means for capturing a first image and a second image capturing means for capturing a second image of an object from different directions; an approximation equation calculation means for calculating first and second approximation equations of a line representing a characteristic shape of the object in the first and second images based on the first and second images acquired by the first and second image acquisition means, respectively; a position calculation means for calculating a position of the object, The position calculation means a first surface passing through the position of the first image acquisition means and the line of the first approximation equation calculated by the approximation equation calculation means; a first intersection line between the first surface and a surface covering the object; and a plurality of first refracted lines obtained by refracting a plurality of lines passing through the first image acquisition means and a plurality of first division points dividing the first intersection line along an extension direction of the first intersection line at the plurality of first division points based on a refractive index within the surface, a second surface passing through the position of the second image acquisition means and the line of the second approximation equation calculated by the approximation equation calculation means; a second intersection line between the second surface and a surface covering the object is calculated; and a plurality of second refracted lines are calculated by refracting a plurality of lines passing through a plurality of second division points that divide the second intersection line along an extension direction of the second intersection line and the second image acquisition means at each of the second division points based on a refractive index within the surface; calculating points at which the plurality of first refraction lines intersect with the plurality of second refraction lines, and calculating a three-dimensional position of the object based on the calculated points; Location estimation device.

2. 2. The position estimation device according to claim 1, the approximation equation calculation means calculates a linear equation or a curve equation of second or higher order as an approximation equation representing the characteristic shape of the object; A position estimation device comprising:

3. 3. The position estimation device according to claim 1, The first and second image acquisition means are infrared cameras. A position estimation device comprising:

4. 4. A position estimation device according to claim 1, further comprising: The object is a blood vessel. A position estimation device comprising:

5. A position estimation device according to claim 4, a puncture needle is inserted into the blood vessel based on the three-dimensional position of the blood vessel calculated by the position estimation device; The automatic injection device is characterized by:

6. 6. The automatic injection device of claim 5, A moving means for moving the puncture needle; a force detection means for detecting a force acting on the puncture needle; a determination means for determining whether or not the puncture needle has reached the blood vessel based on the force of the puncture needle detected by the force detection means; A control means for controlling the moving means based on a result of the determination by the determination means; 1. An automatic injection device comprising:

7. 7. The automatic injection device of claim 6, the determination means determines that the puncture needle has reached the blood vessel when the force of the puncture needle detected by the force detection means reaches a second peak. The automatic injection device is characterized by:

8. 8. The automatic injection device of claim 6 or 7, The control means controls the moving means to withdraw the puncture needle from the blood vessel when the force of the puncture needle detected by the force detection means becomes equal to or greater than a threshold value.

1. An automatic injection device comprising:

9. A process of photographing an object from different directions using first and second image acquisition means to acquire first and second images of the object; a process of calculating first and second approximation equations of a line representing a characteristic shape of the object in the first and second images, respectively, based on the acquired first and second images; a process of calculating a first surface passing through the position of the first image acquisition means and the line of the calculated first approximation equation, calculating a first intersection line between the first surface and a surface covering the object, and calculating a plurality of first refracted lines obtained by refracting a plurality of lines passing through the first image acquisition means and a plurality of first division points dividing the first intersection line along an extension direction of the first intersection line at each of the first division points based on a refractive index within the surface; a process of calculating a second surface passing through the position of the second image acquisition means and the line of the calculated second approximation equation, calculating a second intersection line between the second surface and a surface covering the object, and calculating a plurality of second refracted lines obtained by refracting a plurality of lines passing through the second image acquisition means and a plurality of second division points dividing the second intersection line along an extension direction of the second intersection line at each of the second division points based on a refractive index within the surface; A process of calculating points at which the first refraction lines intersect with the second refraction lines, and calculating a three-dimensional position of the object based on each of the calculated points; A program that causes a computer to execute the following.

Images