Perforation device

Abstract

A drilling device 100 according to the present invention comprises: a display 1 that can make visible a real image which is seen through or imaged; a drill tool 3; a drill marking jib 4; an imaging means 7 (appearance information acquisition means that acquires appearance information of, e.g., the drill tool 3); and a control means 2 that creates a virtual three-dimensional image 12 including a drill direction extension line 31 and a drill end point 41, projects the same onto the real image, and displays the same on the display 1.　The drilling device 100 according to the present invention enables accurate drilling without requiring large equipment such as CT imaging, mock-up creation, or simulation, as was conventionally required.

Description

[Technical field]

[0001] The present invention relates to punching device Regarding. [Background technology]

[0002] Conventionally, when drilling an object such as drilling holes in bones in surgery, drilling holes in wood, metal plates, pillars, etc. in construction and civil engineering work, and drilling holes in concrete block walls, a drilling method has been proposed in which augmented reality (AR) technology is used to extend reality by reflecting a virtual image onto a real image, and a simulation is performed to determine the drilling direction and position, and then the drilling is performed by overlaying the real image in the real field of view.

[0003] Patent Document 1 discloses such a drilling method using AR technology. In this method, when performing implant treatment in which a patient's alveolar tooth (target) is drilled with a drill and a fixture is embedded, the target is CT-photographed. Next, a simulation of fixture embedding is performed using the CT image obtained by CT photography to determine the fixture embedding position, and the position is measured on the CT image. Next, a mark indicating the fixture embedding position and embedding direction determined by the simulation is attached to the CT template. Next, a life-size mock-up (plaster model) of the upper jaw that is molded (impressioned) from inside the patient's mouth is created, and a mouthpiece is attached to this mock-up. Next, a mark is attached to the mouthpiece, and a wire is implanted at the drilling simulation position using this mark as a guide. (The above is the preparatory step.) Next, when actually drilling the patient's alveolar teeth, the mouthpiece with the marker is attached to the patient's remaining teeth, the patient's oral cavity is photographed, and the simulation image of the fixture is superimposed on the photographed image. The drill is positioned using the simulation image of the fixture as a guide to drill the holes.

[0004] Conventional drilling methods involve taking CT scans of the object to be drilled, creating a mock-up (plaster model) to simulate drilling, and then superimposing the simulated virtual image on the real image to drill holes in accordance with the simulation. This method has the advantage of being less likely to result in errors in the drilling position or direction. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2013-34764 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, conventional drilling methods require CT imaging, mock-up creation, fixture simulation, etc., and have the problem of requiring large-scale equipment (or complex processes). The present invention has been made in consideration of such problems, and aims to provide a drilling device and a drilling method capable of accurate drilling without the need for large-scale equipment (or complicated processes), as well as a fixing mechanism capable of properly fixing the object to be drilled after it has been drilled. [Means for solving the problem]

[0007] [1] The drilling device of the present invention is a drilling device for drilling an object to be drilled, and is characterized in that it comprises a display on which a real image obtained by see-through or imaging can be visually viewed, a drilling tool, a drilling mark jig, an appearance information acquisition means for acquiring appearance information of the drilling tool and the drilling mark jig, and a control means for creating a virtual three-dimensional image (virtual three-dimensional image) including a drilling direction extension line and a drilling end point (end point of drilling) by referring to the appearance information of the drilling tool and the drilling mark jig acquired by the appearance information acquisition means, and reflecting the virtual three-dimensional image in the real image and displaying it on the display.

[0008] In this way, the drilling device is equipped with a display on which a real image obtained by see-through or imaging can be visually observed, an appearance information acquisition means for acquiring appearance information of the drilling tool, etc., a control means for creating a virtual three-dimensional image including a drilling direction extension line and a drilling end point by referring to the appearance information of the drilling tool, etc. acquired by the appearance information acquisition means, and displaying the virtual three-dimensional image on the display by reflecting it in a real image, etc., so that it is only necessary to drill based on the drilling direction extension line and the drilling end point (using, as an aim, as a guide), and the drilling device can perform accurate drilling without requiring large-scale equipment for CT scanning of the object to be drilled, creation of a mock-up (plaster model), drilling simulation, etc. In addition, by changing the position (posture) of the drilling mark jig or drilling tool, the drilling end point and the drilling direction extension line can be changed in principle, making it possible to flexibly and accurately drill holes according to the state of the object to be drilled at the time of drilling. In other words, even if the state of the object to be drilled at the actual drilling stage is different from that assumed in advance, it is possible to accurately drill holes without requiring large-scale equipment.

[0009] Incidentally, the present invention is an application of AR technology in the sense that it reflects a virtual three-dimensional image onto a real image. Here, a "real image" is an image of an object that actually exists, and a "virtual 3D image" is a 3D image that does not exist in reality or is invisible, but is created by a CPU (or computer, software). "Visible" means that either a real image that is farther away than the focal distance, or a virtual image that is closer than the focal distance, can be visible. In addition, in this application, "appearance information" includes "shape information" and "characteristic information" of the drilling tool, etc. Since "appearance" is the appearance as seen from the outside, "appearance information" refers to information such as three-dimensional image information obtained directly or indirectly by looking at the drilling tool, etc. from the outside. "External shape information" refers to information relating to the external shape (including part of the external shape) of a drilling tool, etc. Since an "external shape" is a shape seen from the outside, "external shape information" refers to information obtained by looking at a drilling tool, etc. from the outside. For example, it is information on the contour shape, such as an ellipse, triangle, square, etc., of the end of the drilling tool, etc., or the contour shape of the corner if there is a corner. (If the drilling tool is a drill 34 described later in FIG. 1, etc., it is information on the uneven shape of the drill bit itself, the right-angle shape of the attachment part where the drill bit is attached to the chuck, the right-angle shape of the end of the drilling tool body on the drill bit side, the square shape of the end of the grip part 35, etc.) "Characteristic information" is characteristic information about the appearance of a drilling tool, etc. For example, it is the position, shape (form), pattern, meaning of a mark (a mark for identification, a mark for recognition) when the drilling tool, etc. is marked, or characteristic information about the appearance (characteristic information such as whether it is curved). "External shape information" and "characteristic information" are not necessarily clearly distinguished, and a certain appearance may combine both. "Creating a virtual 3D image including a drilling direction extension line and a drilling end point by referring to the appearance information of the drilling tool and the drilling mark jig" means, for example, creating a virtual 3D image of the drilling direction extension line of the drilling tool by referring to appearance information such as the outer shape of the drilling tool or the position of a mark provided on the outside of the tool, or creating a virtual 3D image of the drilling end point of the drilling mark jig by referring to appearance information such as the outer shape of the drilling mark jig or the position of a mark provided on the outside of the drilling mark jig. The appearance information acquiring means may be, for example, an imaging means (camera), or a shape measuring sensor that uses laser, infrared rays, ultrasonic waves, or the like.

[0010] In addition, in the drilling device of the present invention, it is preferable that the drilling tool is provided with a first mark as the appearance information of the drilling tool, and the control means creates the drilling direction extension line by referring to the first mark and displays it on the display.

[0011] In this way, the control means only needs to create the drilling direction extension line by referring to the first mark provided on the drilling tool, which makes it easy to create and makes it even easier to perform accurate drilling without the need for large-scale equipment.

[0012] In addition, in the drilling device of the present invention, it is preferable that the drilling tool is provided with a plurality of first marks based on the drilling direction, and the control means creates the drilling direction extension line by referring to position information of the plurality of first marks and displays it on the display.

[0013] In this way, the control means only needs to create a drilling direction extension line by referring to the position information of multiple first marks set based on the drilling direction, making it easy to create and even easier to perform accurate drilling without the need for large-scale equipment.

[0014] In addition, in the drilling device of the present invention, it is preferable that the drilling marking jig is provided with a second mark as the appearance information of the drilling marking jig, and the control means creates the drilling end point by referring to the second mark and displays it on the display.

[0015] In this way, the control means only needs to create the drilling end point by referring to the second mark provided on the drilling marker jig, which makes it easy to create and makes it even easier to perform accurate drilling without the need for large-scale equipment.

[0016] Furthermore, in the drilling device of the present invention, it is preferable that the drilling marker jig has a plurality of second marks provided in the longitudinal direction of the jig (drilling end point direction), and the control means creates the drilling end point by referring to positional information of the plurality of second marks and displays it on the display.

[0017] In this way, the control means only needs to create the drilling end point by referring to the position information of multiple second marks arranged in the longitudinal direction of the jig (the direction of the drilling end point), which makes it easier to create the end point and makes it even easier to perform accurate drilling without the need for large-scale equipment.

[0018] [2] In the drilling device of the present invention, it is preferable that the control means causes the virtual three-dimensional image of the extension line of the drilling direction and the virtual three-dimensional image of the drilling end point to be displayed on the display during drilling.

[0019] In this way, a virtual three-dimensional image of the extension line of the drilling direction and a virtual three-dimensional image of the drilling end point are displayed on the display during drilling, and these images are not displayed when they are not related to drilling, making it possible to provide a drilling device with an even easier-to-view display.

[0020] [3] In the drilling device of the present invention, it is preferable that the control means displays the virtual three-dimensional image of the extension line of the drilling direction and the virtual three-dimensional image of the drilling end point on the display in a display mode different from the actual image.

[0021] This makes it easier to distinguish between the actual image and the virtual three-dimensional image (the extension line of the drilling direction and the drilling end point), making it possible to perform more accurate drilling.

[0022] Here, "different display modes" refers to display modes such as, for example, when the appearance of a drilling tool, drilling marking jig, etc. is black and white, displaying the virtual 3D image (the drilling direction extension line and the drilling end point) in a different color tone (red, blue, etc.) from that, or displaying it in a flashing manner when it is not flashing.

[0023] [4] In the drilling device of the present invention, it is preferable to further provide a passing notification means for notifying that the drilling direction extension line of the drilling tool passes or does not pass the drilling end point.

[0024] In this way, it is possible to know whether the extension line of the drilling direction passes through the drilling end point, which makes it easier to perform accurate drilling.

[0025] [5] In the drilling device of the present invention, it is preferable to further include a drilling notification means for notifying that the tip of the drilling part of the drilling tool is approaching or has reached the drilling end point.

[0026] In this way, it is possible to know when the tip of the drilling part of the drilling tool is approaching or has reached the drilling end point, making it easier to perform accurate drilling. For example, by slowing down the drilling speed as the tip of the drilling part approaches the end point of drilling, or by terminating drilling when the tip reaches the end point of drilling, it is possible to prevent insufficient or excessive drilling and perform accurate drilling.

[0027] [6] In the drilling device of the present invention, it is preferable that the control means displays on the display as the drilling end point a jig contact point where the tip of the drilling mark jig comes into contact with the object to be drilled, or a point a certain distance away from the jig contact point on a straight line connecting the jig contact point and the drilling start point where the drilling tool starts drilling. The "jig contact point" can also be called the "drilling target point."

[0028] In this way, the drilling end point is the jig contact point where the tip of the drilling mark jig comes into contact with the object to be drilled, or a point a certain distance away from the jig contact point on a line connecting the jig contact point and the drilling start point where the drilling tool begins drilling into the object to be drilled.Therefore, by changing the position of the jig contact point or the drilling start point, the drilling end point can be easily determined or changed, making it even easier to perform accurate drilling without the need for large-scale equipment. For example, in the case where the jig contact point is set as the drilling end point, the drilling end point can be easily determined or changed by contacting the tip of the drilling mark jig with the object to be drilled. In addition, for example, setting the drilling end point to a point a certain distance away from the jig contact point on a straight line connecting the jig contact point and the drilling start point is convenient in cases where the drilling is not performed completely through the object to be drilled, but rather where it is difficult to insert the tip of the drilling marker jig (for example, drilling that stops at a point a certain distance away or at a certain depth from the surface of the object to be drilled, in other words drilling to a certain depth from the surface).

[0029] In addition, in the drilling device of the present invention, it is preferable to further provide a notification means for notifying that the drilling direction extension line coincides with a straight line connecting the jig contact point and the drilling start point, and that the drilling direction coincides with a straight line connecting the jig contact point and the drilling start point (a straight line passing through the drilling end point).

[0030] In this way, it is possible to perform more accurate drilling since it is clear that the drilling direction is based on the straight line connecting the jig contact point and the drilling start point, i.e., the straight line passing through the drilling end point.

[0031] [7] In the drilling device of the present invention, it is preferable that the control means causes an image of the jig contact point to be displayed on the display.

[0032] In this way, an image of the jig contact point is shown on the display, making it easier to determine or change the drilling end point.

[0033] [8] In the perforation device of the present invention, it is preferable that the display is a display that allows a real image to be viewed by see-through. Here, "through vision" means that the real image on the other side of the display can be seen through the display. The real image on the other side of the display can be an image that can be seen directly, or it can be an image reflected by a mirror (reflected in a mirror).

[0034] In this way, since the display allows the real image to be viewed through perspective, the real image can be viewed as it is, and distortion of the real image, reduction in resolution, etc. are unlikely to occur.

[0035] [9] In the punching device of the present invention, it is preferable that the display is a head-mounted display.

[0036] In this way, when the display is a head-mounted display, the person drilling the hole (the surgeon, etc.) wears the display (head-mounted display) on his / her head, so that the real image that changes according to the movement of the head and the virtual 3D image of the drilling end point reflected (superimposed) on it can be seen, making the drilling work easier and enabling more accurate drilling. For example, this is useful when an operator drills a hole in a human or animal bone, or when a carpenter drills a hole in wood.

[0037]

[10] In the punching device of the present invention, it is preferable that the display is a display that allows a real image to be visually observed by imaging. Here, "by imaging" means that an actual image captured by imaging means is displayed on a display and can be viewed.

[0038] In this way, it is easy to create a virtual three-dimensional image (extension line of the drilling direction, drilling end point) by referring to the appearance information of the captured image of the drilling tool, etc. Also, it becomes easier to display the captured image and to reflect (superimpose) the virtual three-dimensional images of the extension line of the drilling direction, the drilling end point, etc. on the display.

[0039]

[11] In the drilling device of the present invention, it is preferable that the control means causes the display to display the drilling end point so that the display position does not change even if the position of the drilling marker jig is changed.

[0040] In this way, the indicated position of the drilling end point does not change even if the position of the drilling marker jig is changed, so the person drilling is freed from the physical constraints of maintaining (holding) the position of the drilling marker jig. For example, when a person drilling holes holds (holds) a drilling marking jig in one hand (left hand) and a drilling tool in the other hand (right hand), if the person places the drilling marking jig in a predetermined position with one hand to determine the display position of the drilling end point, then even if the other hand is then removed from the drilling marking jig and its position changes, the display position of the drilling end point does not change, and the other hand can be used freely.

[0041]

[12] In the drilling apparatus of the present invention, it is preferable to further include a position maintaining means for maintaining the position of the drilling mark jig relative to the object to be drilled.

[0042] In this way, the position of the drilling marker jig relative to the object to be drilled is maintained (kept, fixed), so the person drilling is freed from the physical constraints of maintaining the position of the drilling marker jig, allowing him or her to concentrate more on the drilling. For example, during the drilling operation, the person drilling the hole does not need to hold the drilling mark jig in his / her hand and maintain its position (posture), and is freed from such physical constraints.

[0043]

[13] In the punching device of the present invention, it is preferable that the appearance information acquisition means is an imaging means. The term "imaging means" refers to, for example, a camera (camera) that takes still images (still photos) or moving images (movies, television, videos, etc.). It can also be called imaging means.

[0044] In this way, when the appearance information acquisition means is an imaging means, the appearance information of the object can be acquired even more easily by imaging (photographing) the object. Furthermore, when the captured real image is displayed on a display, it becomes easier to not only obtain appearance information but also to display the real image on the display.

[0045]

[14] In the perforation device of the present invention, it is preferable that the perforation device further includes a two-dimensional perspective image acquisition means for irradiating a detection wave to the object to be perforated and acquiring a two-dimensional perspective image of the object to be perforated, and that the control means reflects the two-dimensional perspective image of the object to be perforated in the real image or the virtual three-dimensional image and displays it on the display.

[0046] Here, the two-dimensional fluoroscopic image acquisition means refers to, for example, an X-ray imaging device, an ultrasound imaging device, a photo-ultrasonic imaging device, etc. that irradiates the object to be perforated with detection waves such as electromagnetic waves including X-rays and laser light, ultrasound, etc., and detects a two-dimensional fluoroscopic image of the object to be perforated (such as the external shape blocked by obstacles, internal structure such as broken parts, etc.) from the state of transmission, absorption, reflection, photoacoustic, etc.

[0047] In this way, a two-dimensional perspective image of the object to be drilled is acquired by the two-dimensional perspective image acquisition means, and is reflected in a real image or a virtual three-dimensional image and displayed on the display (two-dimensional perspective image acquisition does not require large-scale equipment like three-dimensional perspective image acquisition, the use of two-dimensional perspective images allows the reflection (superimposition) of the virtual three-dimensional image on the real image to be performed with even greater positional accuracy (including position correction), even areas blocked by obstacles can be displayed in the two-dimensional perspective image, etc.), making it even easier to perform accurate drilling without the need for large-scale equipment.

[0048]

[15] In the perforation device of the present invention, the two-dimensional radioscopic image acquisition means is preferably an X-ray imaging device.

[0049] In this way, it is possible to use an X-ray imaging device, which is widely used as a means for acquiring two-dimensional fluoroscopic images, making it even easier to perform accurate drilling without the need for large-scale equipment.

[0050]

[16] The drilling method of the present invention is a drilling method for drilling a target object other than a human living body, and is characterized in that it includes the steps of: preparing a display on which a real image obtained by see-through or imaging can be visually viewed; preparing a drilling tool; preparing a drilling marker jig; acquiring appearance information of the drilling tool and the drilling marker jig, creating a virtual three-dimensional image including a drilling direction extension line and a drilling end point by referring to the appearance information, reflecting the virtual three-dimensional image on the real image and displaying it on the display; and drilling the target object based on the drilling direction extension line and the drilling end point.

[0051] In this way, a display is prepared on which the real image obtained by viewing through a fluoroscopy or imaging can be visually observed, and a virtual three-dimensional image including the drilling direction extension line and the drilling end point is reflected in the real image and displayed on the display. Therefore, drilling can be performed based on the drilling direction extension line and the drilling end point, making it possible to provide a drilling method that enables accurate drilling without requiring complex processes such as a CT scan of the object to be drilled, a mock-up (plaster model) creation process, and a drilling simulation process.

[0052]

[17] The fixing mechanism of the present invention is a fixing mechanism for fixing an object to be drilled having a through hole drilled using any of the above-mentioned drilling devices, characterized in that first and second screws, each having a head and a pair of threaded portions, are inserted into the through hole from different openings of the through hole, with the respective threaded portions leading, and the object to be drilled is fixed by engaging the pair of threaded portions with the object to be drilled sandwiched between the heads. Here, a pair of threaded portions refers to a portion constituted by threads that fit together, such as one male thread and the other female thread. Different openings of a through hole refers to two different openings (one opening and the other opening) of a through hole.

[0053] In this way, even if the object to be drilled is broken or split into multiple pieces, the first and second screws are inserted into the through holes formed to pass through the multiple locations (the object to be drilled split into multiple pieces) from different openings of the through holes with their threaded parts at the top, and the object to be drilled (the object to be drilled split into multiple pieces) is sandwiched between their heads, and the threaded parts are fitted together in the through holes, so that the multiple split objects to be drilled are fixed together (securely fixed). In addition, the threaded parts with the sharp points of the first and second screws are located inside the object to be drilled and the heads without sharp points are located outside, so that after fixing, it is possible to prevent injuries due to the sharp points of the threaded parts. Since the sharp points of the threaded parts are not exposed on the surface of the object to be drilled, it is possible to prevent injuries due to the sharp points. In this way, it is possible to properly fix the object to be drilled that has been drilled, including ensuring safety.

[0054]

[18] In the fixing mechanism of the present invention, it is preferable to place a plate having a hole at a location corresponding to the through hole of the object to be drilled between at least one of the areas between the object to be drilled and the head of the first screw, or between the object to be drilled and the head of the second screw, and fix the object to be drilled by sandwiching the object to be drilled between the heads of the first and second screws via the plate and engaging the pair of screw portions.

[0055] In this way, the object to be drilled is clamped between the heads of the first and second screws (via the plates) and fixed. Therefore, even if the material of the object to be drilled is brittle or the object to be drilled is broken, it is possible to fix the object to be drilled more appropriately, for example, by clamping it between the plates and fixing it together.

[0056] Incidentally, a "living organism" refers to the living body of an organism. Organisms include humans, animals (dogs, cats, etc.), and fish. An example of a living organism is the bones of a human or animal. An example of trepanning a living organism is the perforation (making of a hole) of a bone with a drill or insertion pin during surgery on a human or animal. The term "living organism not including a human" is intended to exclude humans from living organisms. [Brief description of the drawings]

[0057] [Figure 1] FIG. 1 is a diagram for explaining an overview of a punching device 100 according to a first embodiment. [Diagram 2] 1 is a diagram for explaining a drilling mark jig 4 of the drilling device 100 according to the first embodiment. FIG. [Diagram 3] 1 is a diagram for explaining a drilling tool 3 of the drilling device 100 according to the first embodiment. FIG. [Figure 4] FIG. 2 is a diagram for explaining an overview of the circuits and the like of the punching device 100 according to the first embodiment. [Diagram 5] 4 is a flowchart for explaining a punching method of the punching device 100 according to the first embodiment. [Figure 6] 1 is a diagram for explaining what appears on the display 1 when punching is performed by the punching device 100 according to the first embodiment. FIG. [Figure 7] 1 is a diagram for explaining the arrangement of a drilling tool 3 and a drilling mark jig 4 when drilling holes with the drilling device 100 according to the first embodiment. FIG. [Figure 8] 10A and 10B are diagrams for explaining a modified example of the first mark 32 in the punching device 100 according to the first embodiment. [Figure 9] 10A and 10B are diagrams for explaining a modified example of the second mark 42 in the punching device 100 according to the first embodiment. [Figure 10] FIG. 11 is a diagram for explaining a punching device 200 according to a second embodiment. [Figure 11] FIG. 11 is a diagram for explaining a punching device 300 according to a third embodiment. [Figure 12] FIG. 11 is a diagram for explaining a punching device 400 according to a fourth embodiment. [Figure 13]FIG. 13 is a diagram for explaining a punching device 500 according to a fifth embodiment. [Figure 14] FIG. 13 is a diagram for explaining a punching device 600 according to a sixth embodiment. [Figure 15] FIG. 13 is a diagram for explaining a punching device 700 according to a seventh embodiment. [Figure 16] FIG. 13 is a diagram for explaining a punching device 900 according to a ninth embodiment. [Figure 17] FIG. 23 is a diagram for explaining a punching device 1000 according to a tenth embodiment. [Figure 18] FIG. 23 is a diagram for explaining a punching device 1100 according to an eleventh embodiment. [Figure 19] FIG. 23 is a diagram for explaining an overview of a punching device 1200 according to a twelfth embodiment. [Figure 20] FIG. 23 is a diagram for explaining a punching device 1200 according to a twelfth embodiment. [Figure 21] FIG. 23 is a diagram for explaining a punching device 1300 according to a thirteenth embodiment. [Figure 22] FIG. 23 is a diagram for explaining a punching device 1400 according to a fourteenth embodiment. [Figure 23] FIG. 23 is a diagram for explaining a punching device 1500 according to a fifteenth embodiment. [Figure 24] FIG. 23 is a diagram for explaining drilling etc. when a plate P is present in the fifteenth embodiment. [Diagram 25] FIG. 23 is a diagram for explaining a punching device 1600 according to a sixteenth embodiment. [Figure 26] FIG. 23 is a diagram for explaining a fixing mechanism 1700 according to a seventeenth embodiment. [Figure 27] FIG. 23 is a diagram for explaining a fixing mechanism 1800 according to an eighteenth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0058] The punching device and punching method of the present invention will be described below based on the embodiments shown in the drawings. Each drawing is a schematic diagram and does not necessarily strictly reflect the actual shape, structure, configuration, process, etc. Each embodiment described below does not limit the invention related to the claims. In addition, not all of the components and combinations thereof described in each embodiment are essential to the present invention. In the following description, the same reference numerals are used across embodiments for components that can be considered to be substantially equivalent, and repeated description may be omitted.

[0059] [Embodiment 1] The punching device and punching method according to the first embodiment will be described with reference to Figs. 1 to 7. First, explanation will be given with reference to Figures 1 to 4. An overview of the drilling device 100 according to the first embodiment will be given with reference to Figure 1, a drilling mark jig 4 will be given with reference to Figure 2, a drilling tool 3 will be given with reference to Figure 3, and an overview of the hardware circuit configuration will be given with reference to Figure 4.

[0060] Overview of the punching device 100 The drilling device 100 of embodiment 1 shown in Figure 1 is a drilling device in which an operator M (the person performing the drilling) drills a hole in a living body (human or animal) as the object to be drilled W (bone W1) using a drilling tool 3, as shown in Figure 1. The drilling device 100 includes a display 1 on which a real image 11 (see FIG. 6) obtained by fluoroscopy or imaging can be viewed, a drilling tool 3, a drilling mark jig 4, an imaging means 7 (appearance information acquisition means for acquiring appearance information of the drilling tool 3, etc.), and a control means 2 for creating a virtual three-dimensional image 12 including a drilling direction extension line 31 and a drilling end point 41, reflecting the virtual three-dimensional image 12 in the real image 11, and displaying the virtual three-dimensional image 12 on the display 1. An operator M drills holes with the drilling tool 3 based on the virtual three-dimensional image 12 including the drilling direction extension line 31 and the drilling end point 41 reflected in the real image 11 seen on the display 1. The drilling device 100 uses a kind of augmented reality (AR) technology in which the virtual three-dimensional image 12 is reflected in the real image 11 on the display 1. Here, the real image 11 is an image of an actual object that actually exists, and is contrasted with the virtual three-dimensional image 12, which does not actually exist and is created by the control means 2 (CPU 21). The drilling end point 41 and the drilling direction extension line 31 are the virtual three-dimensional image 12 created on the display 1 by the control means 2, and are not the real image 11. In Fig. 1, in order to make it easier to understand the invention of the present application, the virtual three-dimensional image 12 (the drilling end point 41 and the drilling direction extension line 31) that cannot be seen in the real world is depicted in the figure.

[0061] As shown in Fig. 1, the display 1 is a head-mounted display 6. A real image 11 can be viewed on the display 1, and there are two types of displays 1: one in which the real image 11 is viewable by see-through (including semi-see-through, see-through), and the other in which the real image 11 captured by an imaging means 7 is displayed on the display 1 and viewed. Either type of display 1 may be used, and these can be easily constructed using, for example, a liquid crystal panel, an EL (Electro Luminescence) panel, or the like. A real image 11 is visually observed on the display 1, and a virtual three-dimensional image 12 of a drilling end point 41 and a drilling direction extension line 31 is also displayed on the display 1. An imaging means (camera) 7 is attached to the display 1 (head mounted display 6) as an appearance information acquisition means. The display 1 and the imaging means (camera) 7 are connected to the control means (controller) 2 by a cable 61, and signals are transmitted and received from one to the other, or signals are transmitted and received between the two. In addition, when the captured real image 11 is displayed on the display 1, the imaging means (camera) 7 has, in addition to its function as an appearance information acquisition means, also a function of capturing the real image 11 for display on the display 1.

[0062] 2, the drilling mark jig 4 has a needle part 44 and a grip part 45. A second mark 42 is provided on the drilling mark jig 4 (here, the needle part 44) by, for example, embossing, printing, attaching a sticker with a mark printed on it, or the like. The control means 2 causes the imaging means 7 to image the second mark 42 (appearance information), calculates (calculates) a distance 46 to the tip 43 of the needle part 44 (by referring to the appearance information), determines the drilling end point 41, and displays it on the display 1. In addition, the control means 2 may cause the imaging means 7 to capture an image of the outer shape (either the entire outer shape or part of the outer shape) of the drilling mark jig 4 instead of the second mark 42, and determine the drilling end point 41 by referring to the appearance information and display it on the display 1.

[0063] As shown in Figs. 1 and 3, the drilling tool 3 held by the operator M in his right hand is an electric drill, and has a drill 34 (drill bit) for drilling the object W to be drilled and a gripping part 35 for gripping the drilling tool 3, and the drill 34 is attached to the gripping part 35 by an attachment mounting part 37 (chuck). The gripping part 35 has a built-in battery (not shown) for power. The drilling tool 3 (here, the drill 34) has a first mark 32 provided at a certain distance 36 from the tip 33, for example, by embossing, printing, or attaching a sticker with a mark printed thereon. The control means 2 causes the imaging means 7 to capture an image of the first mark 32, calculates (calculates) the drilling direction of the drill 34, determines a drilling direction extension line 31, and displays it on the display 1.

[0064] The surgeon M holds the gripping portion 45 of the perforation mark jig 4 with his left hand and brings the tip of the needle portion 44 into contact with the perforation end point 41 of the bone W1. If the bone W1 is exposed, the tip is brought into contact as is, and if there is muscle W2 or the like on the outside, the tip is brought into contact with the bone W1 by penetrating the muscle W2. Reference numeral 430 denotes the point (tool contact point) where the perforation mark jig 4 (the tip of the needle portion 44 of the perforation mark jig 4) comes into contact with the bone W1, and in the first embodiment, the tool contact point 430 is set as the perforation end point 41. As in the case of the drilling tool 3, the control means 2 acquires the appearance information of the drilling tool 3 by, for example, imaging the first mark 32 of the drilling tool 3 or the outer shape (all or part of the outer shape) of the drilling tool 3 using the imaging means 7, and creates a virtual three-dimensional image 12 of the drilling end point 41 with reference to the appearance information of the drilling mark jig 4, which is reflected in the real image 11 and displayed on the display 1. The control means 2 also creates a virtual three-dimensional image 12 of the drilling direction extension line 31 with reference to the appearance information of the drilling tool 3, which is reflected in the real image 11 and displayed on the display 1. When drilling, the surgeon M can perform accurate drilling without requiring large-scale equipment by using the drilling end point 41 and the drilling direction extension line 31 as a drilling guide.

[0065] Overview of the circuitry of the punching device 100 As shown in FIG. 4, the display 1 and the imaging means (camera) 7 are connected by the control means 2 via a cable 61. The control means 2 is composed of a microcomputer, and has a CPU (Central Processing Unit) 21, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 23, an I / O (Input / Output controller) 24, and an internal bus 25 of the microcomputer that electrically connects them. The ROM 22 stores various data and programs (a series of instructions prepared for processing, control, etc.) for performing various controls including display control and input / output control, and calculations, etc. The RAM 23 is used as a working memory for the CPU 21 to perform various processes. The CPU 21 reads and executes a program that describes instructions (processing) for the CPU 21.

[0066] The internal bus 25 of the control means (microcomputer) 2 is connected to a cable 61 via an interface 26, and the cable 61 serves as an external bus. The display 1, the imaging means (camera) 7, etc. are connected to the cable 61, and the control means 2 receives signals such as various data from these via the cable 61, and outputs signals such as various data and control signals to these. The control means 2 can also be referred to as a CPU 21 that executes functions such as reading a program and displaying it on the display 1.

[0067] Drilling method The punching method of the punching device 100 according to the first embodiment will be described with reference to FIGS. Fig. 5 is a flow chart for explaining the drilling method of the drilling device 100 according to embodiment 1. Fig. 6 is a diagram for explaining the appearance on the display 1 when drilling is performed by the drilling device 100 according to embodiment 1. Fig. 6(A) is a diagram for explaining the appearance on the display 1 of the real image 11 and the virtual three-dimensional image 12 of the drilling end point 41 reflected therein, and Fig. 6(B) is a diagram for explaining the appearance on the display 1 of the virtual three-dimensional image 12 of the drilling direction extension line 31.

[0068] In Figures 6(A) and 6(B), the display section of the display 1 is designated by the reference numeral 11. This is to indicate that the image visible on the display 1 (the image to be viewed), whether it is an image viewed through a see-through lens or an image captured by the imaging means 7, is for the most part a real image rather than a virtual image. Meanwhile, the image designated by the reference numeral 12 in parentheses is a virtual image (virtual three-dimensional image) (the same applies to the other figures).

[0069] In addition, in Figure 6, dotted lines are drawn to the left and right of the solid line indicating bone W1, and dotted lines are also drawn to the left and right through the drilling end point 41. However, these are dotted lines indicating the position of bone W1 drawn to make it easier to understand the present invention, and are not displayed on the display 1.

[0070] The punching method of the punching device 100 according to the first embodiment includes the following steps: Step (a): preparing a display 1 on which a real image 11 to be seen through or imaged can be viewed; Step (b): preparing a drilling tool 3; Step (c): preparing a drilling mark jig 4; Step (d): acquiring appearance information of the drilling tool 3 and the drilling mark jig 4, creating a virtual three-dimensional image 12 including a drilling direction extension line 31 and a drilling end point 41 by referring to the appearance information, and reflecting the virtual three-dimensional image 12 in a real image 11 and displaying it on the display 1; Step (e): A step of drilling the object W (W1) to be drilled based on the drilling direction extension line 31 and the drilling end point 41; Includes.

[0071] Step (a) is a step of preparing a display 1 on which a real image 11 to be seen through or imaged can be viewed. In this step, a display 1 on which the actual image 11 seen through or captured can be viewed is prepared. As the display 1, for example, AR (Augmented Reality) glasses (displays) can be used as a matter of course, as well as MR (Mixed Reality) glasses, VR (Virtual Reality) glasses, etc. These glasses differ only in what they display, and can be used by reflecting the virtual three-dimensional image 12 including the drilling direction extension line 31 and the drilling end point 41 on the actual image 11 seen through or captured.

[0072] Step (b) is a step of preparing a drilling tool 3 for drilling a hole in the object W to be drilled. In this step, a drilling tool 3 for drilling a hole in the object W (W1) to be drilled is prepared. In this step, it is preferable that the control means 2 acquires three-dimensional information of the appearance of the drilling tool 3 in advance, prior to drilling, for example, by having the imaging means 7 capture an image of the drilling tool 3. Alternatively, it is preferable that CAD three-dimensional information of the appearance of the drilling tool 3 is stored in advance, for example, in the ROM 22 (instead of the ROM 22, for example, an optical disk, a HDD, or various semiconductor memories other than the ROM 22 may be used).

[0073] Step (c) is a step of preparing a drilling mark jig 4 (for indicating a drilling end point 41). In this step, a drilling mark jig 4 (for indicating a drilling end point 41) is prepared. In this step, it is preferable that the control means 2 obtains three-dimensional information of the appearance of the drilling mark jig 4 in advance of drilling, for example, by having the imaging means 7 take an image of the drilling mark jig 4. Alternatively, it is preferable that CAD three-dimensional information of the appearance of the drilling mark jig 4 is stored in the ROM 22 in advance.

[0074] Step (d) is a step of acquiring appearance information of the drilling tool 3 and the drilling mark jig 4, referring to the appearance information, creating a virtual three-dimensional image 12 including a drilling direction extension line 31 and a drilling end point 41, and reflecting the virtual three-dimensional image 12 in the real image 11 and displaying it on the display 1. In this process, the imaging means 7 acquires external information of the drilling tool 3 etc., and the control means 2 refers to the external information to create a virtual three-dimensional image 12 including a drilling direction extension line 31 etc., and reflects the virtual three-dimensional image 12 in the real image 11 and displays it on the display 1. This process corresponds to, for example, steps S11 to S31 in FIG.

[0075] These steps relate to "the imaging means 7 acquiring appearance information of the drilling tool 3 etc., the control means 2 referring to the appearance information creating a virtual three-dimensional image 12, and reflecting the virtual three-dimensional image 12 in the real image 11 and displaying it on the display 1," and will be explained briefly. First, we will explain the case where a monocular camera is used as the imaging means 7, and the first mark 32 to the second mark 42 (mark 32, etc.) affixed to the drilling tool 3 or the drilling mark jig 4 (the drilling tool 3, etc.) are used as the appearance information acquired or referenced by the imaging means 7. The control means 2, for example, causes the imaging means 7 to capture images of the drilling tool 3 all around, thereby acquiring and storing three-dimensional image information of the appearance of the drilling tool 3, etc., prior to drilling. Alternatively, three-dimensional image information of a design CAD may be stored. In this manner, three-dimensional image information of the overall appearance of the drilling tool 3, etc., and three-dimensional image information of the marks 32, etc. are acquired or stored in advance. As a result, three-dimensional image information regarding their relative positional relationship is also acquired or stored at the same time.

[0076] During drilling, the control means 2 causes the imaging means 7 to capture an image of the marks 32, etc., and compares the captured image with a known three-dimensional image (information) previously acquired. This imaging is preferably performed multiple times by changing the relative positions of the imaging means 7 and the marks 32, etc. This is because, in the case of a monocular camera, the image obtained by imaging is a two-dimensional image, and three-dimensional image information can be acquired by changing the imaging position. Then, a three-dimensional image of the mark 32 etc. is created based on the known three-dimensional image and in accordance with the size of the captured image. Then, the position of the virtual three-dimensional image is aligned with the position of the photographed image. Then, the shape (form) of the virtual 3D image is matched to the shape of the captured image. If the shapes are the same, they are matched as is, but if the shapes are distorted, they are distorted. If the virtual three-dimensional image of the mark 32 or the like created in this way is not superimposed so as to match the photographed image, the above operation is repeated several times until they match as closely as possible. The order in which the size, position, and shape are matched is arbitrary. Then, based on the virtual three-dimensional images of the marks 32 and the like, a virtual three-dimensional image 12 including a drilling direction extension line 31 is created and is displayed on the display 1 while being reflected in the actual image 11 of the drilling tool 3 .

[0077] The appearance (information) acquired or referenced by the imaging means 7 is not limited to the marks 32, etc., and may be, for example, the outer shape (information) of the drilling tool 3, etc., such as a rectangular outline shape at the end of the drilling tool 3, etc., or (information on) the outline shape of a corner when the drilling tool 3, etc. has a corner. Even when such appearance (information) is used instead of the marks 32, etc., a virtual three-dimensional image 12 including a drilling direction extension line 31 can be created in the same way as when the marks 32, etc. are used, and the virtual three-dimensional image 12 can be reflected in the actual image 11 of the drilling tool 3 and displayed on the display 1.

[0078] Also, instead of a monocular camera as the imaging means 7, for example, a stereo camera with cameras installed on both sides of the head mounted display 6 may be used. By using a stereo camera, it is easy to perform 3D measurement or 3D image creation using so-called triangulation by using two or more viewpoints equivalent to both human eyes. 3D image information can be acquired without changing the imaging position.

[0079] In addition, instead of a monocular camera as the imaging means 7, for example, a semiconductor laser that emits radar waves and a light receiving element that receives light reflected from the drilling tool 3, etc. may be used, and three-dimensional measurement or three-dimensional image creation of the drilling tool 3, etc. may be performed by calculating the distance from the time from emission to reception of the light and the phase difference between the original wave and the reflected wave.

[0080] Returning to the description of steps S11 to S31 above, in this step, for example, first, the operator M indicates the end point of drilling using the drilling mark jig 4 (S11, "S" means "step", the same below). FIG. 6(A) shows the state seen on the display 1. Reference numeral 41 is a virtual three-dimensional image 12 of the end point of drilling. Reference numeral 11 is a real image seen on the display 1 by fluoroscopy (see-through) or imaging, and images other than the three-dimensional image of the end point of drilling shown by reference numeral 41 are real images 11. Here, for ease of understanding, the invisible part of the tip of the needle part 44 (the part hidden by the object W to be drilled, leading to the tip 43) is shown by a dotted line. The control means 2 may display the needle part 44 at this hidden part as a virtual three-dimensional image 12 on the display 1. When the surgeon M grasps the gripping portion 45 of the drilling mark jig 4 with his left hand and inserts the needle portion 44 into the drilling target W, the tip 43 comes into contact with the bone W1 below the muscle W2. The surgeon M determines an appropriate contact point as the drilling end point 41.

[0081] When drilling, the control means 2 reflects the virtual three-dimensional image 12 including the drilling direction extension line 31 and the like on the real image 11 of the object to be drilled and displays it on the display 1, which can be used as a drilling guide display when drilling. Here, "during drilling" refers to when drilling is about to be performed or when drilling is being performed. For example, when the operator M is about to start drilling, he / she presses a switch (not shown) provided on the drilling tool 3 to issue a display command signal for the drilling direction extension line to the control means 2 (PU21), and the control means 2 displays the drilling direction extension line 31 on the display 1. Also, for example, when the operator M is about to start drilling, he / she presses a switch (not shown) provided on the drilling mark jig 4 to issue a display command signal for the drilling end point to the control means 2 (PU21), and the control means 2 displays the drilling end point 41 on the display 1. Also, for example, when surgeon M is about to start drilling, he presses a switch (not shown) provided on display 1 (head-mounted display 6), thereby issuing a display command signal to control means 2 (PU21) to display the drilling direction extension line and the drilling end point, and control means 2 causes display 1 to display drilling direction extension line 31 and drilling end point 41. In this way, the control means 2 causes the virtual three-dimensional image 12 including the drilling direction extension line 31 and the like to be reflected on the actual image 11 of the object to be drilled and the like, and displays it on the display 1 during drilling.

[0082] In addition, if the surgeon M is a new or inexperienced person (hereinafter referred to as a "new or experienced person") and the tip 43 of the drilling mark jig 4 is not in contact with the appropriate location of the object W to be drilled (bone W1), for example, a skilled person or a person with high skills (hereinafter referred to as a "experienced person") may first contact the tip 43 of the drilling mark jig 4 with the appropriate location of the object W to be drilled (bone W1) on behalf of the surgeon M, and then, while maintaining that state, the skilled person may hand over the drilling mark jig 4 to the new or experienced surgeon M, so that the correct location is determined as the drilling end point 41.

[0083] Next, the control means 2 causes the imaging means 7 (camera) to capture an image of the drilling mark jig 4 (second mark 42) with the tip 43 of the drilling mark jig 4 in contact with the drilling end point 41 in the object W to be drilled (bone W1) (S13). In this way, the imaging means 7 (and the control means 2) acquires external appearance information of the drilling mark jig 4. It is necessary that the second mark 42 is exposed outside the object W to be drilled (bone W1) so that it can be imaged.

[0084] Then, the control means 2 creates a virtual three-dimensional image 12 of the drilling end point 41 by referring to (comparing) the appearance information (three-dimensional information) of the photographed second mark 42 (of the drilling mark jig 4) with the appearance information (three-dimensional information) of the drilling mark jig 4 previously acquired (S15). The control means 2 causes the imaging means 7 to photograph the appearance information (three-dimensional information) of the second mark 42 (of the drilling mark jig 4), thereby causing the imaging means 7 to function as appearance information acquisition means for acquiring appearance information. In this case, the drilling marking jig 4 has a plurality of second marks 42 based on the long axis direction (drilling end point direction) of the needle portion 44, and the tip 43 of the needle portion, which is located in the plane formed by the plurality of second marks 42 and the long axis direction of the needle portion 44, is the drilling end point 41.Therefore, the control means 2 creates the drilling end point 41 by referring to the plurality of second marks 42 (appearance information of the drilling marking jig 4 obtained by photographing and imaging in S13) against the appearance information obtained in advance.

[0085] Then, the control means 2 causes the display 1 to display the virtual three-dimensional image 12 of the drilling end point 41 (S17). 6(A), the control means displays on the display 1 a virtual three-dimensional image 12 (virtual three-dimensional image 12 of the drilling end point 41) reflected in an actual image 11 that is seen through the display 1 (see-through) or is captured and displayed by the imaging means 7. Also, the control means displays the point of the tip 43 (of the needle part 44) of the drilling mark jig 4 that is in contact with the bone W1 as the drilling end point 41 (virtual three-dimensional image 12).

[0086] In order for the control means 2 to display the perforation end point 41 etc. on the display 1, for example, each memory element of the RAM 23 may be made to correspond to each pixel of the display 1 (in the case of a color display, three memory elements may be made to correspond to the three primary colors (R, G, B) for each pixel), the stored contents of the memory elements may be rewritten, and the display 1 may be driven and displayed in accordance with the stored contents of the memory elements by a driver for driving the display 1. Alternatively, instead of the RAM 23, a memory means such as a semiconductor memory mounted on the display 1 (head mounted display 6) may be used to store data corresponding to each pixel of the display 1, and the display 1 may be driven and displayed by a driver for driving the display 1.

[0087] On the other hand, regarding the drilling tool 3, the surgeon M positions the drilling tool 3 toward the object W to be drilled, determining the drilling direction (S21). In addition, if the surgeon M is a novice, etc., and the drilling direction of the drilling tool 3 is not appropriate, an experienced person, etc., may temporarily point the drilling tool 3 in the appropriate direction on behalf of the surgeon M, and then hand the drilling tool 3 over to the novice, etc., surgeon M while maintaining that direction.

[0088] Next, the control means 2 causes the imaging means 7 (camera) to capture an image of the drilling tool 3 in a state where the drilling direction is determined and the drilling tool 3 is positioned (positioned) toward the object W to be drilled (S23). This allows the imaging means 7 (control means 2) to acquire appearance information of the drilling tool 3. In this case, it is necessary that the first mark 32 is exposed outside the object W to be drilled (bone W1) and can be captured. The control means 2 creates a virtual three-dimensional image 12 of the drilling direction extension line 31 by referring to (comparing) the appearance information (three-dimensional information) of the first mark 32 of the drilling tool 3 captured with respect to the appearance information (three-dimensional information) of the drilling tool 3 previously acquired (S25). The control means 2 causes the imaging means 7 to capture the appearance information (three-dimensional information) of the first mark 32 (of the drilling tool 3), thereby causing the imaging means 7 to function as an appearance information acquisition means for acquiring appearance information. Here, since the drilling tool 3 has a plurality of first marks 32 based on the drilling direction, the control means 2 creates the drilling direction extension line 31 by referring to the plurality of first marks 32 (appearance information of the drilling tool 3 obtained by photographing and imaging in S23) in relation to the appearance information obtained in advance (similar to creating the drilling end point 41 by referring to a plurality of second marks 42).

[0089] The control means 2 causes the display 1 to display the virtual three-dimensional image 12 of the drilling direction extension line 31, similarly to S17 (S27, see FIG. 6(B)).

[0090] In this way, the control means 2 reflects (superimposes) a virtual three-dimensional image (a virtual three-dimensional image of the drilling end point 41 and the drilling direction extension line 31) 12 on (or overlays) a real image 11 (real images of the object to be drilled W, the drilling tool 3, the drilling mark jig 4, etc.) seen through the display 1 or captured by the imaging means 7 and displayed on the display 1, and displays the image on the display 1 (S31, see Figure 6 (B)).

[0091] In addition, when the display 1 is an image in which the real image 11 is seen through the display or an image captured, and in addition to the virtual three-dimensional image 12 of the drilling end point 41 and the drilling direction extension line 31, the virtual three-dimensional image 12 of the drilling tool 3 and / or the drilling mark jig 4 is displayed superimposed on these real images, the control means 2 displays on the display 1 the position, inclination and size (magnification) of the virtual three-dimensional image 12 of the drilling tool 3 and / or the drilling mark jig 4 so as to match those of the real image 11. Furthermore, the control means 2 may create a virtual three-dimensional image 12 of a part (or all) of the appearance of the drilling tool 3 and / or the drilling mark jig 4 in addition to the drilling end point 41 and the drilling direction extension line 31, and display it superimposed on the real image 11. In this way, the position, inclination and size (magnification) of these virtual three-dimensional images 12 match the actual images 111, and it is easy to recognize whether the drilling end point 41 and the drilling direction extension line 31 are correct or not.

[0092] The control means 2 may also display the virtual three-dimensional images 12 of the drilling direction extension line 31 and the drilling end point 41 on the display 1 in a display mode different from that of the real image 11, so that it is easier to recognize that these are not the real images 11. For example, when the real image 11 has a color tone based on black and white, the different display mode may be a color (red, blue, gold, silver) different from the color tone, or a blinking display. The control means 2 may display the virtual three-dimensional image 12 of the drilling direction extension line 31 and the virtual three-dimensional image 12 of the drilling end point 41 in the same display mode (for example, the same color, the same blinking), or in different display modes (for example, different colors, one blinking and the other not blinking).

[0093] Furthermore, when the position (posture, inclination, etc.) of the drilling tool 3 changes, the control means 2 detects the change by imaging with the imaging means 7, changes the direction of the drilling direction extension line 31 in response to the change, and displays it on the display 1. Furthermore, when the position of the drilling mark jig 4 changes, the control means 2 detects the change by imaging with the imaging means 7, changes the position of the drilling end point 41 in response to the change, and displays it on the display 1.

[0094] The display 1 may be either a display that allows the real image 11 to be viewed through a see-through system, or a display that displays an image of the real image 11 photographed (captured) by an imaging means 7. When the display 1 is a display that allows the real image 11 to be viewed through perspective such as see-through, the main image seen on the display 1 is the real image 11 seen through the display 1 or the like. Furthermore, when the display 1 is a display that displays an image of a real image 11 captured (photographed) by the imaging means 7, if it is a visible display, the main image seen on the display 1 is the real image 11 captured by the imaging means 7.

[0095] The control means 2 reflects (superimposes) a virtual three-dimensional image 12 including a drilling direction extension line 31 and a drilling end point 41 on a real image 11 that is seen through a see-through device or is imaged and displayed. The control means 2 may display on the display 1 a virtual three-dimensional image 12 reflected (superimposed) on the real image 11, and may only display the drilling direction extension line 31 and the drilling end point 41, but it may also display other virtual three-dimensional images 12 reflected (superimposed) as well.

[0096] For example, a virtual three-dimensional image 12 of the parts that cannot be seen by the surgeon M due to the drilling tool 3, the drilling mark jig 4, etc. may be created and displayed. In this way, the invisible parts of the drilling tool 3, etc. are displayed as the virtual three-dimensional image 12, making it possible to perform more accurate drilling.

[0097] Also, for example, a virtual three-dimensional image 12 of the portion that can be visually observed by the surgeon M (portion that can be imaged by the imaging means 7) of the drilling tool 3, the drilling mark jig 4, etc. may be created and displayed. In this way, if the virtual three-dimensional image 12 of the drilling tool 3, etc. matches with their actual images 11, it is clear that the extension direction of the drilling direction extension line 31 and the position of the drilling end point 41 are accurate, and more accurate drilling becomes possible. Conversely, if they do not match, the extension direction of the drilling direction extension line 31 and the position of the drilling end point 41 may be inaccurate, making it possible to prevent inaccurate drilling.

[0098] The step (d) is Step (d1): A step of acquiring appearance information of the drilling tool 3, referring to the appearance information, creating a virtual three-dimensional image 12 including a drilling direction extension line 31, and reflecting the virtual three-dimensional image 12 in a real image 11 and displaying it on the display 1. Step (d2): A step of acquiring appearance information of the drilling mark jig 4, creating a virtual three-dimensional image 12 including the drilling end point 41 by referring to the appearance information, and reflecting the virtual three-dimensional image 12 in the real image 11 and displaying it on the display 1. It may be done like this. In other words, the perforation method may be a perforation method including steps (a), (b), (c), (d1), (d2) and (e).

[0099] Step (e) is a step of drilling the object W (W1) to be drilled based on the drilling direction extension line 31 and the drilling end point 41. This step corresponds to S33 in FIG. In this step, the surgeon M performs drilling based on the drilling direction extension line 31 and the drilling end point 41 (the virtual three-dimensional image 12 of these) as drilling guide displays (S33).

[0100] Arrangement (position) of drilling tool 3 and drilling mark jig 4 7 is a diagram for explaining the arrangement of the drilling tool 3 and the drilling mark jig 4 when drilling holes with the drilling device 100 according to embodiment 1. The area enclosed by a square on the display 1 is visually observed. As shown in Fig. 7(A), when the arrangement (position) of the drilling tool 3 and the drilling mark jig 4 is such that, in relation to the imaging means 7, the first mark 32 can be imaged but the second mark cannot be imaged (see the dotted arrow from the imaging means 7 in the figure. Note that this dotted arrow is drawn for ease of understanding and is not actually displayed), a virtual three-dimensional image 12 of the drilling direction extension line 31 can be created but a virtual three-dimensional image 12 of the drilling end point 41 cannot be created. This makes it difficult to perform accurate drilling.

[0101] On the other hand, as shown in Fig. 7(B), when the position of the drilling tool 3 and the drilling mark jig 4 is such that, in relation to the imaging means 7, both the first mark 32 and the second mark can be imaged (see the dotted arrow from the imaging means 7 in the figure. Note that this dotted arrow is drawn for ease of understanding and is not actually displayed. The same applies to Figs. 14 and 15 described below), a virtual three-dimensional image 12 of the drilling direction extension line 31 and the drilling end point 41 can be created. This makes it possible to perform accurate drilling. 7(B), the surgeon M positions the drilling tool 3 and the drilling mark jig 4 in relation to the imaging means 7 so that the first mark 32 and the second mark can be imaged by the imaging means 7. The same applies when the imaging means 7 is not mounted on the display 1 (head mounted display 6) but is mounted or installed in another location.

[0102] Modification of the second mark 42 Fig. 8 is a diagram for explaining modified examples of the second mark 42 (of the drilling mark jig 4) in the drilling device 100 according to embodiment 1. Fig. 8(A) to Fig. 8(G) are diagrams for explaining each modified example. In FIG. 8(A), the second marks 42 are provided on the outer periphery of the needle portion 44, and are provided in a straight line. In FIG. 8(B), the second marks 42 are provided in a straight line on the grip portion 45. In FIG. 8(C), the second marks 42 are provided on the grip portion 45 in a straight line connecting the multiple second marks 42 based on the long axis direction of the needle portion 44. In FIG. 8(D), three second marks 42 are provided on the grip portion 45, symmetrically with respect to the axial direction of the needle portion 44. In FIG. 8(E), the second marks 42 are provided in a dispersed manner on the needle portion 44 and the grip portion 45. In FIG. 8(F), the second marks 42 are provided on the grip portion 45 as a pattern or symbol. In FIG. 8(G), the second marks 42 are provided in a pattern over a wide range of the grip portion 45.

[0103] Modifications of the first mark 32 Figure 9 is a diagram for explaining modified examples of the first mark 32 (of the drilling marker jig 4) in the drilling device 100 according to embodiment 1. Figures 9(A) to 9(G) are diagrams for explaining the respective modified examples, and are similar to Figures 8(A) to 8(G). In FIG. 9(A), the first marks 32 are provided on the outer circumference of the drill 34, and are provided in a straight line. In FIG. 9(B), the first marks 32 are provided in a straight line on the gripping portion 35. In FIG. 9(C), the first marks 32 are provided on the gripping portion 35 based on the longitudinal direction of the drill 34. In FIG. 9(D), three first marks 32 are provided on the gripping portion 35, symmetrically with respect to the axial direction of the drill 34. In FIG. 9(E), the first marks 32 are provided in a dispersed manner on the drill 34 and the gripping portion 35. In FIG. 9(F), the first marks 32 are provided on the gripping portion 35 as a pattern or symbol. In FIG. 9(G), the first marks 32 are provided in a pattern over a wide range of the gripping portion 35.

[0104] The above-mentioned punching device 100 has the effects described in [1][2][3][6][8][9]

[10]

[13] above. In addition, the drilling method has the effects described in

[14] above.

[0105] [Embodiment 2] Fig. 10 is a diagram for explaining a punching device 200 according to embodiment 2. Embodiment 2 will be explained with reference to Fig. 10 showing how it appears on the display 1, and drawings of other parts will be omitted (the same applies to other embodiments). The drilling apparatus 200 according to the second embodiment is basically the same as the drilling apparatus 100 according to the first embodiment, but differs in that it further comprises a passing notification means for notifying that the drilling direction extension line 31 of the drilling tool 3 passes the drilling end point 41 or that the drilling direction extension line 31 of the drilling tool 3 does not pass the drilling end point 41. In the drilling apparatus 200, a drilling end point passing notification display unit 84a is provided on the display 1, which serves as a passing notification means for displaying whether the drilling direction extension line 31 passes the drilling end point 41 or not.

[0106] As shown in Figure 10 (A), when the drilling direction extension line 31 of the drilling tool 3 passes through the drilling end point 41, the control means 2 displays a ``○'' mark and the message ``The drilling direction extension line passes through the drilling end point'' on the drilling end point passing notification display unit 84a. On the other hand, if the drilling direction extension line 31 of the drilling tool 3 does not pass through the drilling end point 41, as shown in Figure 10 (B), the control means 2 displays an "X" mark and the message "The drilling direction extension line does not pass through the drilling end point" on the drilling end point passing notification display unit 84a. In addition, it is acceptable to have either a mark such as "o" or "x" or a message. Furthermore, the control means 2 may also use sound (buzzer, music, voice, etc.).

[0107] The punching device 200 according to the second embodiment has the effects described in [4] above. The drilling device 200 of embodiment 2 is similar to the drilling device 100 of embodiment 1 except that it is equipped with a passing notification means (punching end point passing notification display unit 84a) that indicates whether the drilling direction extension line 31 passes through the drilling end point 41 or not, and therefore also has the relevant effects among the effects of the drilling device 100 or drilling method of embodiment 1.

[0108] [Embodiment 3] FIG. 11 is a diagram for explaining a punching device 300 according to the third embodiment. The drilling device 300 according to the third embodiment is basically the same as the drilling device 100 according to the first embodiment, but differs in that it further comprises a drilling notification means for notifying that the tip of the drilling part of the drilling tool 3 is approaching or has reached the drilling end point 41. In the drilling device 300, a drilling end point distance notification display unit (85a, 85b) is provided on the display 1, which serves as a drilling notification means for notifying that the tip of the drilling part of the drilling tool 3 is approaching or has reached the drilling end point 41.

[0109] 11(A), when the tip of the drilling part of the drilling tool 3 approaches the drilling end point 41, the control means 2 of the drilling device 200 of the second embodiment causes the drilling end point distance notification display unit 85a to display "5 mm to the drilling end point (tip of the drilling part)". The control means 2 also causes an arrow pointing from the tip of the drill 34 toward the drilling end point 41 and "5 mm remaining" to be displayed near the drilling direction extension line 31 (drilling end point distance notification display unit 85b). On the other hand, when the tip of the drilling part of the drilling tool 3 reaches the drilling end point 41, as shown in Fig. 11(B), the control means 2 displays "(Tip of drilling part) Reached drilling end point" (Drilling end point distance notification display unit 85a). The control means 2 also displays "Reached" near the drilling direction extension line 31 (Drilling end point distance notification display unit 85b). Either one of the drilling end point distance notification display sections 85a and 85b may be used. The display of the drilling end point distance notification display section 85b may be either a graphic such as an arrow or a message such as "5 mm left." Furthermore, the control means 2 may also use sound (buzzer, music, voice, etc.).

[0110] The punching device 300 according to the third embodiment has the effects described in [5] above. The drilling device 300 of embodiment 3 is similar to the drilling device 100 of embodiment 1 except that it further comprises a drilling notification means for notifying that the tip of the drilling portion of the drilling tool 3 is approaching or has reached the drilling end point 41, and therefore also has the corresponding effects of the drilling device 100 or drilling method of embodiment 1.

[0111] [Embodiment 4] FIG. 12 is a diagram for explaining a punching device 400 according to the fourth embodiment. The drilling apparatus 400 of embodiment 4 is basically the same as the drilling apparatus 100 of embodiment 1, but differs in that it is equipped with a position maintaining means (drilling marker jig mounting fixture 48) that maintains the position of the drilling marker jig 4 relative to the object W to be drilled.

[0112] 12, a drilling device 400 according to the fourth embodiment has a drilling mark jig attachment tool 48 having multiple gripping parts (48a, 48c). One gripping part 48a among the multiple gripping parts is configured to be able to grip the drilling mark jig 4, and another gripping part 48c is configured to be able to grip the object to be drilled W. Each gripping part (48a, 48c) has a spring (48f, 48g) and grips the drilling mark jig 4 and the object to be drilled W by its force, respectively. These gripping parts (48a, 48c) are connected to arms (48b, 48d), and the arms (48b, 48d) are connected by a rotatable joint 48e, so that the positional relationship between the gripping parts 48a, 48c is maintained constant (kept), and even if the surgeon M is not holding the drilling mark jig 4, the arrangement position (posture) of the drilling mark jig 4 with respect to the target W to be drilled is maintained (the position of the point where the tip 43 of the drilling mark jig 4 contacts the target W (W1) is maintained). And the display position of the drilling end point 41 is also maintained. In addition, members such as bolts and nuts may be used instead of the springs (48f, 48g).

[0113] The punching device 400 according to the fourth embodiment has the effects described in

[12] above. The drilling apparatus 400 of embodiment 4 is similar to the drilling apparatus 100 of embodiment 1 except that it further comprises a position maintaining means (drilling marker jig mounting fixture 48) for maintaining the position of the drilling marker jig 4 relative to the object W to be drilled, and therefore has the corresponding effects of the drilling apparatus 100 or drilling method of embodiment 1.

[0114] [Embodiment 5] FIG. 13 is a diagram for explaining a punching device 500 according to the fifth embodiment. The drilling device 500 of embodiment 5 is basically the same as the drilling device 100 of embodiment 1, but differs in that it is equipped with a position maintaining means for maintaining the position of the drilling marker jig 4 relative to the object W to be drilled. The drilling apparatus 500 of embodiment 5 (see Figure 13) is similar to the drilling apparatus 400 of embodiment 4 (see Figure 12) in that it has a position maintaining means for maintaining the position relative to the object to be drilled W. However, the difference is that in embodiment 4 the position maintaining means (drilling marker jig mounting fixture 48) for maintaining the position relative to the object to be drilled W is separate from the drilling marker jig 4, whereas in embodiment 5 the position maintaining means (drilling marker jig mounting fixtures 49A, 49B) for maintaining the position relative to the object to be drilled W are integrated with the drilling marker jig (4A, 4B).

[0115] Figure 13 is a diagram for explaining the drilling device 500 of embodiment 5, where Figure 13(A) is a diagram for explaining the state in which the drilling marker jig 4A is placed in a predetermined location relative to the object W (W1) to be drilled, as viewed on the display 1, Figure 13(B) is a diagram for explaining the placed state, Figure 13(C) is a cross-sectional view of the placed location, and Figure 13(D) is an explanatory diagram of a modified example.

[0116] In the drilling device 500 of embodiment 5, as shown in Figures 13(A), 13(B) and 13(C), the drilling mark jig (4A) is integrated with a position maintaining means (drilling mark jig mounting fixture 49A) that maintains its position relative to the object to be drilled W, thereby maintaining the position (posture) of the drilling mark jig (4A) relative to the object to be drilled W. The drilling mark jig 4A has a screw portion 45a (female screw) and a needle portion 44a, and when rotated, the tip 43a drills a hole in the object W (W1) to be drilled. The screw portion 45a (female screw) may extend to the tip 43a without the needle portion 44a. The screw portion 45a (female screw) does not need to be provided over the entire drilling direction, and may be provided only in a part of it.

[0117] The drilling mark jig attachment tool 49A has a screw part 49a2 (female screw) and a visor part 49a1 (a part that protrudes like a visor around the screw part 49a2). The screw part 49a2 (female screw) has a configuration (screw pitch, unevenness, etc.) corresponding to the screw part 45a (male screw) provided on the drilling mark jig 4A. The visor part 49a1 has a surface such as a flat surface or a curved surface (concave surface, convex surface) at a portion facing the drilling target W (W1), and the surface helps to maintain the position of the drilling target W (W1) constant. An adhesive (including a sticky material) may be further provided between the drilling target W (W1). If the surface of the drilling target W (W1) has unevenness, a corresponding unevenness may be provided so that the two engage with each other. By rotating the drilling mark jig 4A a predetermined number of times in a direction intersecting the drilling direction (by turning the screw), the depth to which the tip 43a drills into the target W (W1) is adjusted, and the eaves portion 49a1 maintains the position of the drilling mark jig 4A relative to the target W (W1). In this way, it is configured like a kind of insect pin or thumb tack.

[0118] The drilling mark jig 4A has a second mark 42a provided at a location such that the tip 43a remains outside the object W to be drilled even when it penetrates inside the object W to be drilled and reaches the end point of drilling.

[0119] A modified example of the punching device 500 according to the fifth embodiment will be described with reference to FIG. In this modification, a drilling mark jig 4B is used, but instead of the drilling mark jig attachment tool 49A (drilling mark jig 4A) shown in Figures 13(A) to (C), a drilling mark jig attachment tool 49A is used as shown in Figure 13(D). The drilling mark jig attachment tool 49B is a drilling mark jig attachment tool 49A to which a suction cup 49a3 has been added (added). The suction cup 49a3 is joined to the drilling mark jig 4B around the central screw portion 49a2, and is not joined at other points. By pressing the drilling mark jig attachment fixture 49B against the object W (W1) to be drilled, air is released between the suction cup 49a3 and the object W (W1), and the drilling mark jig 4B (drilling mark jig attachment fixture 49B) is adsorbed to the object W (W1). In addition, in FIGS. 13(A) to (D), the threaded portion 45a is a male thread and the threaded portion 49a2 is a female thread, but the threaded portion 45a may be a female thread and the threaded portion 49a2 may be a male thread.

[0120] The punching device 500 according to the fifth embodiment has the effects described in

[12] above. The drilling apparatus 500 of embodiment 5 is similar to the drilling apparatus 100 of embodiment 1 except that it further comprises a position maintaining means (drilling marker jig mounting fixtures 49A, 49B) for maintaining the position of the drilling marker jig (4A, 5B) relative to the object W to be drilled, and therefore has the corresponding effects of the drilling apparatus 100 or drilling method of embodiment 1.

[0121] [Embodiment 6] FIG. 14 is a diagram for explaining a punching device 600 according to the sixth embodiment. The drilling apparatus 600 of embodiment 6 is basically the same as the drilling apparatus 100 of embodiment 1, but differs from the drilling apparatus 100 of embodiment 1 in that the control means 2 displays the drilling end point 41 on the display 1 so that the display position does not change even if the position of the drilling marker jig 4 is changed.

[0122] 14(A), when the drilling mark jig 4 is positioned, the drilling end point 41 at the display position determined from the position of the position is displayed on the display 1 by the control means 2. The drilling tool 3 is also positioned such that its drilling direction faces the drilling end point 41, and the control means 2 displays a drilling direction extension line 31 on the display 1. This is similar to the drilling device 100 of the first embodiment.

[0123] However, in embodiment 6, even if the position of the drilling mark jig 4 is subsequently changed, when the control means 2 receives a command to maintain the drilling end point display position, it displays the drilling end point 41 determined at the specified position on the display 1 without changing its display position (see Figure 14 (B)). When the surgeon M wishes to maintain the display position of a certain drilling end point 41 (or a certain drilling end point 41 and a certain drilling direction extension line 31) as it is regardless of the physical position of the drilling end point 41 (when the surgeon M wishes to maintain the display position at a certain point), the surgeon M issues a drilling end point display position maintenance command. For example, the surgeon M issues a drilling end point display position maintenance command (signal) by pressing a switch (not shown) provided on the drilling tool 3, and the CPU 21 (control means 2) detects that the command has been issued and executes the command, thereby maintaining the display of the drilling end point 41 (or a certain drilling end point 41 and a certain drilling direction extension line 31) at a certain display position.

[0124] In this case, it is convenient to provide a drilling end point display (position) maintenance mode display section 88a to display "Drilling end point display maintenance mode" and to indicate that the control means 2 is in the drilling end point display maintenance mode in which the display position of the drilling end point 41 is maintained regardless of the position of the drilling mark jig 4. 14(B) is a diagram showing a state in which the control means 2 displays the drilling end point 41 determined at the predetermined arrangement position on the display 1 without changing the display position. In this case, even though the surgeon M releases (removes, etc.) the drilling mark jig 4 held in his left hand and the drilling mark jig 4 is not directed toward the drilling target W (not in the predetermined arrangement), the control means 2 displays the drilling end point 41 on the display 1 without changing the display position once determined.

[0125] Furthermore, even if the drilling end point 41 is once determined, the object W (W1) to be drilled may move, and the position (location) of the drilling end point 41 determined as optimal may actually change, but the same position may continue to be displayed as before the change. In such a case, the control means 2 may display a warning on the display 1 that the drilling end point 41 may not be displayed in the correct position. Whether or not the object W (W1) to be punched has moved can be detected, for example, by providing a third mark W3 (a mark for detecting a change in the position of the object W (W1) to be punched) on the object W (W1) to be punched by attaching a sticker with the mark or by writing the mark by hand, and then capturing an image of this state with imaging means 7.

[0126] For example, the control means 2 may display a warning such as "Warning: Not displayed correctly" or "The object to be drilled has changed. The drilling end point or drilling direction extension line may not be displayed correctly" on the display 1 to indicate that the drilling end point 41 may not be displayed correctly. Alternatively, the area in which the drilling end point 41 or the drilling direction extension line 31 that may not be displayed correctly is displayed may be surrounded as a warning area display area 88d, or may be displayed in white, black, or gray, colored, with diagonal lines, or with a mesh pattern, etc. In addition, 88b, 88c, and 88d may be 88b alone, 88c alone, 88d alone, a combination of 88b and 88c, a combination of 88b and 88d, or a combination of 88c and 88d. Furthermore, the control means 2 may also use sound (buzzer, music, voice, etc.).

[0127] The punching device 600 according to the sixth embodiment has the effects described in

[11] above. The drilling device 600 of embodiment 6 is similar to the drilling device 100 of embodiment 1 except that the control means 2 displays on the display 1 the display position of the drilling end point 41 such that once determined, the display position does not change even if the position of the drilling marker jig 4 is changed, and therefore has the corresponding effects of the drilling device 100 or drilling method of embodiment 1.

[0128] [Embodiment 7] Fig. 15 is a diagram for explaining a punching device 700 (700A, 700B) according to embodiment 7. Fig. 15(A) is a diagram for explaining the punching device 700A, and Fig. 15(B) is a diagram for explaining 700B. The drilling apparatus 700 (700A, 700B) of embodiment 7 is basically the same as the drilling apparatus 100 of embodiment 1, but differs in that while in embodiment 1 the jig contact point 430 is the drilling end point 41, in embodiment 7 the drilling end point 41 is a point a certain distance 440 away from the jig contact point 430 on a line connecting the jig contact point 430 and the drilling start point 38 where the drilling tool 3 starts drilling the object W (W1).

[0129] In the drilling apparatus 700A of embodiment 7, as shown in Figure 15 (A), the control means 2 causes the display 1 to display a point located a certain distance 440 away from the jig contact point 430 on a straight line 431 connecting the jig contact point 430 and the drilling start point 38 where the drilling tool 3 starts drilling the object W (W1) to be drilled, as the drilling end point 41.

[0130] For example, in order to avoid damaging sensitive tissues such as blood vessels and nerves in the drilling target W (W1), it may be necessary to stop drilling the drilling tool 3 deep into the drilling target W (W1) or to stop drilling midway without penetrating a hole (hole). In such a case, for example, the operator M presses a switch (not shown) provided on the drilling tool 3 to issue a signal commanding that a point on a straight line 431 connecting the jig contact point 430 and the drilling start point 38 where the drilling tool 3 starts drilling the drilling target W (W1) be a point a certain distance 440 away from the jig contact point 430, is set as the drilling end point 41, and the CPU 21 receives the signal to execute the command. The certain distance 440 is stored in the ROM 22 in advance, or the operator M communicates it to the CPU 21 using a switch (not shown) provided on the drilling tool 3. Then, the control means 2 causes the display 1 to display a point spaced a certain distance 440 from the jig contact point 430 as the drilling end point 41 . A straight line 431 connecting the jig contact point 430 and the drilling start point 38 (a straight line passing through both points) is a straight line that coincides with the drilling direction extension line 31.

[0131] In this case, if the control means 2 causes the display 1 to also display the jig contact point 430, it becomes even easier to determine or change the drilling end point 41. Furthermore, if the control means 2 causes the display 1 to display the jig contact point 430 in a display mode that is different in size, color, etc. from the drilling end point 41, it becomes easy to distinguish between the two. In addition, when the display of the drilling direction extension line 31 is only the display of the virtual three-dimensional image 12 between the drilling start point 38 and the drilling end point 41, the control means 2 may cause the display 1 to display the virtual three-dimensional image 12 of the straight line between the jig contact point 430 and the drilling end point 41. Both straight lines may be displayed in a display mode with different line colors, thicknesses, etc.

[0132] The drilling device 700B shown in Fig. 15(B) is similar to the drilling device 700A shown in Fig. 15(A) in that the control means 2 displays on the display 1 a point a certain distance 440 away from the jig contact point 430 on a line 431 connecting the jig contact point 430 and the drilling start point 38 where the drilling tool 3 starts drilling the object W (W1) as the drilling end point 41. However, it differs in that the longitudinal direction (straight line) of the needle part 44 of the drilling mark jig 4 does not coincide with the straight line 431 in Fig. 15(A), whereas it coincides with the straight line 431 in Fig. 15(B). The straight line 431 can also be referred to as the drilling direction extension line 31.

[0133] Furthermore, if the control means 2 causes the display 1 to display a different display mode (for example, changing from a non-blinking display to a blinking display, making the line thicker, changing the color, etc.) when the longitudinal direction (straight line) of the needle part 44 of the drilling mark jig 4 coincides with the straight line 431 (drilling direction extension line 31) from when it does not coincide, accurate drilling becomes even easier. Other points are the same as those of the punching device 700A shown in FIG. 15(A).

[0134] The drilling device according to the seventh embodiment has the corresponding effects described in [6] and [7] above. The drilling device 700 (700A, 700B) of embodiment 7 is similar to the drilling device 100 of embodiment 1 except that the drilling end point 41 is a point located a certain distance 440 away from the jig contact point 430 on a straight line connecting the jig contact point 430 and the drilling start point 38, and therefore has the corresponding effects of the drilling device 100 or drilling method of embodiment 1.

[0135] [Embodiment 8] The drilling device of embodiment 8 is basically the same as the drilling device 100 of embodiment 1, but differs from the drilling device 100 of embodiment 1 in that it uses a bone wire instead of the drilling tool 3.

[0136] The drilling device according to the eighth embodiment is similar to the drilling device 100 according to the first embodiment except that a bone wire is used instead of the drilling tool 3, and therefore a description thereof will be omitted.

[0137] Conventionally, when the subject W (W1) has large fracture deformations and is unstable in various locations, percutaneous bone wire insertion is performed as a surgical treatment. The direction and depth of bone wire insertion during surgery are determined by visual inspection and X-ray fluoroscopy. In contrast to this, in the eighth embodiment, it is possible to accurately grasp the direction and depth of bone wire insertion without requiring a large-scale device. In addition, conventionally, bone wires are inserted after fracture reduction during surgery, so if a CT scan is taken before surgery, the CT image cannot be used as is. In contrast, in the eighth embodiment, it is possible to insert the bone wire accurately without the need for CT imaging before surgery. In particular, doctors with little surgical experience have the problem of not knowing in which direction and to what depth to insert the bone wire, but by using the method of embodiment 8, this problem can be resolved. Furthermore, when taking images using X-rays, there is a problem of exposure of the doctor (operator M) and the patient (perforation subject W) to radiation, but by using the method according to embodiment 8, it is possible to solve this problem.

[0138] The drilling device of embodiment 8 is similar to the drilling device 100 of embodiment 1 except for using a bone wire instead of the drilling tool 3, and therefore has the corresponding effects of the drilling device 100 or drilling method of embodiment 1.

[0139] [Embodiment 9] FIG. 16 is a diagram for explaining a punching device 900 according to the ninth embodiment. The drilling device 900 of embodiment 9 is basically the same as the drilling device 100 of embodiment 1, but differs in that the person drilling is a carpenter M9 rather than an operator M, and that the object to be drilled is wood W9 (wooden boards W91, W92) rather than a living body.

[0140] 16, the drilling device 900 according to the ninth embodiment is a drilling target W9 (W91, W92) which is wood W9 (wooden boards W91, W92). The drilling device 900 is a drilling device for a carpenter M9 to stack the boards W91 and W92 of the drilling target W9 (wooden), drill holes in them, and insert bolts into the holes (holes) and fix them with nuts.

[0141] The drilling device 900 of embodiment 9 is similar to the drilling device 100 of embodiment 1 except that the person drilling is a carpenter M9 rather than an operator M, and the object to be drilled is wood W9 (wooden board) rather than a living body, and therefore has the relevant effects of the drilling device 100 or drilling method of embodiment 1.

[0142] [Embodiment 10] 17 is a diagram for explaining a drilling device 1000 according to embodiment 10. The drilling device 1000 according to embodiment 10 is basically the same as the drilling device 100 according to embodiment 1, but differs in that the person who drills holes is a steel frame worker M10 instead of an operator M, and that the object to be drilled is a steel frame W10 instead of a living body.

[0143] 17, the drilling device 1000 according to the tenth embodiment is configured such that a drilling target W10 is a steel frame (a steel frame pillar). The drilling device 1000 is used by a steel frame construction worker M10 to drill holes in the steel frame W10, which is the drilling target, and pass electric wires or pipes through the holes (holes).

[0144] The drilling device 1000 of embodiment 10 is similar to the drilling device 100 of embodiment 1 except that the person performing the drilling is a steel construction worker M10 rather than an operator M, and the object to be drilled is a steel frame W10 rather than a living body, and therefore has the relevant effects among the effects of the drilling device 100 or drilling method of embodiment 1.

[0145] [Embodiment 11] 18 is a diagram for explaining a drilling device 1100 according to embodiment 11. The drilling device 1100 according to embodiment 11 is basically the same as the drilling device 100 according to embodiment 1, but differs in that the person who drills holes is a concrete wall construction worker M11 instead of an operator M, and that the object to be drilled is a concrete wall W11 instead of a living body.

[0146] 18, the drilling device 1100 according to the eleventh embodiment is a drilling target W11 which is a concrete wall. The drilling device 1100 is a drilling device for a concrete wall construction worker M11 to drill holes in the concrete wall W11 which is the drilling target, and install a guide plate.

[0147] The drilling device 1100 of embodiment 11 is similar to the drilling device 100 of embodiment 1 except that the person who drills is a concrete wall construction worker M11 rather than an operator M, and the object to be drilled is a concrete wall W11 rather than a living body, and therefore also has the relevant effects of the drilling device 100 or drilling method of embodiment 1.

[0148] [Embodiment 12] 19 is a diagram for explaining a drilling device 1200 according to embodiment 12. The drilling device 1200 according to embodiment 12 is basically the same as the drilling device 100 according to embodiment 1, but further includes a two-dimensional perspective image acquiring means 81 for acquiring a two-dimensional perspective image 8 of the object W1 to be drilled by irradiating the object W1 to be drilled with a detection wave 84, and the two-dimensional perspective image 8 is reflected (superimposed) on the real image 11 or the virtual three-dimensional image 12 and displayed on the display 1.

[0149] In the twelfth embodiment, an X-ray imaging device is used as the two-dimensional fluoroscopic image acquiring means 81 (see FIG. 19). The two-dimensional fluoroscopic image acquiring means 81 (X-ray imaging device) generates a detection wave 84 (X-ray) from a detection wave generating unit 82 (X-ray tube) and irradiates the target W1 to be drilled, and detects the detection wave 84 (X-ray) that has passed through the target W1 to be drilled with a detection unit 83 (image tube) to visualize it as a two-dimensional fluoroscopic image 8. Then, the control means 2 reflects the two-dimensional fluoroscopic image 8 in a real image 11 or a virtual three-dimensional image 12 by a general-purpose image recognition technology and displays it on the display 1. The detection wave 84 (X-ray) may be irradiated to the object W1 to be drilled all the time or continuously during drilling, but this is not necessarily required. For example, in order to reduce the risk of X-ray exposure, it may be irradiated only once, or the upper limit of the number of times may be determined, such as twice or three times.

[0150] FIG. 20 is a diagram for explaining a perforation device 1200 according to embodiment 12, and FIGS. 20(A) to (D) are diagrams for explaining what appears on the display 1 at each stage of perforation.

[0151] First, before the detection wave 84 is irradiated onto the object W1 to be drilled, it looks as shown in FIG. 20(A).

[0152] Then, when the detection wave 84 is irradiated, a two-dimensional perspective image 8 of the object W1 to be drilled (the outline of the object W1 to be drilled and its diagonal broken portion shown by dotted lines) is reflected (superimposed) on a real image 11 of the object W1 to be drilled etc. (or its virtual three-dimensional image 12) and displayed (see Figure 20 (B)).

[0153] Next, the surgeon M moves the drilling mark jig 4 while viewing the two-dimensional perspective image 8 of the target W1 to determine the drilling end point 41 (see FIG. 20(C)). The determined drilling end point 41 is displayed on the display 1 by the control means 2 as a circle (or in red).

[0154] Then, the operator M moves the drilling tool 3 while viewing the virtual three-dimensional image 12 of the drilling end point 41 reflected (superimposed) and displayed on the real image 11, the two-dimensional perspective image 8 of the object W1 to be drilled, etc., so that the drilling direction extension line 31 passes through the drilling end point 41 (see FIG. 20(D)). When it passes through, the control means 2 changes the drilling end point 41 to, for example, a square mark (or green).

[0155] In addition, once the drilling end point 41 is determined by the surgeon M, even if the drilling mark jig 4 moves away from the target W1 to be drilled or the target W1 to be drilled moves, the drilling end point 41 may be moved in accordance with the movement of the target W1 to be drilled, and the drilling end point 41 may be displayed to indicate the same position relative to the target W1 to be drilled. Such control may be performed, for example, by the control means 2 using a general-purpose image recognition technique or the like to specify the drilling end point 41 on the two-dimensional perspective image 8 of the target W1 to be drilled. This can be done.

[0156] The punching device 1200 according to the twelfth embodiment described above has the effects described in

[14] above. The drilling device 1200 of embodiment 12 further includes a two-dimensional perspective image acquisition means 81, and is similar to the drilling device 100 of embodiment 1 except for displaying a two-dimensional perspective image 8 of the object W1 to be drilled, and therefore has the corresponding effects of the drilling device 100 of embodiment 1. In addition, if the two-dimensional fluoroscopic image acquisition means is an X-ray imaging device, the effect described in

[15] above is also achieved.

[0157] [Embodiment 13] Fig. 21 is a diagram for explaining a punching device 1300 according to embodiment 13, and Figs. 21(A) to (D) are diagrams for explaining the appearance on the display 1 at each stage of punching. The punching device 1300 according to embodiment 13 is an embodiment that further embodies the punching device 1200 according to embodiment 12. The following description will be given with reference to Fig. 21.

[0158] [0 Display of 2D perspective image 8] 21(A), like Fig. 20(B) described in the twelfth embodiment, shows how a two-dimensional perspective image 8 (dotted line) of the object W1 to be drilled acquired by the two-dimensional perspective image acquiring means 81 is reflected in a real image 11 by the control means 2 and displayed on the display 1. On the display 1, the boundary between the bone W1 and the muscle W2 (two dotted lines in the vertical direction), the fractured part of the bone W1 (diagonal dotted line), etc. are displayed in the two-dimensional perspective image 8.

[0159] [1 Fixing the tip of the drilling mark jig] First, the surgeon M pierces the drilling mark jig 4 into an appropriate place in the target (W, W1) while looking at the display 1 on which the 2D perspective image 8 of the target W1 is displayed and fixes its tip (see FIG. 21(B)). Then, the surgeon M determines the jig contact point 430 (contact point with the target W1) as the drilling end point 41.

[0160] [2 Image recognition of drilling mark jig 4] Then, the control means 2 performs image recognition of the appearance, etc., of the drilling mark jig 4. The image recognition may be performed based on, for example, an image recognition command by the surgeon M using a command button (not shown), a command to determine the drilling end point 41, or the like.

[0161] [3 Virtual 3D image display of drilling end point 41] Next, the control means 2 reflects (superimposes) the virtual three-dimensional image 12 of the drilling end point 41 on the display 1, for example, as a circle (or in red), based on the recognized image of the drilling mark jig 4 (for example, a recognized image of the appearance, a second mark 42 (not shown), etc.) on the actual image 11 of the object W1 to be drilled (see FIG. 21(B)). The display of the drilling end point 41 may be performed based on a command from the surgeon M to display the drilling end point 41, a command to determine the drilling end point 41, etc.

[0162] Furthermore, even if the object to be drilled (W1) or the drilling mark jig 4 moves (moves) slightly, for example, the drilling end point 41 may be moved in accordance with the movement based on those images to display the correct location. Furthermore, when the once determined drilling end point 41 moves and the distance connecting the points before and after the movement becomes equal to or greater than a certain value, a warning may be issued by displaying a warning message, erasing the display of the drilling end point 41, emitting a warning sound, or the like.

[0163] [4 Virtual 3D image display of drilling direction extension line 31] Next, the operator M searches for the drilling direction while looking at the display 1 on which the two-dimensional perspective image 8 of the object W1 to be drilled and the like are displayed (see FIG. 21(C)). Here, like the drilling mark jig 4, the control means 2 also performs image recognition of the appearance, etc., of the drilling tool 3. Then, based on the image of the recognized appearance, etc., a virtual three-dimensional image 12 of the drilling direction extension line 31 is superimposed on the real image 11, etc. and displayed on the display 1. Note that this display may be performed, for example, when the surgeon M issues a command to display the drilling direction extension line 31, etc.

[0164] Then the next operation, control, etc. is carried out. [5 Adjust so that the drilling direction extension line 31 passes through the drilling end point 41] [6 Display change of drilling end point 41] To explain these, while looking at the display 1 on which the two-dimensional perspective image 8 of the drilled object W1 and the like are displayed, the surgeon M adjusts the drilling direction of the drill 34 so that the drilling direction extension line 31 passes through the drilling end point 41. The control means 2 displays this state as shown in Fig. 21(C) before the passage, but once the passage is passed, the control means 2 changes the drilling end point 41 to, for example, a square mark (or green), as shown in Fig. 21(D), indicating that the drilling direction is correct (appropriate).

[0165] [7 Drilling towards drilling end point 41] Next, the surgeon M performs drilling along the drilling direction extension line 31 . Here, if the actual drilling direction shifts during drilling and the drilling direction extension line 31 no longer passes through the drilling end point 41, the control means 2 may, for example, erase the display of the drilling end point 41, return to the original display (circle or red, see Figure 21 (C)), sound an alarm, etc., to inform the user that the drilling direction is incorrect.

[0166] [8 Reaching the end point 41 of the drilling hole] Then, when the perforation reaches the perforation end point 41, the control means 2 changes the display of the perforation end point 41 to, for example, an X mark (or yellow) to notify that the perforation end point 41 has been reached (see FIG. 21(E)).

[0167] As in embodiment 12, once the drilling end point 41 is determined by the surgeon M, the drilling end point 41 may be displayed to indicate the same location relative to the object W1 to be drilled even if the drilling marker jig 4 moves away from the object W1 to be drilled or the object W1 to be drilled moves.

[0168] The punching device 1300 of the above-mentioned embodiment 13 is a more specific version of the punching device 1200 of embodiment 12, and therefore has the corresponding effects among the effects of the punching device 1200 of embodiment 12. Furthermore, by changing the display of the drilling end point 41 in accordance with whether the drilling direction extension line 31 passes through / does not pass through the drilling end point 41, it becomes possible to easily know whether the drilling direction is correct or not. Alternatively, if the display of the punching end point 41 is changed in accordance with whether the punching has reached the punching end point 41 or not, it becomes possible to easily know whether the punching has reached the punching end point 41 or not.

[0169] [Embodiment 14] FIG. 22 is a diagram for explaining a punching device 1400 according to the fourteenth embodiment. The drilling device 1400 of embodiment 14 is basically the same as the drilling device 1300 of embodiment 13, but differs in that instead of the jig contact point 430 being the drilling end point 41, the drilling end point 41 is a point a certain distance (L1) away from the jig contact point 430 in the direction opposite to the drilling progress direction (see Figure 22 (D)).

[0170] [0 Display of 2D perspective image 8] The operation, control, etc. in this step are similar to those in the thirteenth embodiment (see FIG. 21(A)) (see FIG. 22(A)).

[0171] [1 Fixing the tip of the drilling mark jig] The operation, control, etc. in this step are similar to those in the thirteenth embodiment (see FIG. 21(B)) (see FIG. 22(B)).

[0172] [2 Image recognition of drilling mark jig 4] As for the operation and control in this step, similarly to the thirteenth embodiment, the control means 2 performs image recognition of the appearance of the drilling mark jig 4, etc.

[0173] [3 Virtual 3D image display of drilling end point 41] The operations, controls, etc. in this step are the same as those in embodiment 13. The control means 2 displays the virtual three-dimensional image 12 of the jig contact point 430 on the display 1, for example, as a circle (or in red) (see FIG. 22(B)).

[0174] [4 Virtual 3D image display of drilling direction extension line 31] As in the thirteenth embodiment, the control means 2 displays a virtual three-dimensional image 12 of the drilling direction extension line 31 on the display 1 (see FIG. 22(C)).

[0175] Then the next operation, control, etc. is carried out. [5-1 Adjust so that the drilling direction extension line 31 passes through the jig contact point 430] [6 Display change of jig contact point 430] [6-2 Display of drilling end point 41] To explain these, similarly to the thirteenth embodiment, the surgeon M adjusts the drilling direction of the drill 34 so that the drilling direction extension line 31 passes through the drilling end point 41. The control means 2 displays this state as shown in Fig. 22(C) before the passage, but once the passage is completed, as shown in Fig. 22(D), changes the display of the jig contact point 430 to, for example, a square mark (or green), indicating that the drilling direction is correct (appropriate).

[0176] At the same time, the control means 2 displays a point on the drilling direction extension line 31, a certain distance L1 away from the jig contact point 430 in the opposite direction (leftward) to the drilling proceeding direction (rightward in the figure), as the drilling end point 41, for example, as a diamond (or in blue) (see FIG. 22(D)). This drilling end point 41 is, so to speak, the position where the drilling stops just before it reaches the jig contact point 430. The control means 2 determines and displays the drilling end point 41 by a software algorithm, for example, which draws a circle 435 with the jig contact point 430 as its center and a radius of the distance l1, and sets the point where the circle 435 and the drilling direction extension line 31 intersect as the drilling end point 41.

[0177] [7 Drilling towards drilling end point 41] Next, the surgeon M performs drilling along the drilling direction extension line 31 in the same manner as in the thirteenth embodiment.

[0178] [8 Reaching the end point 41 of the drilling hole] Then, when the drilling reaches the drilling end point 41, the control means 2 changes the display of the jig contact point 430 to, for example, a triangular mark (or yellow) (see FIG. 22(E)). Instead of changing the display of the jig contact point 430, the control means 2 may change the display of the drilling end point 41 to, for example, an X mark (or yellow) in the same manner as in the thirteenth embodiment.

[0179] Furthermore, once the drilling end point 41 is determined by the surgeon M, as in embodiment 12, the drilling end point 41 may be displayed to indicate the same location relative to the object W1 to be drilled, even if the drilling marker jig 4 moves away from the object W1 to be drilled or the object W1 to be drilled moves.

[0180] The drilling device 1400 of the above-mentioned embodiment 14 is similar to the drilling device 1300 of embodiment 13 except that the drilling end point 41 is a point a certain distance away from the jig contact point 430 in the direction opposite to the drilling progression direction, and therefore has the relevant effects among the effects of the drilling device 1300 of embodiment 13. In addition, since the drilling end point 41 is set to a point a certain distance away from the jig contact point 430, it is possible to set the drilling end point 41 even in a location where the tip of the drilling mark jig 4 cannot contact the object W1 to be drilled, and it has the effect of preventing over-drilling (for example, drilling until the object W1 to be drilled is completely penetrated).

[0181] [Embodiment 15] FIG. 23 is a diagram for explaining a hole-drilling device 1500 according to the fifteenth embodiment, and FIG. 24 is a diagram for explaining hole-drilling and the like in the case where a plate P is provided in the fifteenth embodiment. The drilling device 1500 according to the fifteenth embodiment is basically the same as the drilling device 1300 according to the thirteenth embodiment, but differs in that a plate P is disposed outside the object to be drilled (W1) (see FIGS. 23 and 24). Here, the plate P is screwed to the drilling target W1 by a screw 95 (thread) and is used to fix the fractured part of the drilling target W1. In addition, a hole P01 is provided in the plate P to guide the screw 95 into the plate P (see Figs. 24(B) and (C)). In addition to a method of preventing the tip of the screw 95 from protruding out of the drilling target W1 (see Fig. 24(B)), there is a method of protruding the tip slightly (see Fig. 24(C)). According to the former method, injury due to the tip is suppressed, and according to the latter method, the effect of the drilling target W1 (its bone cortex) and the screw 95 is further strengthened (note that in this embodiment, the tip is covered with the muscle W2, so injury due to the tip is unlikely to occur). For the plate P and the screw 95, for example, metals such as stainless steel and titanium, ceramics, plastics, and other materials can be used. If the plate P and the screw 95 are made of different materials, the difference in thermal expansion coefficients can easily cause the plate P and the screw 95 to loosen, but if they are made of the same material, such loosening can be reduced.

[0182] [0 Display of 2D fluoroscopic image 8] to [8 Reaching perforation end point 41] in the 13th embodiment are basically the same as those in the 15th embodiment. Figures 23(A) to (E) in the 15th embodiment correspond to Figures 21(A) to (E) in the 13th embodiment, respectively, and are basically the same as those.

[0183] However, because a plate P having a hole P01 is disposed on the outside of the object to be drilled (W1), the surgeon M first inserts the drill 34 into the hole P01 of the plate before drilling the object to be drilled W1. Then, the surgeon M moves the drilling tool 3 and adjusts the drilling direction extension line 31 so that it passes through the (provisional) drilling end point 41, as in the 13th embodiment. However, unlike the 13th embodiment, the drilling direction of the drill 34 (drilling direction extension line 31) may deflect in a direction different from the target location due to the hole P01 of the plate for guiding drilling (the drilling direction may be easily deflected) (see FIG. 24(A)). Therefore, while looking at the two-dimensional perspective image 8 of the drilling target W1, the surgeon M moves the jig contact point 430 while searching for a suitable location for the drilling end point 41 in accordance with the swing direction of the drill 34 (drilling direction extension line 31) (within the swing direction range), and determines a suitable drilling end point 41 and drilling direction extension line 31 to perform drilling (see Figures 23(B) to (E)). Alternatively, the location of the plate P itself is moved to align the drilling direction with the target location. It is not unreasonable to use these methods in combination. Note that when the drilling direction extension line 31 passes through the drilling end point 41 or the drilling reaches the drilling end point 41, the display of the drilling end point 41 changes, as in embodiment 13 (see Figures 23(D) and (E)).

[0184] Then, when drilling is completed, the surgeon M screws the screw 95 into the hole W21 opened in the drilling target W1, and fixes the plate P to the drilling target W1 (see FIGS. 24(B) and (C)).

[0185] The drilling device 1500 of the above-mentioned embodiment 15 is similar to the drilling device 1300 of embodiment 13 except that the plate P is positioned outside the object to be drilled (W1), and therefore has the relevant effects among the effects of the drilling device 1300 of embodiment 13. Furthermore, if there is a two-dimensional perspective image 8 of the object W1 to be drilled, by looking at it, it is possible to determine an appropriate drilling end point 41 and drilling direction extension line 31 even in cases where the drilling direction extension line 31 is distorted by the hole P01 in the plate, making it possible to perform even more accurate drilling.

[0186] [Embodiment 16] Fig. 25 is a diagram for explaining a punching device 1600 according to embodiment 16. Fig. 25(A) is a diagram for explaining punching by the punching device 1600, and Fig. 25(B) is a diagram for explaining fixation of the punched target W1. The drilling apparatus 1600 of embodiment 16 is basically similar to the drilling apparatus 1500 of embodiment 15, except that the plate P' is positioned outside the object W1 to be drilled so that the direction of the drill 34 (or screw 95') relative to the hole P01' in the plate P' is constant. Examples of such plates P' include a plate P' in which the diameter (or shape) of the hole P01' is approximately the same as the diameter of the drill 34 (or has a shape that does not cause any play), and a so-called locking plate (a plate in which a pair of male and female threads are formed between the corresponding drill 34 or screw 95').

[0187] When the plate P' is located outside the drilling target W1, the drill 34 (or the screw 95') is placed on the hole P01', and the direction of the drill 34 (or the screw 95') is determined uniquely without deflection (see FIG. 25(A)). Then, the control means 2 displays a virtual three-dimensional image 12 of the uniquely determined drilling direction extension line 31 on the display 1. The surgeon M places the tip of the drilling mark jig 4 on the intersection point of the drilling direction extension line 31 with the outside of the drilling target W1. Then, the control means 2 displays the virtual three-dimensional image 12 on the display 1 with the intersection point as the jig contact point 430. This jig contact point 430 may be the drilling end point. However, similar to the drilling device 1400 of the embodiment 14 (see FIG. 22), a point a certain distance (L1) away from the jig contact point 430 in the opposite direction to the drilling proceeding direction may be the drilling end point 41. Furthermore, when the drilling reaches the drilling end point 41, the display (shape, color, etc.) of the jig contact point 430 (or the drilling end point 41) may be changed (see FIG. 25(A)). Other points are the same as those in the fifteenth embodiment (or the fourteenth embodiment), so a description thereof will be omitted.

[0188] In the drilled object W1, the screw 95' is placed in a hole (drilled hole) formed so as to straddle the fractured part of the drilled object W1. Then, in combination with the plate P', the fractured part of the drilled object W1 is fixed (see FIG. 25(B)).

[0189] The drilling device 1600 of the above-mentioned embodiment 16 is similar to the drilling device 1500 (or 1400) of embodiment 15 (or 14) except that the plate P' is positioned outside the object W1 to be drilled so that the direction of the drill 34 (or screw 95') relative to the hole P01' of the plate P' is constant, and therefore has the corresponding effects of the drilling device 1500 (or 1400) of embodiment 15 (or 14).

[0190] [Embodiment 17] Fig. 26 is a diagram for explaining a fixing mechanism 1700 according to embodiment 17. Fig. 26(A) shows a state before the first and second screws (95A, 95B) are inserted into the hole W21, Fig. 26(B) shows a state during insertion, and Fig. 26(C) shows a state in which the drilling target W1 is fixed, all in a cross-sectional view centered on the hole W21.

[0191] In order to facilitate understanding of the seventeenth embodiment, this cross-sectional view is not an accurate cross-sectional view. For example, the view is omitted for the purpose of omitting what would normally be visible on the other side of the hole W21. A virtual three-dimensional image 12 of the drilling end point 41 and the drilling direction extension line 31 is displayed on the display 1. When the two-dimensional fluoroscopic image acquiring means 81 (e.g., an X-ray imaging device) described in the twelfth to fifteenth embodiments is not used, the cross sections of the first and second screws (95A, 95B) are not displayed on the display 1. However, when the two-dimensional fluoroscopic image acquiring means 81 (e.g., an X-ray imaging device) is used, a display similar to that shown in FIG. 26 is displayed.

[0192] The fixing mechanism 1700 shown in Figure 26 is a fixing mechanism that fixes the object to be drilled (W1) by inserting first and second screws (95A, 95B), each having a head (95A3, 95B3) and a pair of screw portions (95A1, 95B1) that can be fitted together, into the through hole (W21) from different openings (the upper and lower openings in the figure) of the through hole (W21) with the screw portions leading, and sandwiching the object to be drilled (W1) between the heads (95A3-95B3), thereby fitting the pair of screw portions (95A1, 95B1) in the through hole (W21).

[0193] More specifically, a hole W21 (through hole) is formed in the target W1 to be drilled by the drilling device (100, etc.) of any of the embodiments described so far (see FIG. 26(A)).

[0194] Then, the surgeon M uses the virtual three-dimensional image 12 of the drilling end point 41 and the drilling direction extension line 31 displayed on the display 1 as a guide to insert the first and second screws (95A, 95B) into the hole W21 of the through hole (see Figure 26 (B)). These first and second screws (95A, 95B) have threaded portions (95A1, 95B1), heads (95A3, 95B3), and cylindrical portions (95A2, 95B2) provided between them as necessary. The threaded portion 95A1 of the first screw 95A is provided on the outer periphery of a cylindrical portion formed in a cylindrical shape, and the threaded portion 95B1 of the second screw 95B is provided on the inner periphery of a cylindrical portion whose outer periphery enters the hole W21 and whose inner periphery corresponds to the outer periphery of the cylindrical shape. One of the threaded portions (95A1, 95B1) is male threaded and the other is female threaded, and both threads are formed to fit together. The first and second screws (95A, 95B) are inserted from both openings of the hole W21 (through hole) with the threaded portions (95A1, 95B1) at the top. The heads of both screws (95A3, 95B3) are, for example, spherical, aspherical, flat, etc. and do not have a sharp point. On the other hand, the threaded portions (95A1, 95B1) have a sharp point (including thread irregularities) at the top portion.

[0195] Then, the surgeon M inserts the first and second screws (95A, 95B) into the hole W21 (through hole) from the threaded side, and engages the threaded parts (95A1, 95B1) in the hole W21. Meanwhile, the heads (95A3, 95B3) are caught and retained at the mouth of the hole W21, so that the drilling target W1 is sandwiched and fixed between the heads (95A3-95B3) (see FIG. 26(C)).

[0196] The fixing mechanism 1700 has the effects described in

[16] above. In addition, as in the twelfth and thirteenth embodiments, if a two-dimensional perspective image acquiring means 81 (X-ray imaging device) is further used, the surgeon M can view the two-dimensional perspective image 8 and perform fixation even more appropriately.

[0197] [Embodiment 18] 27 is a diagram for explaining a fixing mechanism 1800 according to embodiment 18. The fixing mechanism 1800 according to embodiment 18 is basically the same as the fixing mechanism 1700 according to embodiment 17, but differs in that the target W1 to be drilled is fixed via plates (P1, P2) arranged on the outside thereof.

[0198] Fig. 27(A) shows the state of drilling the hole W21, Fig. 27(B) shows the state of inserting the first and second screws (95A, 95B) into the drilled hole W21, and Fig. 27(C) shows the state of fixing the object W1 to be drilled. Note that the object W1 to be drilled is crushed or pulverized in a wide area (the area between the two dashed lines between the top and bottom of the object W1 to be drilled).

[0199] When drilling a hole W21 in a drilling target W1, the surgeon M first prepares plates (P1, P2) (see FIG. 27(A)). The first and second plates (P1, P2) may both be prepared, or only one (P1) may be prepared. In the embodiment 18 described here, both are prepared.

[0200] These first and second plates (P1, P2) have holes (P11, P21) of a size or shape that the heads (95A2, 95B2) of the first and second screws (95A, 95B) cannot pass through. The surgeon M places the first and second plates (P1, P2) on the outside (both sides) of the drilling target W1 so as to cover the drilling target W1 that has been divided into multiple pieces by fracturing or the like.

[0201] Next, the operator M places the tip of the drill 34 of the drilling tool 3 against the hole P11 of the first plate P1. At this time, the direction of the drill 34 changes due to the hole P11, and the direction of the drilling direction extension line 31 is likely to change (easily swings), so the operator M appropriately moves the position of the first plate P1 or the second plate P2 in accordance with the change in the direction of the drilling direction extension line 31, and changes the location of the drilling end point 41. Or, the operator realigns the direction of the drilling direction extension line 31 with the drilling end point 41. The location of this drilling end point 41 becomes the location of the hole P21 of the second plate P2. While moving the drill 34, the operator determines the direction of the drilling direction extension line 31 and the location of the drilling end point 41 while taking into account the state of the target W1 to be drilled, and places the first and second plates (P1, P2) in appropriate locations according to the determination. According to such an arrangement position, the surgeon M drills the drilling target W1 to form a hole P21 (through hole).

[0202] Next, the surgeon M inserts the first and second screws (95A, 95B) from different openings of the hole W21 (through hole) with the threaded portions (95A1, 95B1) at the head (see FIG. 27(B)).

[0203] Next, the surgeon M fits the threaded portions (95A1, 95B1) of the first and second screws (95A, 95B) into the holes W21 (see FIG. 27(C)). At this time, the heads (95A2, 95B2) are retained at the openings of the holes (P11, P21) of the plates (P1, P2) and cannot advance any further into the hole W21. As a result, the object W1 to be drilled is sandwiched and fixed between the heads (95A3-95B3) of the first and second screws (95A, 95B) via the plates (P1, P2).

[0204] The fixing mechanism 1800 according to the eighteenth embodiment has the effects described in

[17] above. In addition, the fixing mechanism 1800 of embodiment 18 is similar to the fixing mechanism 1700 of embodiment 17 except that the object W1 to be drilled is fixed via plates (P1, P2) arranged on the outside thereof, and therefore has the corresponding effects of the fixing mechanism 1700 of embodiment 17.

[0205] [Embodiment 19] In the 19th embodiment, the surgeon M in the 12th to 18th embodiments is replaced by a carpenter (woodworker), and the object W1 to be drilled is wood (broken wooden board or pillar) instead of a bone (not shown). This method has the same effects as those of the twelfth to eighteenth embodiments, except that the object W1 to be drilled is made of wood instead of bone.

[0206] [Embodiment 20] In the twentieth embodiment, the surgeon M in the twelfth to eighteenth embodiments is replaced by a steel frame worker, and the object W1 to be drilled is a steel frame (broken steel frame) instead of a bone (not shown). This embodiment has the same effects as those of the twelfth to eighteenth embodiments, except that the object W1 to be drilled is a steel frame instead of a bone.

[0207] [Embodiment 21] In the twenty-first embodiment, the surgeon M in the twelve to eighteenth embodiments is replaced by a concrete wall construction worker, and the drilling target W1 is replaced by a concrete wall (broken concrete wall) instead of a bone (not shown). Doing so only differs in that the object W1 to be drilled is a concrete wall instead of a bone, and therefore has the corresponding effects among the effects of the twelfth to eighteenth embodiments.

[0208] Although the present invention has been described based on the above embodiment, the present invention is not limited to the above embodiment. It is possible to implement the present invention in various forms without departing from the spirit of the present invention. For example, the following modifications are also possible.

[0209] (1) In the above-mentioned embodiments 1 to 21, the control means 2 uses the imaging means 7 to refer to the appearance information of the drilling tool 3 or the drilling mark jig 4. However, for example, a shape measurement sensor that uses a laser, infrared rays, ultrasound, etc. may also be used. (2) In the above-described first to eighth embodiments, the imaging means 7 (appearance information acquisition means) is one, but there may be more than one. In the case of more than one, it becomes easier to obtain three-dimensional appearance information. (3) In the above-mentioned embodiments 1 to 21, one imaging means 7 (appearance information acquisition means) is provided, and the imaging means 7 is used to capture images and refer to the appearance information. However, the imaging means 7 may be used for capturing images, and an appearance information acquisition means (another imaging means or a shape measurement sensor) that refers to the appearance information may also be provided. (4) In the above-mentioned embodiments 1 to 21, the imaging means 7 is mounted on the display 1 (head-mounted display 6), but it may also be mounted, for example, on a floor, wall, ceiling, ground, fence, tripod, stand (with or with casters), etc. (5) In the above-described embodiments 1 to 21, the display 1 is a head-mounted display 6 mounted on the head, but instead of a head-mounted display 6, the display 1 may be mounted on, for example, a floor, a wall, a ceiling, the ground, a fence, a tripod, a stand (with or with casters), etc. (6) In the above-described embodiments 1 to 21, the person who holds the drilling tool 3 and the drilling mark jig 4 (such as the surgeon M) is the same person, but they may be held by different persons. (7) In the above-mentioned embodiments 12 to 15, the two-dimensional fluoroscopic image acquiring means 81 is an X-ray imaging device, the detection wave 84 is an X-ray, the detection wave generating unit 82 is an X-ray tube, and the detection unit 83 is an image tube. However, the two-dimensional fluoroscopic image acquiring means 81 may be an ultrasonic imaging device, the detection wave 84 is an ultrasonic wave, the detection wave generating unit 82 is an ultrasonic generator, and the detection unit 83 is a reflected wave detecting means. Alternatively, the two-dimensional fluoroscopic image acquisition means 81 may be a photoultrasonic imaging device, the detection wave 84 may be laser light (pulsed light), the detection wave generating unit 82 may be a laser light generator, and the detection unit 83 may be an ultrasound detector that detects ultrasound emitted from a location irradiated with the laser light. [Explanation of symbols]

[0210] W... object to be drilled, W1... object to be drilled (bone), W2... muscle, W3... third mark, W9... object to be drilled (wood), W91... object to be drilled (wooden board), W92... object to be drilled (wooden board), W10... object to be drilled (steel frame), W11... object to be drilled (concrete wall), W21... hole, M... person to drill (operator), M9... person to drill (carpenter), M10... person to drill (steel frame worker), M11... person to drill (concrete wall worker), 100, 200, 300, 400, 500, 600, 700 (700A, 700B), 800, 900, 1000, 1100, 1200, 1300, 1400, 1500...Drilling device, 1600, 1700...Fixing mechanism, 1...Display, 11...Real image, 12...Virtual three-dimensional image, 2...Control means, 21...CPU, 22...ROM, 23...RAM, 24...I / O, 25...Internal bus, 26...Interface, 3...Drilling tool, 31...Drilling direction extension line, 32...First mark, 33...Tip, 34...Drill (drill bit), 35...Grip, 36...Distance, 37...Attachment mounting part (chuck), 38...Drilling start point, 4...Drilling mark jig, 41...Drilling end point, 42...Second mark, 43...Tip, 44...Needle part , 45...gripping portion, 46...distance, 48...perforation mark jig attachment, 48a...gripping portion, 48b...arm, 48c...gripping portion, 48d...arm, 48e...joint, 48f...spring, 48g...spring, 4A...perforation mark jig, 43a...tip, 44a...needle portion, 45a...screw portion (male screw), 42a...second mark, 49A...perforation mark jig attachment, 49a1...eaves portion, 49a2...screw portion (female screw), 49B...perforation mark jig attachment, 48a3...suction cup, 430...jig contact point, 6...head mounted display, 61...cable, 7...imaging means (camera), 8...two-dimensional perspective image , 81...2D fluoroscopic image acquisition means, 82...detection wave generation unit, 83...detection unit, 84...detection wave, 84a...drilling end point passing notification display unit, 85a...drilling end point distance notification display unit, 85b...drilling end point distance notification display unit, 88a...drilling end point display maintenance mode display unit, 88b...warning display unit, 88c...warning display unit, 95, 95'...screw, 95A...first screw, 95B...second screw, 95A1, 95B1...threaded portion, 95A2, 95B2...head, 95A3, 95B3...cylindrical portion, P, P'...plate, P01, P01'...hole, P1...first plate, P2...second plate, P11,21…hole,

Claims

1. A perforation device for perforating a living body that is a subject to be perforated, The punching device is A display that is a head-mounted display on which a see-through or imaged real image can be viewed and on which an imaging means is attached; A drilling tool that can be held in one hand; A drilling marking jig; an appearance information acquisition means for acquiring appearance information of the drilling tool and the drilling mark jig using the imaging means; a control means for creating a virtual three-dimensional image including a drilling direction extension line and a drilling end point by referring to the appearance information of the drilling tool and the drilling mark jig acquired by the appearance information acquisition means, and for reflecting the virtual three-dimensional image on the real image and displaying it on the display; A drilling device comprising: The control means is capable of controlling the display of the drilling end point in a drilling end point display maintenance mode in which the display is caused to display the end point so that the display position does not change even if the position of the drilling marker jig is changed, and when displaying in this mode, it displays an indication that the drilling end point display maintenance mode is being used, and if the position of the drilling end point has actually changed due to movement of the object to be drilled, but the same position as before the change continues to be displayed, a warning is displayed to the effect that the drilling end point may not be displayed in the correct position.

2. A perforation device for perforating a living body that is a subject to be perforated, The punching device is A display that is a head-mounted display on which a see-through or imaged real image can be viewed and on which an imaging means is attached; A drilling tool that can be held in one hand; A drilling marking tool; an appearance information acquisition means for acquiring appearance information of the drilling tool and the drilling mark jig using the imaging means; a control means for creating a virtual three-dimensional image including a drilling direction extension line and a drilling end point by referring to the appearance information of the drilling tool and the drilling mark jig acquired by the appearance information acquisition means, and for reflecting the virtual three-dimensional image on the real image and displaying it on the display; A drilling device comprising: The control means A drilling device characterized in that an image captured by the imaging means for the drilling tool and the drilling mark jig is compared with a known three-dimensional image previously obtained, a virtual three-dimensional image of the drilling tool and the drilling mark jig is created based on the known three-dimensional image by matching the position, size and shape to the captured image and distorting the shape if it is distorted, and the created virtual three-dimensional image of the drilling tool and the drilling mark jig is displayed on the display by superimposing it on the real image.

3. A perforation device for perforating a living body that is a subject to be perforated, The punching device is A display that is a head-mounted display on which a see-through or imaged real image can be viewed and on which an imaging means is attached; A drilling tool that can be held in one hand; A drilling marking tool; an appearance information acquisition means for acquiring appearance information of the drilling tool and the drilling mark jig using the imaging means; a control means for creating a virtual three-dimensional image including a drilling direction extension line and a drilling end point by referring to the appearance information of the drilling tool and the drilling mark jig acquired by the appearance information acquisition means, and for reflecting the virtual three-dimensional image on the real image and displaying it on the display; A drilling device comprising: The perforation mark jig is The needle has a long and thin screw portion, a needle portion provided at the tip of the screw portion, and a visor portion with a suction cup provided around the screw portion, This drilling device is characterized in that the depth of drilling into the object to be drilled is adjusted by rotating the screw portion, the drilling mark jig is adsorbed to the object to be drilled by the eaves portion with the suction cup, and the position of the drilling mark jig relative to the object to be drilled is maintained.

4. In the punching device according to any one of claims 1 to 3, The drilling device is characterized in that the control means displays the virtual three-dimensional image of the drilling direction extension line and the virtual three-dimensional image of the drilling end point on the display in a display mode different from the actual image.

5. In the punching device according to any one of claims 1 to 3, The punching device further comprises: a passage notification means for notifying that the drilling direction extension line of the drilling tool passes through the drilling end point or does not pass through the drilling end point; A drilling device comprising:

6. In the punching device according to any one of claims 1 to 3, The punching device further comprises: A drilling notification means for notifying that the tip of the drilling part of the drilling tool has approached or reached the drilling end point. A drilling device comprising:

7. In the punching device according to any one of claims 1 to 3, The control means of the drilling device is characterized in that it displays on the display as the drilling end point a jig contact point where the tip of the drilling mark jig comes into contact with the object to be drilled, or a point located a certain distance away from the jig contact point on a straight line connecting the jig contact point and the drilling start point where the drilling tool begins drilling.

Images