Implant surgery support device, method and program

Abstract

To assist an implant surgery without attaching a surgical guide plate or the like into the oral cavity of a patient.SOLUTION: An implant surgery assisting device comprises: a simulation processing part which, using hardware provided with a camera, a computer, and a display device (display), arranges a simulation implant and performs simulation for an embedding position and embedding direction of an implant; an AR processing part which displays, on a display device in real time, an implant 3D object indicative of the position and direction of the embedding of the implant; and a cooperative processing part which makes a display position of the implant 3D object displayed superimposed on an image of an oral cavity or an image of an oral cavity model being captured with the camera coincide with the arrangement position of the simulation implant arranged on the simulation image and when an operation of moving the simulation implant is performed, makes the implant 3D object also move in connection with the operation.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] An embodiment of the present invention relates to an implant surgery assistance device, method, and program. [Background technology]

[0002] Conventionally, in dental implant surgery, a surgical guide plate has been used that is fitted into the patient's mouth to assist in the accurate drilling position, depth, and angle when embedding the implant.

[0003] Then, drilling is performed into the jawbone according to the guide grooves of the surgical guide plate. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2014-236957 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the conventional technology has the following problems. (1) The surgical guide plate covered the surgical field, making it difficult for the surgeon to see directly the surgical field, which made the surgeon feel uneasy.

[0006] (2) The surgical guide plate could block the flow of cooling water during surgery, raising the risk of bone burns. (3) In cases of distal end fractures, the surgical guide plate was unstable and prone to displacement due to unilateral support.

[0007] (4) In cases involving flap surgery, a wide flap (gingival separation) is required to attach the surgical guide plate, and there is a risk that the bone surface exposed by the flap will not fit well with the inner surface of the surgical guide plate.

[0008] (5) In the molar region, there was a risk that the method would not be applicable to patients who could not open their mouths wide enough to insert a drill from above the surgical guide plate. (6) Every time the drill was changed from a thin drill to a thick drill, the guide ring had to be replaced with one having a different diameter.

[0009] The present invention has been made in consideration of the above, and aims to provide an implant surgery assistance device, method, and program that can assist implant surgery without attaching a surgical guide plate or the like in the patient's oral cavity. [Means for solving the problem]

[0010] The implant surgery support device of the embodiment includes a simulation processing unit that uses hardware including a camera, a computer, and a display device to place a shim implant, which is a simulation implant, on a simulation image such as a panoramic image, a horizontal cross-sectional image, or an oblique cross-sectional image of a patient who is a target of implant surgery, to simulate the implant insertion position and the implant insertion direction, and a simulation processing unit that performs a simulation of a part or The implant surgery target mouth of the patient An AR processing unit defines and registers a part of the intraoral model as an AR marker, detects and tracks the AR marker from within an image captured by a camera, and displays a 3D implant object indicating the position and direction of implant insertion on a display device in real time by superimposing the 3D implant object on the image of the oral cavity captured by the camera or the image of the intraoral model. MuThe image processing unit is provided with a linking processing section that matches the position of the shim implant placed on the simulation image and, when an operation to move the shim implant is performed, moves the implant 3D object in conjunction with the operation. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic block diagram of an implant surgery support system according to an embodiment. [Diagram 2] FIG. 2 is a flowchart showing a procedure for an implant insertion simulation according to the embodiment. [Diagram 3] FIG. 3 is an explanatory diagram of an example of a display screen of the display. [Figure 4] FIG. 4 is a flowchart of a first processing procedure of the simulation AR preparation processing. [Figure 5A] FIG. 5A is an explanatory diagram of a first process of the simulation AR preparation process. [Figure 5B] FIG. 5B is an explanatory diagram of the second process of the simulation AR preparation process. [Figure 6] FIG. 6 is a flowchart of a second processing procedure of the simulation AR preparation processing. [Figure 7] FIG. 7 is a flowchart showing the procedure of an implant insertion surgery according to an embodiment. [Figure 8] FIG. 8 is a flowchart of the processing procedure of the AR preparation processing for implantation surgery. [Figure 9] FIG. 9 is an explanatory diagram of the 3D contra model data CON and the drill shaft 3D object DRL. [Figure 10] FIG. 10 is an explanatory diagram showing the state inside the oral cavity during the implant insertion surgery. [Figure 11] FIG. 11 is an explanatory diagram showing a drilling situation using a practice model. [Figure 12] FIG. 12 is an illustration of an implant 3D object with a guide ring. [Figure 13]FIG. 13 is an explanatory diagram showing a state in which an implant 3D object with a guide ring is used. [Figure 14] FIG. 14 is an explanatory diagram showing a state where the drill is aligned without using the drill shaft 3D object DRL. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Next, a preferred embodiment will be described with reference to the drawings.

[0013] FIG. 1 is a schematic block diagram of an implant surgery support system according to an embodiment. The hardware of the implant surgery support system 10 includes a computer 11 that controls the entire implant surgery support system 10, a camera 12 that takes images and outputs the video data, displays 13A and 13B that are connected to the computer 11 as display devices for displaying various information, and an input device 14 such as a keyboard and a mouse.

[0014] In addition, in the implant surgery support system 10, simulation software SS and AR application software AA are installed in the computer 11 as software, and function as a simulation processing unit, an AR processing unit, and a cooperation processing unit, respectively.

[0015] In the above configuration, various simulation images of the simulation software SS, which will be described later, are displayed on (a predetermined display area of) the display 13A. Also, the display 13B (in a predetermined display area) is set to switch between displaying a camera image or an AR space (image) of the AR application software AA. However, by default, the camera image is displayed.

[0016] Here, the simulation software SS and the AR application software AA will be explained.

[0017] First, the simulation software SS will be described. The simulation software SS reads DICOM (Digital Imaging and Communications in Medicine) data obtained by CT scanning of the patient, and displays various simulation images on the display 13A, such as a panoramic image GP, a cross-sectional image at any horizontal plane (horizontal cross-sectional image) GS, and a cross-sectional image at any inclined plane (inclined cross-sectional image) GL, and places shim implants SI (shown as cylinders in Figure 3), which are simulation implants that match the diameter and length of the implants to be actually inserted, on the screen (simulating the implant insertion surgery on the screen) and creates an optimal implant insertion plan while considering in which position and from which direction the number of implants should be inserted.

[0018] Next, the AR application software AA will be described. First, conventional AR (Augmented Reality) technology will be described. In conventional AR technology, preparations are first made by defining and registering an AR marker (e.g., a shape), and then creating a three-dimensional digital image called a 3D object and placing it in the AR space (the space where the AR marker is defined and registered).

[0019] Next, for example, an AR marker printed on paper or a board is attached to a suitable location in the real world. Then, the camera takes a picture, detects and tracks the AR marker in the image being taken by the camera, and recognizes the position, angle, and size of the image of the AR marker in the image.The system then compares the 3D object placed in the AR space with the registered AR marker and converts its coordinates to display the 3D object superimposed on the image of the AR marker in the camera image in real time.

[0020] This makes it appear as if 3D objects are attached to the AR marker in live-action footage of the real world.

[0021] In recent years, advances in AR technology have led to the development of technologies that can recognize objects or areas in the real world as AR markers without placing (images of) the AR markers in the real world. First, the extent of the AR marker is defined and registered, for example, on a 3D model of a real-world object or area.

[0022] The system detects and tracks an image that can be regarded as having the same shape as the AR marker in the image captured by the camera, and recognizes the position, angle, and size of the image of the AR marker in the image.Then, by comparing it with the registered AR marker and converting the coordinates of the 3D object placed in the AR space, the system displays the 3D object superimposed on the image of the AR marker in the camera image in real time.

[0023] This allows 3D objects to be displayed in live-action footage of the real world as if they were attached to real-world objects or areas.

[0024] For example, by registering a part of the shape of the plaster model PLM as an AR marker, an image that can be regarded as having the same shape as the AR marker is detected and tracked in the image of the plaster model PLM captured by a camera, and the position, angle, and size of the image of the AR marker in the image are recognized.Then, by comparing with the registered AR marker and performing coordinate conversion on the implant 3D object SIO (3D object of an implant shape) placed in the AR space, the implant 3D object SIO is displayed in real time superimposed on the image of the plaster model PLM captured by the camera.

[0025] As a result, the implant 3D object SIO is displayed as if it is attached to the plaster model PLM in the actual image captured by a camera.

[0026] Next, an example of the type of 3D model used in the implant surgery support system 10 will be described. Examples of the 3D models used in the implant surgery support system 10 include a 3D plaster model, a 3D bone model, a 3D intraoral model, and a 3D contrast model CON.

[0027] The 3D plaster model 3DM0 is a 3D model obtained by scanning a plaster model PLM made by taking an impression of the oral cavity with a 3D scanner and then surface rendering the resulting STL (Standard Triangulated Language) data.

[0028] The 3D bone model is a 3D model obtained by taking a CT scan of the patient and performing volume rendering of the resulting DICOM data.

[0029] The 3D intraoral model is a 3D model obtained by taking an optical impression of the patient's oral cavity using an optical scanner and surface rendering the resulting STL data.

[0030] The 3D contra model CON is a 3D model obtained by scanning the dental contra DCN used in surgery when drilling implant insertion holes with a 3D scanner and surface rendering the resulting STL data.

[0031] Next, an example of the type of 3D object used in the implant surgery assistance system 10 will be described. Examples of 3D objects used in the implant surgery support system include an implant 3D object SIO, a plate-shaped 3D object, a drill shaft 3D object DRL, an implant 3D object with a superstructure, a mandibular canal 3D object, a maxillary sinus floor 3D object, and an implant 3D object with a guide ring.

[0032] The implant 3D object SIO is a 3D object having an implant shape (for example, cylindrical or columnar) and a central axis DAX indicating the implantation position and direction. The plate-like 3D object is a plate-shaped 3D object that is used to check the oblique cross-section position in the simulation.

[0033] The drill shaft 3D object DRL is a 3D object having the shaft shape of a drill used in surgery, and is displayed superimposed on the drill attached to the dental contra DCN. The 3D implant object with superstructure is a 3D object having the shape of an implant with a superstructure, and is used when conducting prosthetic studies in a simulation.

[0034] The mandibular canal 3D object is a 3D object that has a shape that resembles the mandibular canal, and is used to confirm its position during surgery. The maxillary sinus floor 3D object is a 3D object that has a shape that resembles the floor of the maxillary sinus, and is used to confirm its position during surgery. An implant 3D object with guide rings is an implant 3D object in which two upper and lower guide rings UDL and LDL, separated by the drill length DL, are integrally arranged along a central axis DAX, as shown in Figure 12, and it is possible to position the drill without using a drill axis 3D object.

[0035] Next, the operation of the embodiment will be described. FIG. 2 is a flowchart showing a procedure for an implant insertion simulation according to the embodiment. First, the operator starts the simulation software SS and causes it to read DICOM (Digital Imaging and Communications in Medicine) data obtained by CT scanning of a patient from a predetermined folder in an external storage device (step S11).

[0036] FIG. 3 is an explanatory diagram of an example of a display screen on two displays. When the DICOM data is read, a panoramic image GP, a cross-sectional image at an arbitrary horizontal plane (horizontal plane cross-sectional image) GS, and a cross-sectional image at an arbitrary inclined plane (inclined plane cross-sectional image) GL are displayed on the display 13A as shown in FIG. 3 (step S12).

[0037] In this state, the operator (implant surgery planner) places shim implants SI (shown as cylinders in Figure 3) that match the diameter and length of the implant to be actually inserted on the panoramic image GP, horizontal cross-sectional image GS, or oblique cross-sectional image GL of the display 13A (step S13), and simulates the implant insertion surgery on the screen to consider in which position and from which direction how many implants should be inserted.

[0038] In addition, the shim implant SI placed on the panoramic image GP, the horizontal plane cross-sectional image GS, or the oblique cross-sectional image GL can be freely moved by dragging it with the mouse of the input device 14, and is displayed in conjunction with all of the images, the panoramic image GP, the horizontal plane cross-sectional image GS, and the oblique cross-sectional image GL.

[0039] Furthermore, when the AR application software AA is started, an implant 3D object SIO indicating the position and orientation of the shim implant SI placed on the panoramic image GP, horizontal cross-sectional image GS, and oblique cross-sectional image GL of the display 13A is displayed on the display 13B, superimposed on the image of the plaster model PLM photographed by the camera 12. Furthermore, the implant 3D object SIO also moves in conjunction with the movement of the shim implant SI.

[0040] Here, the AR engineer performs a simulation AR preparation process in order to link the AR application software AA with the simulation software SS (step S14).

[0041] FIG. 4 is a flowchart of a first processing procedure of the simulation AR preparation processing. First, the AR technician switches the display on the display 13B (from the camera image) to the AR space image, then has the AR application software AA read the (STL) data of the 3D plaster model 3DM0 (step S31), and displays the 3D plaster model 3DM0 in the AR space (step S32).

[0042] FIG. 5A is an explanatory diagram of a first process of the simulation AR preparation process. Next, the AR technician refers to the various simulation images (panoramic image GP, cross-sectional image GS, and cross-sectional image GL) on display 13A, and selects and marks three characteristic points on the screen, such as the cusp (apex) and cavity bottom (valley bottom) of the tooth (three points A, B, and C in the example of Figure 5A), using a mouse or the like (step S33).

[0043] When the AR engineer selects the three points A, B, and C, the simulation software SS displays the position coordinates of the three points A, B, and C in the space of the simulation software SS, which the AR engineer reads (step S34).

[0044] The AR technician places marks in the AR space displayed on the display 13B at positions that are the same as the position coordinates of the selected three points A, B, and C (step S35).

[0045] Next, the AR technician marks three points a, b, and c that correspond to three points A, B, and C on the 3D plaster model 3DM0 in the AR space (step S36).

[0046] At this point in time, since the coordinate systems (coordinate axes and scale) of the space of the simulation software SS and the AR space are different, the distances among the three points A, B, and C are different from the distances among the three points a, b, and c.

[0047] Therefore, the AR technician enlarges or reduces the size of the 3D plaster model 3DM0 displayed in the AR space to match the distances between the three points (or between two points) (step S37).

[0048] This results in a 3D plaster model 3DM1 in the AR space whose scale (distance between three points) matches that of the space of the simulation software SS. Therefore, in order to align the coordinate axes, the AR technician manually moves the 3D plaster model 3DM1 using a mouse or the like, and aligns the three points a, b, and c with the three points A, B, and C, respectively, as shown in FIG. 5A (step S38). This makes the coordinate system of the AR space coincident with the coordinate system of the space of the simulation software SS.

[0049] In this state, the AR technician uses the data import function of the AR application software AA to read in the coordinates of the shim implant SI arranged in the space of the simulation software SS from the simulation software SS (step S39).

[0050] Here, the AR technician uses the functions of the AR application software AA to generate an implant 3D object SIO corresponding to the shim implant SI in the AR space. Furthermore, since the coordinate systems of the space of the simulation software SS and the AR space match in this state, the generated implant 3D object SIO is placed at the same coordinates as the coordinates of the loaded sim implant SI (step S40). In this state, the AR technician defines the range of the AR marker on the 3D plaster model 3DM1 and registers it as an AR marker (step S41).

[0051] As a result, it becomes possible to display the implant 3D object SIO at the implantation position of the shim implant SI placed in the simulation software SS. Furthermore, when the shim implant SI is moved in the simulation software SS, the implant 3D object SIO moves in conjunction with it.

[0052] This result is stored as a project in the project storage folder (directory) 11B of the AR application software AA, and the first process of the simulation AR preparation process is terminated (step S42).

[0053] The first processing procedure of the simulation AR preparation processing for linking the AR application software AA with the simulation software SS has been described above, but linking is also possible through the following processing procedure.

[0054] FIG. 6 is a flowchart of a second processing procedure of the simulation AR preparation processing. First, the AR technician switches the display on the display 13B (from the camera image) to the AR space, then has the AR application software AA read the (STL) data of the 3D plaster model 3DM0 (step S51), and displays the 3D plaster model 3DM0 in the AR space (step S52).

[0055] FIG. 5B is an explanatory diagram of the second process of the simulation AR preparation process. Next, the AR technician defines the range of the AR marker on the loaded 3D plaster model 3DM0 and registers it as an AR marker (step S53). Next, the AR technician refers to the various simulation images (panoramic image GP, cross-sectional image GS, and cross-sectional image GL) on display 13A, and selects and marks three characteristic points (A, B, and C in the example of Figure 5B) such as the cusp (apex) and cavity bottom (valley bottom) of the tooth on the screen using a mouse or the like (step S54).

[0056] When the AR engineer selects the three points A, B, and C, the simulation software SS displays the position coordinates of the selected three points A, B, and C in the space of the simulation software SS, which the AR engineer reads (step S55).

[0057] Next, the AR technician marks three points a, b, and c that correspond to three points A, B, and C on the 3D plaster model 3DM0 in the AR space displayed on the display 13B (step S56). The AR application software AA then displays the position coordinates of the three points a, b, and c in the AR space, which the AR technician reads (step S57).

[0058] Next, the AR technician uses the calculation function of the computer 11 to determine a coordinate conversion formula for converting the position coordinates of the three points A, B, and C into the position coordinates of the three points a, b, and c (step S58).

[0059] In this state, the AR technician uses the data import function of the AR application software AA to read in the coordinates of the shim implant SI arranged in the space of the simulation software SS from the simulation software SS (step S59). The AR technician converts the coordinates of the read shim implant SI using the coordinate conversion formula obtained in step S58 (step S60).

[0060] Here, the AR technician uses the functions of the AR application software AA to generate an implant 3D object SIO corresponding to the shim implant SI in the AR space, and then places the generated implant 3D object SIO at the coordinates obtained by converting the coordinates of the loaded shim implant SI in step S60 (step S61).

[0061] As a result, it becomes possible to display the implant 3D object SIO at the implantation position of the shim implant SI placed in the simulation software SS. Furthermore, when the shim implant SI is moved in the simulation software SS, the implant 3D object SIO moves in conjunction with it. The result is stored as a project in the project storage folder (directory) 11B of the AR application software AA (step S62), and the second process of the simulation AR preparation process is terminated.

[0062] Next, the display on display 13B is switched (from the AR space) to a camera image, the plaster model PLM is photographed by camera 12, and the image of the plaster model PLM photographed by camera 12 is displayed on display 13B. As a result, on the display 13B, as shown in FIG. 3, the implant 3D objects SIO of all implants to be inserted are superimposed on the image of the plaster model PLM and displayed (step S15).

[0063] Now that everything is ready for the implant insertion simulation, we can begin the simulation. First, the operator (implant surgery planner) performs anatomical examination while viewing various simulation images (panoramic image GP, horizontal cross-sectional image GS, and oblique cross-sectional image GL) displayed on the display 13A (step S16). Then, if necessary, the placement position of the shim implant SI is corrected (step S17).

[0064] Next, the implant 3D object SIO displayed on the display 13B is examined in relation to the opposing teeth, and the like, while the dental technician or the like holds the plaster model in his or her hands and performs prosthetic examination (step S18).

[0065] Then, it is determined whether there is any problem with the placement of the shim implant SI (step S19). If it is determined in step S19 that there is a problem with the placement of the shim implant SI (step S19; No), the shim implant SI is moved and corrected (step S20).

[0066] In this state, the process returns to step S16, and an anatomical review is performed again while viewing the various simulation images (panoramic image GP, horizontal cross-sectional image GS, and oblique cross-sectional image GL) displayed on the display 13A, and if necessary, the shim implant SI is moved and corrected again (step S17), and then a prosthetic review is performed again (step S18). The above-mentioned process is repeated until the determination in step S19 indicates that there is no problem with the placement of the shim implant SI.

[0067] If it is determined in step S19 that there is no problem with the placement of the shim implant (step S19; Yes), the implant insertion position is determined and saved as a project in the project storage folder (directory) 11A of the simulation software SS (step S21), and the implant insertion simulation is terminated.

[0068] Next, the procedure of the implant insertion surgery in this embodiment will be described. FIG. 7 is a flowchart showing the procedure of an implant insertion surgery according to an embodiment. First, the simulation software SS is caused to read the implant embedding position information determined by the simulation and stored as a project in the project storage folder (directory) 11A (step S71).

[0069] Next, an AR preparation process for implantation surgery is performed (step S72). FIG. 8 is a flowchart of the processing procedure of the AR preparation processing for implantation surgery. FIG. 9 is an explanatory diagram of the 3D contra model CON and the drill shaft 3D object DRL.

[0070] First, the display on display 13B is switched (from the camera image) to the AR space, and then the dental contra DCN to be used in the surgery when drilling the implant insertion hole is scanned with a 3D scanner, and the (STL) data of the obtained 3D contra model is loaded into the AR application software AA (step S91), and the 3D contra model CON is displayed in the AR space (step S92). Next, a part or the whole of the 3D contra model CON is defined and registered as a contra AR marker (step S93).

[0071] Furthermore, a drill shaft 3D object DRL having the shaft shape of the drill used in surgery is generated, and the drill shaft 3D object DRL is accurately aligned in the AR space so that the drill shaft 3D object DRL is displayed perfectly superimposed on the drill attached to the dental contra DCN in the camera image (step S94). The result is saved as a project in the project storage folder (directory) 11B of the AR application software AA (step S95), and the AR preparation process for implantation surgery is completed.

[0072] Now that all preparations for the implant insertion surgery are complete, the implant insertion surgery can begin. Note that the AR preparation process for the implant insertion surgery only needs to be performed prior to the implant insertion surgery, and does not need to be performed immediately before. Furthermore, by using an interface incorporating a program that automates the above-mentioned AR preparation processing for simulation and AR preparation processing for implantation surgery, these processes can be performed by anyone, even if they are not an AR technician.

[0073] FIG. 10 is an explanatory diagram showing the state inside the oral cavity during the implant insertion surgery. During an implant insertion operation, the display on display 13B is switched (from the AR space) to a camera image, the inside of the patient's mouth is photographed by camera 12, and the image is displayed on display 13B. As a result, as shown in FIG. 10, the implant 3D object SIO is displayed superimposed on the image of the intraoral cavity at the implant insertion position determined by the simulation (step S73).

[0074] In this state, the surgeon first incises (flaps) the gums or punches the gums, and provides a starting point with a round bar or the like (step S74).

[0075] Here, when a pilot (first) drill is attached to the dental contra DCN and the dental contra DCN is brought into the field of view of the camera 12, the drill axis 3D object DRL is displayed superimposed on the pilot (first) drill attached to the dental contra DCN, as shown in FIG. 10 (step S75).

[0076] Next, the surgeon aligns the tip of the pilot (first) drill with the starting point, and positions the pilot (first) drill by overlapping the drill axis 3D object DRL with the central axis DAX of the implant 3D object SIO (step S76).

[0077] FIG. 11 is an explanatory diagram showing a drilling situation using a practice model. Then, the surgeon performs drilling to a predetermined depth while maintaining the state in which the drill axis 3D object DRL is superimposed on the central axis DAX of the implant 3D object SIO as shown in FIG. 11 (step S77).

[0078] Then, the surgeon judges whether the diameter of the embedding hole formed by drilling is large enough and necessary for the diameter of the implant to be embedded (step S78). If it is determined in step S78 that the diameter of the hole drilled by the drill is not sufficiently large (step S78; No), the drill is replaced with a thicker one (step S79), and the process returns to step S76, where the tip of the drill is aligned with the embedding hole created by drilling, and the drill axis 3D object DRL is superimposed on the central axis DAX of the implant 3D object SIO to position the drill. Next, drilling is performed to a predetermined depth while maintaining the state in which the drill axis 3D object DRL is superimposed on the central axis DAX of the implant 3D object SIO (step 77). The above-mentioned process is repeated until it is determined in step S78 that the diameter of the embedding hole is sufficiently large for the diameter of the implant to be embedded.

[0079] If it is determined in step S78 that the diameter of the embedding hole is large enough for embedding the implant, the surgeon embeds the implant (step S80). Next, the surgeon determines whether or not the placement of all the implants has been completed (step S81).

[0080] In the determination at step S81, if embedding of all the implants has not yet been completed (step S81; No), the process returns to step S74 again, and the above-mentioned processes are repeated. If it is determined in step S81 that all implants have been inserted (step S81; Yes), a healing cap is attached and the gums are sutured, and the implant insertion surgery is completed (step S82).

[0081] As described above, according to this embodiment, nothing that would interfere with the implant insertion surgery is inserted into the oral cavity, and all of the disadvantages associated with using a surgical guide plate are eliminated.

[0082] That is, (1) Because the surgical guide plate does not cover the surgical field, the surgeon can directly view the surgical field and perform surgery with peace of mind. (2) The surgical guide plate does not block the supply of cooling water during surgery, reducing the risk of bone burns. (3) Even in distal end cases, there is no need for a surgical guide plate, so there is no risk of misalignment. (4) Even in cases involving flap surgery, a surgical guide plate is not attached, so there is no need for a large flap (gingival separation), and there is no need to worry about poor fit between the surgical guide plate and the bone surface. (5) Even surgery on molars can be performed if the mouth can be opened wide enough to insert a drill.

[0083] (6) Every time a drill was changed from a thin drill to a thick drill, it was necessary to change the guide ring to one with a different diameter. However, since there is no guide ring, this is no longer necessary. (7) Furthermore, since there is no need to produce a surgical guide plate, the cost and time required for producing a surgical guide plate is saved.

[0084] (8) In addition, when formulating a treatment plan, it is now possible to simultaneously consider the implant placement position from both anatomical and prosthetic perspectives, enabling a more optimal implant placement simulation to be performed for each patient.

[0085] In the above description, the implant 3D object SIO has a cylindrical shape, but it is also possible to display the implant 3D object SIO in a shape that matches the shape of an actual implant.

[0086] The implant surgery support device (implant surgery plan creation support device) of this embodiment is equipped with a control device such as a CPU, a storage device such as a ROM (Read Only Memory) or RAM, an external storage device such as a HDD or a CD drive device, a display device such as a display device, and input devices such as a keyboard and a mouse, and has a hardware configuration that utilizes a normal computer.

[0087] The software (program) executed by the implant surgery support device of this embodiment is provided as a file in an installable or executable format recorded on a computer-readable disk-shaped recording medium such as a DVD (Digital Versatile Disk), a USB memory, a semiconductor storage device such as an SSD (Solid State Disk), or the like.

[0088] The program executed by the implant surgery assistance device of this embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The program executed by the implant surgery assistance device of this embodiment may be provided or distributed via a network such as the Internet. Furthermore, the program for the implant surgery support device of this embodiment may be configured to be provided by being pre-installed in a ROM or the like.

[0089] Although some embodiments of the present invention have been described, these embodiments are merely illustrative and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are within the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims.

[0090] In the oral cavity or on the plaster model PLM, the parts of the 3D object that overlap with the teeth or alveolar bone (mucosal surface) and are in shadow, or the parts inside the alveolar bone, may be hidden or displayed semi-transparently. This configuration makes it possible to give the 3D object a more three-dimensional visual appearance. Furthermore, when displaying a drill axis 3D object, the color of the intrabone portion of the drill axis 3D object may be changed. This configuration makes it possible to visually recognize the drill depth.

[0091] Also, a mandibular canal / maxillary sinus floor 3D object simulating the mandibular canal or maxillary sinus floor may be displayed within the camera image on display 13B, superimposed on an image of the intraoral region or a model of the intraoral region. According to this configuration, by displaying the mandibular canal / maxillary sinus floor on the camera image of the display 13B, it becomes possible to visually recognize the distance between the drill and the intrabone portion.

[0092] Furthermore, in this case, the distance between the tip of the drill shaft 3D object and the mandibular canal / maxillary sinus floor 3D object may be displayed. According to this configuration, the distance between the drill and the inside of the bone can be grasped more accurately.

[0093] When the central axis DAX of the implant 3D object SIO and the drill axis 3D object DRL coincide in an image (of the inside of the oral cavity or an intraoral model) captured by the camera 12, this fact may be notified by a display color or sound. This allows the surgeon to easily position the drill.

[0094] FIG. 12 is an illustration of an implant 3D object with a guide ring. FIG. 13 is an explanatory diagram showing a state in which an implant 3D object with a guide ring is used. Even if a drill shaft 3D object is not used, the drill can be positioned using the implant with guide ring 3D object, as shown in Figure 12. In this case, the method for using the 3D object of the implant with guide ring is to first align the lower guide ring LGL with the bone surface (surface), and then position the drill by aligning the tip of the drill with the center (starting point) of the lower guide ring LGL and the rear end of the drill with the center of the upper guide ring UGL, as shown in Figure 13.

[0095] The AR application software AA has a function for registering a part of the 3D contra model CON with a drill attached as a contra AR marker, and recognizing in real time the front-to-back relationship between the part of the 3D contra model CON with the drill attached and the central axis DAX of the implant 3D object SIO when tracking the contra AR marker. By using this function, it is possible to partially hide in real time the central axis DAX of the implant 3D object SIO that overlaps (is behind) the part of the 3D contra model CON with the drill attached, making it possible to position the drill without displaying the drill axis 3D object DRL as shown in FIG. 14. In this case, the method of positioning the drill is to align the tip of the drill with the starting point as shown in FIG. 14, and align the rear end of the drill with the position displayed as if the central axis DAX is extending from the center of the drill.

[0096] By adding a LiDAR (Light Imaging Detection and Ranging) scanner to the hardware, it becomes possible to recognize the front-to-back relationship between the dental contra DCN with the drill attached and the central axis DAX of the implant 3D object SIO in real time. By using this scanner to partially hide the central axis DAX of the implant 3D object SIO that overlaps (is behind) the dental contra DCN with the drill attached in real time, it becomes possible to position the drill without displaying the drill axis 3D object DRL as shown in Figure 14.

[0097] In this case, the method of positioning the drill is to align the tip of the drill with the starting point as shown in FIG. 14, and align the rear end of the drill with the position displayed as if the central axis DAX is extending from the center of the drill.

[0098] In the above description, the system is configured to include two displays 13A and 13B, but it is also possible to configure the system to include one display and display the images displayed on the two displays 13A and 13B in multiple display areas of the single display. It is also possible to configure the system to include three or more displays and display a simulation image, a camera image, and an AR space on each of these displays. Also, instead of the display, other display devices such as a projector may be used.

[0099] In the above explanation, the local computer 11 realizes the functions of the simulation processing unit, AR processing unit, and collaboration processing unit, but it is also possible to have these functions (by installing them) in a cloud server computer, upload images captured by a camera to the cloud server computer, and display the processed images on the display of the local computer. [Explanation of symbols]

[0100] 10 Implant surgery support system 11. Computers 11A Simulation software SS project storage folder (directory) 11B AR application software AA project storage folder (directory) 12 Camera 13A Display (display device) 13B Display (display device) 14 Input Devices AA AR application software CON 3D Contra Model DAX implant 3D object SIO central axis DCN Dental Contra DL Drill length DRL Drill axis 3D object GP Panorama Image GS horizontal cross-sectional image GL slope cross section image LGL Lower Guide Ring PLM plaster model SI Shim Implant (Simulation Implant) SIO implant 3D object SS Simulation software Guide ring on UGL 3DM0 3D plaster model 3DM1 3D plaster model (after scale matching)

Claims

1. a simulation processing unit that uses hardware having a camera, a computer, and a display device to place a shim implant, which is a simulation implant, on a simulation image such as a panoramic image, a horizontal cross-sectional image, or an oblique cross-sectional image of a patient who is a target of an implant surgery, and performs a simulation of the implantation position and implantation direction; an AR processing unit that defines and registers a part of the oral cavity of the patient that is the implant surgery target or a part of an intraoral model of the patient that is the implant surgery target as an AR marker, detects and tracks the AR marker from within an image captured by the camera, and displays an implant 3D object indicating a position and a direction of implant insertion on the display device in real time by superimposing the AR marker on the image of the oral cavity or the image of the intraoral model captured by the camera; a linking processing unit that matches a display position of the implant 3D object, which is displayed by being superimposed on the image of the oral cavity captured by the camera or the image of the intraoral model, with a placement position of the shim implant placed on the simulation image, and when an operation to move the shim implant is performed, causes the implant 3D object to move in conjunction with the operation; An implant surgery support device equipped with

2. The cooperation processing unit enlarges / reduces, rotates, and moves a 3D model in the AR space, and when the position coordinates of the three points in the AR space that are the same as the position coordinates of the three points that are pre-specified on the simulation image and the position coordinates corresponding to the three points on the 3D model loaded into the AR space match, the cooperation processing unit treats the coordinates of the shim implant as they are as the coordinates of the implant 3D object in the AR space. The implant surgery support device according to claim 1.

3. 2. The implant surgery support device according to claim 1, wherein the link processing unit obtains a coordinate transformation formula for transforming position coordinates of three different points that are specified in advance on the simulation image into position coordinates corresponding to the three points on the 3D model loaded into the AR space, and treats coordinates obtained by substituting the coordinates of the shim implant into the coordinate transformation formula as coordinates of the implant 3D object in the AR space.

4. The implant surgery support device according to claim 2 or 3, wherein the 3D model is any one of a 3D plaster model, a 3D bone model, and a 3D intraoral model.

5. The implant surgery assistance device according to claim 1 , wherein the implant 3D object includes a central axis of the implant 3D object.

6. The implant 3D object includes two guide rings attached to the central axis and spaced apart by a drill length in a direction along the central axis. The implant surgery support device according to claim 5.

7. The AR processing unit scans the dental contra with a 3D scanner, and defines and registers a part of the contra 3D model obtained by surface rendering the obtained STL data as a contra AR marker, and displays a drill shank 3D object having a drill shank shape superimposed on the drill attached to the dental contra in the image captured by the camera. The implant surgery support device according to any one of claims 1 to 6.

8. When a central axis of the implant 3D object and the drill axis 3D object coincide with each other in the image of the intraoral cavity or the image of the intraoral model, the AR processing unit notifies the user of this by a display color or a sound. The implant surgery support device according to claim 7.

9. The AR processing unit displays a 3D object of the mandibular canal / maxillary sinus floor simulating the mandibular canal or the maxillary sinus floor by superimposing it on the image of the oral cavity or the image of the intraoral model. The implant surgery support device according to any one of claims 1 to 8.

10. The AR processing unit, in the image of the oral cavity or the image of the intraoral model, hides or displays in a semi-transparent manner a portion of the implant 3D object and the drill shank 3D object that overlaps with a tooth, a mucosa, or an alveolar bone present in the oral cavity region or the model of the oral cavity and is in shadow, or a portion that is inside the alveolar bone. The implant surgery support device according to any one of claims 1 to 8.

11. the AR processing unit partially hides in real time a portion of the image of the intraoral cavity or the image of the intraoral model that overlaps with a portion of the contra 3D model with a drill attached, the portion being defined and registered as the contra AR marker with a drill attached, with respect to a central axis of the implant 3D object; An implant surgery support device according to claim 7.

12. The implant surgery support device according to any one of claims 1 to 8, further comprising a LiDAR (Light Imaging Detection and Ranging) scanner added to the hardware, and partially hiding in real time a portion of the central axis of the implant 3D object that overlaps with a dental contra with a drill attached.

13. A process of simulating the implant placement position and the implant placement direction by placing a shim implant, which is a simulation implant, on a simulation image such as a panoramic image, a horizontal cross-sectional image, an oblique cross-sectional image, etc. of a patient who is a target of an implant surgery, using hardware equipped with a camera, a computer, and a display device; a step of defining and registering a part of the oral cavity of the patient that is the implant surgery target or a part of an intraoral model of the patient that is the implant surgery target as an AR marker, detecting and tracking the AR marker from within an image captured by the camera, and displaying an implant 3D object indicating the position and orientation of implant insertion on the display device in real time by superimposing the AR marker on the image of the oral cavity or the image of the intraoral model captured by the camera; a step of matching a display position of the implant 3D object (indicating the position and direction of implant insertion) that is displayed by being superimposed on the image of the oral cavity captured by the camera or the image of the oral cavity model with a placement position of the shim implant placed on the simulation image, and when an operation to move the shim implant is performed, making the implant 3D object also move in conjunction with the operation; The implant surgery support method includes:

14. A program for controlling an implant surgery support device that uses hardware having a camera, a computer, and a display device to support implant surgery by utilizing simulation images such as panoramic images, horizontal cross-sectional images, and oblique cross-sectional images of a patient who is a target of implant surgery, by using the computer, The computer, a means for arranging a shim implant, which is a simulation implant, on the simulation image to simulate an implantation position and an implantation direction; a means for defining and registering a part of the oral cavity of the patient that is a target for the implant surgery or a part of an intraoral model of the patient that is a target for the implant surgery as an AR marker, detecting and tracking the AR marker from within an image captured by the camera, and displaying an implant 3D object, which indicates a position and a direction of implant placement, on the display device in real time by superimposing the AR marker on the image of the oral cavity or the image of the intraoral model captured by the camera; a means for matching a display position of the implant 3D object (indicating the position and direction of implant insertion) that is displayed by being superimposed on the image of the oral cavity captured by the camera or the image of the oral cavity model with a placement position of the shim implant placed on the simulation image, and for moving the shim implant when an operation is performed, so that the implant 3D object also moves in conjunction with the operation; A program that makes things work.

Images