Spinal surgery planning device and method based on two-dimensional medical images

Abstract

The present invention relates to a spinal surgery planning device and method. The spinal surgery planning method according to the present invention includes: acquiring a two-dimensional spinal image of a patient through a medical imaging device; calculating the registration relationship between the coordinates in the image space and the coordinates in the surgical space by aligning the surgical space in which the spinal surgery is performed on the patient and the image space of the spinal image; generating a virtual three-dimensional graphic in the surgical space; projecting the three-dimensional graphic onto the spinal image based on the registration relationship; adjusting the three-dimensional graphic so that the 3D graphic corresponds to a predetermined landmark on the spinal image; and setting the implantation position and implantation path of the spinal prosthesis based on the spinal image and the three-dimensional graphic. This enables planning to be performed based on a two-dimensional spinal image without the need for CT imaging, reducing the patient's radiation exposure and surgical complexity caused by CT image registration. In addition, planning can be performed accurately, effectively replacing CT images.

Description

Technical Field

[0001] The present disclosure relates to an apparatus and method for planning spinal surgery, and more particularly, to an apparatus and method for planning navigation surgery or robotic surgery based on two-dimensional (2D) spinal medical images. Background Art

[0002] Spinal surgeries using navigation systems or surgical robots require a planning step to create a surgical plan based on medical images of the patient. For example, before pedicle screw fixation surgery, in which pedicle screws are implanted and fixed to the vertebral body through the pedicles, the length and diameter of the pedicle screws must be appropriately determined based on the patient and the surgical site, and the placement and path of the screws must be planned.

[0003] When planning for pedicle screw fixation surgery, an axial view of the spine is most advantageous. Therefore, according to related art, a preoperative computed tomography (CT) axial image is provided to the operator so that preoperative planning can be performed based on the axial image.

[0004] Conventional planning methods based on CT images have the advantage of being able to plan on axial images, but have the disadvantage of being exposed to radiation for extended periods during CT scans, which is harmful to the human body. In particular, in spinal surgery, even during actual surgery, mobile C-arms or similar imaging devices are used to capture images in order to verify compliance with the surgical plan or to navigate surgical tools in real time, exacerbating the radiation exposure issue.

[0005] Furthermore, the need to register the preoperative planning CT images with the images captured during surgery by a C-arm or similar device is not only cumbersome (because registration must be performed again every time the patient or imaging device moves), but also causes problems with delays in the surgical process. Furthermore, registration between two-dimensional (2D) and three-dimensional (3D) images requires a high level of registration technology, and performing registration with high accuracy depending on image quality is technically very difficult. Furthermore, due to the nature of the spine, the relative spinal relationship between preoperative and intraoperative images can vary significantly depending on the patient's posture, making it cumbersome to repeat the registration multiple times for each spine that will undergo surgery.

[0006] Therefore, planning based on C-arm images or similar 2D medical images reduces radiation exposure from CT scans and eliminates the need for registration between 2D and 3D images, thus resolving the aforementioned issues. However, since axial views cannot be obtained using 2D medical images, using 2D medical images as a basis for planning is practically difficult. Summary of the Invention

[0007] The present disclosure aims to solve the above-mentioned problems of conventional spinal surgery planning technology, and one aspect of the present disclosure is to provide an apparatus and method for planning spinal surgery based on two-dimensional (2D) medical images such as C-arm images.

[0008] Aspects of the present disclosure can be implemented through a spinal surgery planning method, which includes: acquiring a two-dimensional (2D) spinal image of a patient through a medical imaging device; calculating the registration relationship between coordinates in the image space and coordinates in the surgical space by aligning the image space for the spinal image and the surgical space for spinal surgery on the patient; generating a virtual three-dimensional (3D) graphic in the surgical space; projecting the 3D graphic onto the spinal image based on the registration relationship; adjusting the 3D graphic so that the 3D graphic corresponds to a predetermined landmark on the spinal image; and setting the implantation position and implantation path of the spinal prosthesis based on the spinal image and the 3D graphic.

[0009] In the present disclosure, the adjusting the 3D graphic may include adjusting at least one of a shape, a size, and a position of the 3D graphic based on a user input with respect to the 3D graphic projected on the spine image.

[0010] At the same time, the setting of the implantation position and implantation path of the spinal prosthesis is based on a plurality of spinal images corresponding to different viewing angles.

[0011] Furthermore, the plurality of spinal column images corresponding to different viewing directions, respectively, may include an anterior-posterior (AP) image and a lateral-lateral (LL) image.

[0012] Furthermore, the setting of the implantation position and implantation path of the spinal prosthesis may include setting the implantation position and implantation path of the spinal prosthesis based on a user input using a user input unit.

[0013] Furthermore, adjusting the 3D graphic may include adjusting the 3D graphic so that a boundary of a graphic formed by projecting the 3D graphic onto the spinal column image may correspond to a boundary of a vertebral body on the spinal column image.

[0014] In the present disclosure, the 3D graphic has a columnar shape.

[0015] In addition, adjusting the 3D graphic may include adjusting the 3D graphic so that a first lateral line of the cylindrical shape can be aligned with a centerline of a spinal spinous process on a spinal image, wherein the first line is perpendicular to a first axis on the base of the cylindrical shape passing through a center of the base.

[0016] At the same time, the spinal surgery planning method further includes setting a reference point as a reference for implanting a spinal prosthesis on a spinal image, wherein setting the implantation position and implantation path of the spinal prosthesis may include limiting the allowable range of the implantation position or implantation path of the spinal prosthesis according to the reference point.

[0017] Furthermore, generating the 3D graph may include detecting positions of vertebrae on the spinal column image, and generating the 3D graph at the detected positions corresponding to the vertebrae.

[0018] Furthermore, generating the 3D graphic may include generating the 3D graphic corresponding to the features of the vertebrae identified in the spinal column image based on a library, wherein at least one attribute of the 3D graphic among size, shape and height is defined in the library according to the features of the vertebrae.

[0019] In addition, aspects of the present disclosure can be implemented by a spinal surgery planning device, which includes: a memory for storing a two-dimensional (2D) spinal image of a patient taken by a medical imaging device; a aligner for calculating the alignment relationship between coordinates in the image space and coordinates in the surgical space by aligning the image space for the spinal image and the surgical space for spinal surgery on the patient; a graphics generator for generating a virtual 3D graphic in the surgical space and projecting the 3D graphic onto the spinal image based on the alignment relationship; a graphics adjuster for adjusting the 3D graphic so that the 3D graphic corresponds to a predetermined landmark on the spinal image; and a planner for setting the implantation position and implantation path of the spinal prosthesis based on the spinal image and the 3D graphic.

[0020] In the present disclosure, the spinal surgery planning apparatus may further include a display for displaying a spinal image and a 3D graphic projected on the spinal image.

[0021] Furthermore, the spinal surgery planning device may further include a 3D view generator, which is used to generate a 3D image for the 3D graphics based on the alignment relationship, wherein the planner controls the changes of the spinal prosthesis on one of the spinal image and the 3D image based on the alignment relationship to reflect each other on the spinal image and the 3D image, thereby facilitating improving user convenience.

[0022] According to the present disclosure, planning can be performed based on two-dimensional (2D) medical images, thereby reducing radiation exposure caused by traditional computed tomography (CT). In addition, the registration process with three-dimensional (3D) images can be omitted, thereby increasing the convenience of surgery.

[0023] In addition, according to the present disclosure, while planning is performed based on 2D medical images, a three-dimensional (3D) view image is generated and provided for grasping the positional relationship between the vertebral body and the pedicle screw at the user's required perspective, thereby effectively replacing the CT image and ensuring the accuracy of navigation surgery and robotic surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a diagram schematically illustrating a spinal surgery system having a spinal surgery planning device according to an embodiment of the present disclosure;

[0025] Figure 2 is a block diagram of a spinal surgery planning apparatus according to an embodiment of the present disclosure;

[0026] Figure 3 and Figure 4 is a schematic diagram of projecting a virtual three-dimensional (3D) graphic generated by a graphic generator onto a two-dimensional (2D) spinal image according to an embodiment of the present disclosure;

[0027] Figure 5 and Figure 6 is an example of adjusting 3D graphics by a graphics adjuster according to an embodiment of the present disclosure;

[0028] Figure 7 An example of a 3D image generated by a 3D view generator according to an embodiment of the present disclosure is shown;

[0029] Figure 8 shows an example of a screen provided by a display during a planner's planning process according to an embodiment of the present disclosure; and

[0030] Figure 9 is a flowchart illustrating a spinal surgery planning method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] The specific embodiments of the present disclosure will be described below with reference to the accompanying drawings. However, detailed descriptions of well-known functions or configurations may obscure the gist of the present disclosure and will therefore be omitted in the following description and drawings. It should be noted that, where possible, similar reference numerals refer to similar elements throughout the text.

[0032] Figure 1 FIG. 1 is a diagram schematically illustrating a spinal surgery system having a spinal surgery planning apparatus according to an embodiment of the present disclosure.

[0033] Reference Figure 1 The spinal surgery system 1 placed in the operating room includes a medical imaging device 100, a surgical robot 200, a tracking device 300 and a spinal surgery planning device 400.

[0034] The medical imaging device 100 is a device for capturing images of a patient's spine. Figure 1 A C-arm is shown as an example of medical imaging apparatus 100. For reference, a C-arm refers to an apparatus that captures two-dimensional (2D) images using an X-ray source and detectors provided at both ends of a C-shaped frame. However, the C-arm is merely one example of medical imaging apparatus 100, and other apparatuses capable of capturing 2D images by irradiating a human body with X-rays or the like may also be used as medical imaging apparatus 100.

[0035] The medical imaging apparatus 100 can obtain multiple medical images at various viewing angles (i.e., observed from different directions or angles) while simultaneously moving the X-ray source and detector relative to the patient by rotating or translating the frame. For example, anterior-posterior (AP) images can be obtained by irradiating X-rays from the front to the back of the human body, and lateral-lateral (LL) images can be obtained by irradiating X-rays from side to side.

[0036] The surgical robot 200 is a robot for performing pedicle screw implantation or similar spinal surgery. The surgical robot 200 includes a robot base 201, a robotic arm 203, and a robot controller 205 for controlling the operation of the robotic arm 203. Various surgical tools 203a (such as reaming tools and drivers) can be coupled to the end of the robotic arm 203, i.e., the end effector. In addition, the surgical robot 200 can be provided with a force / torque sensor (not shown) that can detect the force and torque applied to the end effector. The base 201 and surgical tools 203a of the surgical robot 200 can be provided with optical markers that can be used as a reference for tracking the position of the surgical robot 200 during surgery.

[0037] For reference, Figure 1 The surgical robot 200 is shown being used to perform spinal surgery, but a medical navigation system may be used instead of the surgical robot 200 .

[0038] The tracking device 300 is used to track the position and posture of optical markers fixed to the surgical site and the surgical robot 200, and can be implemented as an optical tracking system (OTS). For reference, OTS refers to a system in which two infrared cameras are used to track markers and the distances of these markers are converted by triangulation, thereby tracking the position and posture in a three-dimensional (3D) space in real time. Commercially available OTS not only provides the distance, direction and height of the optical marker, but also has the function of converting between optical marker coordinate systems. The tracking principle of this OTS is well known, so for the sake of simplicity, its detailed description will be omitted.

[0039] For spinal surgery using a navigation system or a surgical robot, the spinal surgery planning device 400 can identify the length, diameter, size, etc. of various spinal prostheses (such as pedicle screws or the like) to be implanted and fixed during spinal surgery, and provide a computer simulation environment in which surgical planning can be made for the implantation position, direction, angle, path, depth, etc.

[0040] The spinal surgery planning apparatus 400 may include a processor and a display. Figure 1 The spinal surgery planning device 400 is shown as being implemented as an additional device physically separate from the surgical robot 200. However, if necessary, the processor of the spinal surgery planning device 400 may be placed within the surgical robot 200, and the display may be connected to the tracking device 300. In this case, the processor and the display may transmit and receive various information through the communication module.

[0041] Figure 2 is a block diagram showing a detailed configuration of a spinal surgery planning apparatus 400 according to an embodiment of the present disclosure. Figure 2 The spinal surgery planning apparatus 400 according to an embodiment of the present disclosure includes a receiver 410 , a user input unit 420 , a display 430 , a memory 440 , and a controller 450 .

[0042] The receiver 410 is used to receive signals or various data from the outside, and may include, for example, a high-definition multimedia interface (HDMI) connector and an analog interface connector for connecting to external devices, or a communication module for connecting to a wired / wireless network (such as the Internet). The receiver 410 sends and receives various information from the medical imaging device 100, the surgical robot 200, and the tracking device 300. For example, the receiver 410 can receive images captured by the medical imaging device 100 and information related to various robot states detected from the surgical robot 200. It can also receive information related to the position and posture of the surgical robot 200 and information related to the surgical site tracked by the tracking device 300.

[0043] The user input unit 420 is used to receive various user inputs during spinal surgery planning and surgical operations and transmit the received inputs to the controller 450 to be described later, and the user input unit 420 may include various input devices such as a keyboard, a mouse, and buttons.

[0044] The display 430 is used to display various information on the screen, including images, graphics, etc., and the display 430 may include a liquid crystal display (LCD) panel, a light-emitting diode (LED) panel, an organic light-emitting diode (OLED) panel, etc. In addition, the user input unit 420 and the display 430 may be integrated and implemented as a single device, such as a touch screen. The display 430 displays spinal images, planning images for setting the implantation position and implantation path of a spinal prosthesis, etc. during robotic surgery.

[0045] The user input unit 420 and the display 430 may be physically separated from other components. For example, the receiver 410, the memory 440, and the controller 450 may be integrated into the main body of the surgical robot 200, while the user input unit 420 and the display 430 may be implemented as separate devices that are connected to and communicate with the tracking device 300 and the surgical robot 200.

[0046] The memory 440 may be implemented as a storage device such as a random access memory (RAM) and may be used to store various operating systems (OSs), middleware, platforms, and various applications of the spinal surgery planning device 400, as well as program code, processed video and audio signals, and various data. Furthermore, the memory 440 may be used to store 2D spinal images of a patient captured by the medical imaging device 100, libraries used during spinal surgery planning, and reference information regarding the implantation of spinal prostheses.

[0047] The controller 450 controls the overall operation of the spinal surgery planning device 400 based on user input received via the user input unit 420 or based on an internal program. The controller 450 may include program code for processing and controlling signals, and a processor for executing the program. The controller 450 provides a planning function for spatially registering the spinal image with the surgical space in which the patient's spinal surgery is to be performed, based on the 2D spinal image received via the receiver 410 and the position / posture information of the optical marker received from the tracking device 300, and for determining the implantation position and implantation path of the spinal prosthesis.

[0048] Reference Figure 2 Controller 450 includes a register 451, a graphic generator 453, a graphic adjuster 455, a 3D view generator 457, and a planner 459. For ease of description, controller 450 is functionally divided into subcomponents. However, controller 450 can be implemented using a software program having instructions for executing the functions of the subcomponents, namely, register 451, graphic generator 453, graphic adjuster 455, 3D view generator 457, planner 459, and a processor for running the software program.

[0049] The aligner 451 calculates the alignment relationship between the image space coordinates and the surgical space coordinates by aligning the image space of the spinal image captured by the medical imaging device 100 with the surgical space in which the patient's spinal surgery is performed. In this case, the aligner 451 can align multiple 2D images (e.g., AP images and LL images) captured from different directions of the patient to the 3D surgical space. For reference, the image space is defined relative to the image coordinate system, while the surgical space is defined relative to the coordinate system of the surgical space, wherein the coordinate system of the surgical space can be defined based on markers in the surgical space, such as markers attached to the surgical site or the like of the patient and detected by the tracking device 300.

[0050] The register 451 can align the coordinate systems of the surgical space and the image space based on various known registration algorithms. For example, the register 451 calculates the spatial coordinates of the X-ray source and detector by detecting markers attached to the X-ray source and detector using the tracking device 300, thereby identifying X-ray projection parameters when capturing 2D spinal images. The register 451 then performs registration based on the calculated parameters by calibrating the medical imaging device 100. Therefore, the registration of the image space with the surgical space based on the calibration parameters of the medical imaging device 100 is well known to those skilled in the art, and for the sake of simplicity, a detailed description thereof will be omitted.

[0051] Meanwhile, registration can be performed using the method disclosed in Korean Patent Publication No. 2019-0028592, filed by the present applicant with the Korean Intellectual Property Office on March 13, 2019, entitled "C-arm Medical Imaging System and Registration Method for 2D Images and 3D Space." In short, the register 451 performs registration based on a projection image generated by back-projecting a 2D spinal image captured by the medical imaging device 100 onto another plane on the X-ray projection path, where the spatial position is accurately grasped using a warping algorithm. This registration method may be particularly useful when the positions of the X-ray source and detector of the medical imaging device 100 are difficult to identify.

[0052] Therefore, as a result of registration by the registrar 451 , it is possible to grasp the matching position between the space of the plurality of 2D spine images photographed by the medical imaging apparatus 100 and the surgical space.

[0053] The graphic generator 453 generates a virtual 3D graphic in the surgical space and projects the 3D graphic onto the spinal image based on the registration relationship calculated by the aligner 451. In this case, the virtual 3D graphic may have a cylindrical shape, such as an elliptical cylinder, a circular cylinder, and a polygonal prism. The graphic generator 453 may generate the 3D graphic in a position space corresponding to the center of the spinal image based on the registration relationship calculated by the aligner 451. In addition, the graphic generator 453 may alternatively detect the position of the surgical target vertebra in the spinal image and generate the 3D graphic in a position space corresponding to the detected vertebra. The graphic generator 453 may generate a virtual 3D graphic at a position in the surgical space corresponding to the vertebra on the 2D spinal image based on the registration relationship calculated by the aligner 451 and project the virtual 3D graphic onto the spinal image.

[0054] In this case, the position of the vertebral body can be detected by having the user personally input the position of the vertebral body through the user input unit 420, or it can be automatically detected using various well-known image recognition algorithms, wherein these image recognition algorithms can analyze the brightness and color of the spinal image, the shape of the image object, etc. and detect specific parts or regions. Furthermore, the position of the vertebral body can be automatically detected by performing machine learning based on labeled spinal image training data, or by performing deep learning based on algorithms such as convolutional neural networks (CNNs) and convolutional deep belief networks (CDBNs).

[0055] The image generator 453 can generate 3D images based on any shape and size, but can generate 3D images corresponding to the shapes, sizes, and similar unique features of vertebrae identified in spinal images based on a library stored in memory 440. In the library, the size, shape, height, and similar attributes of the 3D images are defined based on vertebral characteristics or patient characteristics. Therefore, when providing a 3D image tailored to each patient, differences in vertebral size, shape, and other aspects between patients can be reflected. The library can be pre-established and stored based on standard statistical data, taking into account factors related to the shape, size, height, and similar attributes of vertebrae corresponding to the patient's age, gender, height, and similar characteristics, as well as the location of the vertebra currently undergoing surgery.

[0056] The graphic generator 453 may generate a 3D graphic selected by the user from the library, or may generate a 3D graphic by receiving patient information and vertebral attribute information input by the user input unit 420 and selecting a 3D graphic from the library based on the received information. Furthermore, the graphic generator 453 may process the patient's spinal image using an image recognition algorithm to identify the shape and size of the patient's vertebral body, and generate a 3D graphic suitable for the patient from the library based on the recognition result.

[0057] The pattern generator 453 can generate a 3D pattern whose column type, column shape, column height, size, base radius, and ratio between the major and minor axes of the ellipse vary according to the specific shape of the vertebral body, the location of the surgical target vertebral body, and patient characteristics. For example, if the patient is young or short, or if the patient's vertebral body is slightly smaller than the statistically corresponding vertebral body, a matching 3D pattern with a relatively smaller size or height can be generated.

[0058] The 3D graphics generated by the graphics generator 453 and projected onto the 2D spinal column image are displayed overlapping with the spinal column image and provided to the user via the display 430 .

[0059] As described above, the graphics generator 453 adaptively selects the initially generated 3D graphics based on the patient's characteristics (age, gender, height, etc.), the position of the surgical target vertebra, and the vertebral characteristics, thereby minimizing the process of adjusting the 3D graphics to be described below, thereby improving user convenience and shortening the time required for planning.

[0060] Figure 3 and Figure 4 FIG. 4 is a schematic diagram illustrating projecting a virtual 3D graphic generated by the graphic generator 453 onto a 2D spine image according to an embodiment of the present disclosure.

[0061] Reference Figure 3 and Figure 4 , Figure 3 It is a schematic diagram of projecting a 3D figure V having an elliptical cylindrical shape onto the AP image. Figure 4 This is a schematic diagram of projecting a 3D figure V onto an LL image. In this disclosure, the first line vd1, corresponding to the centerline of the upper portion of the patient's vertebral body, is the uppermost of two lateral lines vd1 and vd2. These lines vd1 and vd2 are perpendicular to a first axis va1 on the base v1, which passes through the center O of the base v1 of the 3D figure. The second line vd2, corresponding to the centerline of the lower portion of the patient's vertebral body, is the lowermost of two lateral lines vd1 and vd2. These lines vd1 and vd2 are perpendicular to the first axis va1. Thus, vd1 and vd2 are arranged symmetrically with respect to the central axis of the 3D figure.

[0062] At the same time, Figure 3 In, vd1 AP and vd2 AP is a line formed by projecting the lateral lines vd1 and vd2 of the 3D figure V onto the AP spine image, and vd1 LL and vd2 LL It is a line formed by projecting the lateral lines vd1 and vd2 of the 3D figure V onto the LL spine image. For reference, vd1AP 、vd2 AP Position and VD1 on AP spine images LL and vd2 LL The position on the LL spine image may vary depending on the specific position of the X-ray source XS of the medical imaging apparatus 100 .

[0063] The graphic adjuster 455 adjusts the 3D graphic so that the 3D graphic projected on the spinal image (hereinafter referred to as the projected graphic) can correspond to the predetermined landmark on the spinal image. The graphic adjuster 455 can change the shape, size, height, position, etc. of the 3D graphic. In this case, the 3D graphic can be adjusted according to the user input input through the user input unit 420. To this end, the graphic adjuster 455 can generate control points that can be adjusted by the user and provide the control points to a part of the 3D graphic projected onto the spinal image by the graphic generator 453, so that the user can change the shape, size, height, position, etc. of the graphic projected and displayed superimposed on the spinal image by moving the position of the control point through the mouse, keyboard, etc. of the user input unit 420.

[0064] Figure 5 and Figure 6 4 is an example showing that the graphics adjuster 455 adjusts the 3D graphics in response to the control implemented by the user using the user input unit 420. For reference, Figure 5 shows that the 3D graphics are adjusted on the AP image, and Figure 6 It is shown that the three-dimensional graphics are adjusted on the LL image.

[0065] Reference Figure 5 , projection graph V on AP spine image AP 'With four sides S AP , I AP 、L AP and R AP , and as described above, includes a projection line vd1 corresponding to two lines vd1 and vd2 in the lateral direction of the 3D graphic V AP and vd2 AP .

[0066] The user can control the projection image V by using the user input unit 420 AP 'The control points p1, p2, p3, p4, p5 and p6 are provided to change the projection figure V AP 'position, size, shape, etc.

[0067] Figure 5 Shows the change of projection pattern V on AP spine image AP ' Example of the position and size of the projection graphic V. For example, the user can adjust the projection graphic V AP' position, size, etc., so that the projection graphic V AP 'The boundary, that is, the four sides S AP , I AP 、L AP and R AP , corresponding to the boundary of the surgical target vertebra on the AP spine image. In this case, S AP The sides are adjusted to correspond to the upper part of the vertebral body, I AP The sides are adjusted to correspond to the lower part of the vertebral body, R AP The edge is adjusted to correspond to the right border of the vertebral body, and L AP The edge is adjusted to correspond to the left border of the cone. Figure 5 In the projection graph V AP 'Before adjustment, it was misaligned with the vertebra on the image, but after adjustment, it corresponded to the position and shape of the vertebra on the image.

[0068] At the same time, users can adjust the projection graphics V AP ', so that the first projection line vd1 corresponding to the first side line vd1 of the 3D graphic V AP It can be aligned with the center line of the spinous process on the AP spine image, wherein the first lateral line vd1 corresponds to the center line of the upper part of the vertebral body. AP When the position of is moved, the second projection line vd2 set symmetrically with it AP The position has also been moved.

[0069] Figure 6 The projection graph V of the LL spine image is shown. LL ' Example of location and size. Figure 6 , projection figure V LL 'With four edges P LL 、S LL 、A LL and I LL , and includes a projection line vd1 corresponding to two lines vd1 and vd2 in the lateral direction of the 3D graphic V LL and vd2 LL For reference, Figure 6 In the middle, six lines, namely P LL 、S LL 、A LL , I LL 、vd1 LL and vd2 LL , all are visible to the naked eye before adjustment, but only four lines are visible after adjustment, because the first projection line vd1 LL With P LL The edges overlap and look like a line, the second projection line vd2 LL With ALL The edges overlap and appear to be a single line, but in reality, there are still six lines as before the adjustment.

[0070] The user can adjust the projection image V by using the user input unit 420 LL 'The control points p7, p8, p9, p10, p11 and p12 are provided to change the projection figure V LL ' position, size, shape, etc. For example, the user can adjust the projection graphic V LL ', so that the projection graph V LL 'The four sides P LL 、S LL 、A LL and I LL Can correspond to the boundaries of the vertebral bodies on the LL spine image. In the present disclosure, the S LL The side is adjusted to correspond to the upper part of the target vertebral body, I LL The edge is adjusted to correspond to the lower part of the vertebral body, P LL The edges are adjusted to correspond to the posterior border of the vertebral body, A LL The edge is adjusted to correspond to the front border of the vertebral body. In addition, the first projection line vd1 LL can be adjusted to align with the centerline of the spinous processes on the LL spine image. In this case, as with the AP spine image, when the first projection line vd1 LL When the position of is moved on the LL spine image, the second projection line vd2 set symmetrically with it LL The position will also be moved.

[0071] In this manner, when a user changes the position, size, shape, etc. of a graphic projected on a 2D spinal image, such as the AP spinal image and the LL spinal image, the graphic adjuster 455 adjusts the position, size, shape, etc. of the 3D graphic by equally reflecting the changes in the projected graphic in the 3D graphic of the surgical space based on the registration results of the registerer 451. Furthermore, given multiple 2D spinal images, adjustments to the projected graphic on one spinal image can even be reflected in the other spinal images based on the registration results. For example, when a user adjusts the projected graphic on the AP image, the adjustments to the AP image are also applied to the LL image.

[0072] In addition, the graphic adjuster 455 can be used to identify predetermined landmarks on the spinal image based on various well-known image recognition algorithms without the need for the aforementioned user input, and can automatically adjust the 3D graphic so that specific portions of the 3D graphic can be aligned with the landmarks. The landmarks can be identified by machine learning or deep learning. At the same time, the landmarks can include the boundaries of the vertebral bodies, points on the boundaries of the vertebral bodies, the centerline of the spinous processes of the spine, or points on the centerline of the spinous processes of the spine. Therefore, as described in the previous example, the projected graphic can be automatically adjusted to correspond to the boundaries of the vertebral bodies and the centers of the spinous processes of the spine, wherein the projected graphic is adjusted based on user input.

[0073] In this way, as the projected graphics on the 2D spinal image correspond to changes in the vertebral body, the 3D graphics in the surgical space also change accordingly, thereby approximating the position, size, shape, etc. of the patient's vertebral body in the surgical space through the 3D graphics.

[0074] The 3D view generator 457 generates a 3D image for the 3D graphics based on the registration result of the register 451 and provides a view of the 3D image that can be transformed according to a user control using the user input unit 420. The generated 3D image is displayed on the display 430.

[0075] 3D view generator 457 may generate a 3D image after the graphics have been fully adjusted by graphics adjuster 455. However, it may also generate and provide a 3D image immediately after the 3D graphics have been generated by graphics generator 453, so that the user can intuitively and directly check the changes through the 3D view while adjusting the 3D graphics. In this case, changes in the projected graphics on the 2D spine image can be reflected in real time in the 3D graphics on the 3D image based on the registration relationship calculated by registerer 451.

[0076] Figure 7 An example of a 3D image generated by the 3D view generator 457 according to an embodiment of the present disclosure is shown, where it is assumed that the 3D figure is an elliptical cylinder.

[0077] refer to Figure 7 , Figure 7 (a) shows an axial view image according to the axis direction, and (b) shows a 3D image deviated from the axis direction. This example shows the view of the 3D image, which can be freely transformed by the user through the user input unit 420.

[0078] The planner 459 is used to formulate a surgical plan for the length, diameter, size, implantation position, implantation direction, implantation angle, implantation path, and implantation depth of the spinal prosthesis based on the spinal image and 3D graphics. In this case, pedicle screws can be used as the spinal prosthesis. However, spinal prostheses can include various prostheses implanted and fixed to the spine during spinal surgery, as well as pedicle screws. Pedicle screws will be described below as an example of a spinal prosthesis.

[0079] Before setting the specific implantation position, direction, angle, path, etc. of the spinal prosthesis, the planner 459 may first set reference feature points or positions on the spinal image. For example, when implanting a pedicle screw, a pivot point may be set at a position corresponding to the center of the pedicle on the spinal image as a reference point for implanting the spinal prosthesis. In this case, the user may personally input the pivot point through the user input unit 420 by considering a 3D image and the shape of the pedicle, wherein the 3D image is an approximation of the vertebral body using a 2D spinal image and a 3D image provided by the 3D view generator 457. Alternatively, the planner 459 may automatically identify the pivot point on the spinal image based on a previously trained learning model such as machine learning or deep learning.

[0080] The planner 459 can establish a surgical plan for a spinal prosthesis based on user input and a reference point, wherein the reference point is set like the aforementioned fulcrum and is used as a reference for implanting the spinal prosthesis. To this end, the planner 459 can generate and provide a virtual spinal prosthesis, which is displayed overlapping with the 2D spinal image and the 3D image of the 3D graphics through the display 430, and the user can control the implantation position, angle or direction of the virtual spinal prosthesis through the user input unit 420. In this case, even when the user moves the position or angle of the spinal prosthesis through the user input unit 420, the planner 459 can still limit the movable range of the virtual spinal prosthesis so that it rotates around the fulcrum and ensures the implantation path of the spinal prosthesis or ensures that its extension line passes through the fulcrum.

[0081] For planning, the user can use multiple spinal images with different viewpoints, such as AP and LL images, as well as 3D images generated by the 3D view generator 457 for 3D graphics. When the user changes the position, angle, and other aspects of the spinal prosthesis in one of the multiple images, the planner 459 controls the corresponding changes to be reflected in the spinal images and 3D images with other viewpoints based on the registration relationship calculated by the register 451. As described above, the multiple view images are interconnected, allowing the user to create a surgical plan without having to repeat the changes for each view image.

[0082] The planner 459 may pre-set standards for position and angle limits of the spinal prosthesis relative to the 3D graphics approximating the vertebral body, thereby providing a warning message when the user changes the position or angle of the virtual spinal prosthesis beyond the position or angle based on these standards, or limiting the change to not exceed the set standards.

[0083] Figure 8 An example of a screen provided by the display 430 during a planning process of the planner 459 according to an embodiment of the present disclosure is shown.

[0084] Reference Figure 8 , an AP spine image (a) is displayed on the left side of the screen, an LL spine image (b) is displayed on the right side of the screen, and a 3D image (c) for the 3D graph is displayed in the middle upper portion between the two images (a) and (b). Projection graph V AP ' and V LL ' can be displayed in colors different from the vertebral body colors (e.g. green and red) and overlapped with the 2D spine images (a) and (b). Alternatively, the projection graph V AP ' and V LL ' can be displayed based on light grey or similar colors with different saturation or brightness so as not to obstruct the vertebral body during the planning process. Of course, the projection graphic V AP ' and V LL ' may not be displayed on the spine image according to the user's selection or setting.

[0085] like Figure 8 As shown, planning can be performed while checking the implantation position, path, etc. of the spinal prosthesis in the 2D spinal image and the 3D image through the display 430 based on different views, thereby improving user convenience and facilitating accurate planning.

[0086] When the user changes the position, angle, etc. of the virtual spinal prosthesis 601 in one of the three images (a), (b), and (c) through the user input unit 420, the planner 459 associates the three images with each other so that the corresponding changes are applied to all images. In this case, as described above, the planner 459 uses the set fulcrum PE AP and PE LL The spinal prosthesis 601 is rotated around the fulcrum as the fulcrum center, and the implantation path of the spinal prosthesis 601 or its extension line is ensured to pass through the fulcrum, thereby limiting the allowable range of the implantation position, movement, implantation path, etc. of the spinal prosthesis 601.

[0087] At the same time, the planner 459 can automatically implant a virtual spinal prosthesis according to pre-stored or user-set implantation standards by using various image recognition algorithms, machine learning, and deep learning image recognition to identify specific areas or locations of the spine on the spinal image, rather than relying solely on user input as described above, and display the implanted virtual spinal prosthesis through the display 430.

[0088] For example, Figure 8 As shown, when selecting the implantation position of the pedicle screw 601, the planner 459 can position the implantation path of the pedicle screw 601 or its extension line to pass through the fulcrum PE. AP and PE LL , and simultaneously the pedicle screw 601 is automatically implanted according to a preset implantation angle.

[0089] In addition to the implantation position and angle of the pedicle screws, the diameter and length of the pedicle screws can be selected based on pre-stored standards or standards previously set by the user. To this end, the memory 440 can pre-store appropriate diameters, lengths, or similar reference specifications or products of spinal prostheses corresponding to the shape, size, and length of the 3D graphic and the target spinal position for surgery, and select virtual prosthesis specifications or products that are suitable for the adjusted 3D graphic. For example, if the 3D graphic is long or large, a prosthesis with a relatively large length or diameter can be selected as the virtual prosthesis. The planner 459 can provide the user with an opportunity to modify the position, angle, length, diameter, etc. of the automatically implanted virtual spinal prosthesis.

[0090] Figure 9 4 is a flowchart illustrating a spinal surgery planning method according to an embodiment of the present disclosure. Hereinafter, the organic operations between the aforementioned elements of the spinal surgery planning apparatus 400 will be described. This will avoid repetitive descriptions of the aforementioned embodiments.

[0091] Reference Figure 9 The spinal surgery planning method according to the embodiment of the present disclosure is based on the premise of obtaining a 2D spinal image of the patient (S10) through the medical imaging device 100. In this case, multiple images with different viewing angles, such as AP images and LL images, can be collected as the 2D spinal image.

[0092] Then, the register 451 registers the image space of the acquired 2D spinal image with the surgical space and calculates the registration relationship between the image space coordinates and the surgical space coordinates (S20). In this case, as described above, the registration can be implemented by a well-known registration method based on the calibration parameters of the medical imaging device 100, or by the method disclosed in Korean Patent Publication No. 2019-0028592, filed by the present applicant with the Korean Intellectual Property Office on March 13, 2019, entitled "C-ARM MEDICAL IMAGING SYSTEM AND REGISTRATION METHOD OF 2D IMAGE AND 3D SPACE".

[0093] When the registration is completed, the graphics generator 453 generates a virtual 3D graphics in the surgical space and projects the 3D graphics onto the 2D spinal image based on the registration relationship (S30). The 3D graphics may have a cylindrical shape. In this case, the 3D graphics may be generated according to any shape, size, and position. However, a library may be considered to generate a 3D graphics customized for the patient, that is, in the library, the size, shape, height, and other attributes of the 3D graphics are defined according to the unique characteristics of the patient's vertebrae or the patient's characteristics (e.g., gender and age, number of vertebrae targeted for surgery, etc.).

[0094] The graphic adjuster 455 adjusts the shape, size, height, and position of the 3D graphic so that the 3D graphic projected onto the spinal image corresponds to a predetermined landmark on the spinal image (S40). In this case, the 3D graphic in the surgical space can be adjusted by changing the shape, size, height, and position of the 3D graphic projected onto the spinal image based on the registration relationship. As a result, the shape, size, and position of the patient's vertebral body are approximated by the 3D graphic.

[0095] Then, the planner 459 sets the implantation position and path of the spinal prosthesis based on the spinal image and the 3D graphics approximating the vertebral body (S50). In this case, a 3D image of the 3D graphics in the surgical space is provided so that the user can refer to the 3D image and the 2D spinal image for planning, thereby having an effect similar to that of using a CT image.

[0096] At the same time, the planner 459 can first set a reference point as a reference for implanting the spinal prosthesis based on user input, a well-known image recognition algorithm, or a learning model, as described above, before setting the specific implantation position and path of the spinal prosthesis. The planner 459 can automatically set the implantation position and path of the spinal prosthesis, or limit the allowable range of the implantation position or path based on user control implemented using the user input unit 420 to meet implantation conditions, thereby allowing the spinal prosthesis to rotate about the reference point set above, for example, about a fulcrum, or allowing the implantation path of the spinal prosthesis or its extension to pass through the fulcrum.

[0097] Furthermore, the planner 459 automatically implants and provides a virtual spinal prosthesis according to pre-stored positions and angles or according to implantation criteria set by the user, thereby improving user convenience.

[0098] As described above, the spinal surgery planning device 400 and spinal surgery planning method according to the present disclosure provide a 3D image that approximates the size, shape, and other characteristics of the patient's spine during the planning process, thereby enabling planning based on 2D spinal images without the need for CT scanning. This allows accurate planning while reducing the patient's radiation exposure and the surgical complexity associated with CT image registration, effectively replacing CT images.

[0099] Although it has been described above that all elements constituting the embodiments of the present disclosure are combined into a single unit or coupled into a single unit to operate, the present disclosure is not necessarily limited to such embodiments. In other words, at least two elements can be selectively combined and operated without departing from the scope of the present disclosure. In addition, each element can be implemented as independent hardware, but some or all elements can be selectively combined with each other, so the elements can be implemented as a computer program, and the computer program has a program module, and the program module is used to perform some or all functions combined in one or more hardware. The code and code segment constituting the computer program can be easily conceived by a person of ordinary skill in the art of the present disclosure. Such computer programs can implement the embodiments of the present disclosure as being stored in a computer-readable medium and read and executed by a computer. The medium of the computer program may include a magnetic recording medium and an optical recording medium.

[0100] In addition, terms such as "include," "comprise," or "have" refer to the presence of corresponding elements, unless otherwise specifically described, and should be interpreted as including one or more other elements without excluding their presence. Unless otherwise defined, all terms, including technical or scientific terms, have the same meaning as understood by a person having ordinary knowledge in the field to which this disclosure relates. General terms defined in dictionaries should be interpreted as having a meaning consistent with the context of the relevant technology and will not be interpreted as having an idealistic or overly formal meaning unless otherwise clearly defined in this disclosure.

[0101] Although the embodiments of the present disclosure have been described for illustrative purposes, it will be understood by those skilled in the art that various modifications and variations can be made without departing from the basic features of the present disclosure. Therefore, the embodiments disclosed in the present disclosure are intended only to describe the technical spirit of the present disclosure without limitation, and the technical spirit of the present disclosure is not limited by these embodiments. In addition, the scope of the present disclosure should be interpreted based on the appended claims, and all technical ideas within the scope of the appended claims should also be interpreted as included within the scope of the present disclosure.

Claims

1. A spinal surgery planning method comprising: Acquiring a two-dimensional (2D) spinal image of the patient through a medical imaging device; Calculating a registration relationship between coordinates in the image space and coordinates in the surgical space by registering an image space for the 2D spinal image and a surgical space for performing spinal surgery on the patient; generating a virtual three-dimensional (3D) graphic having a cylindrical shape in the surgical space; projecting the 3D graphics onto the 2D spinal column image based on the registration relationship, and displaying the projected 3D graphics overlapping the 2D spinal column image; adjusting the 3D shape so that the projected 3D shape corresponds to predetermined landmarks on the 2D spinal image; as well as The implantation position and implantation path of the spinal prosthesis are set based on the 2D spinal column image and the 3D graph.

2. The spinal surgery planning method according to claim 1, wherein: The adjusting the 3D graphic includes adjusting at least one of a shape, a size, and a position of the 3D graphic based on user input to the 3D graphic projected on the 2D spine image.

3. The spinal surgery planning method according to claim 1, wherein: The implantation position and implantation path of the spinal prosthesis are set based on a plurality of 2D spinal images corresponding to different viewing angles.

4. The spinal surgery planning method according to claim 3, wherein: The plurality of 2D spine images corresponding to different viewing directions respectively include an anterior-posterior (AP) image and a lateral-lateral (LL) image.

5. The spinal surgery planning method according to claim 1, wherein: The setting the implantation position and the implantation path of the spinal prosthesis includes setting the implantation position and the implantation path of the spinal prosthesis based on a user input using a user input unit.

6. The spinal surgery planning method according to claim 1, wherein: The adjusting the 3D graphic includes adjusting the 3D graphic so that a boundary of a graphic formed by projecting the 3D graphic onto the 2D spine image corresponds to a boundary of a vertebra on the 2D spine image. 7 . The spinal surgery planning method of claim 1 , further comprising generating a 3D image for the 3D graphic of the surgical space.

8. The spinal surgery planning method according to claim 1, wherein: Adjusting the 3D graphic includes: adjusting the 3D graphic so that a first lateral line of the columnar shape is aligned with a centerline of a spinal spinous process on the 2D spine image, and the first line is perpendicular to a first axis on the base of the columnar shape passing through the center of the base.

9. The spinal surgery planning method according to claim 1, further comprising setting a reference point as a reference for implanting the spinal prosthesis on the 2D spinal image, wherein The setting of the implantation position and implantation path of the spinal prosthesis includes limiting an allowable range of the implantation position or the implantation path of the spinal prosthesis according to the reference point.

10. The spinal surgery planning method according to claim 1, wherein: The generating of the 3D graph includes: detecting the position of the vertebral body on the 2D spine image, and generating the 3D graph at the detected position corresponding to the vertebral body.

11. The spinal surgery planning method according to claim 1, wherein: Generating the 3D graph includes generating a 3D graph corresponding to features of the vertebra identified in the 2D spine image based on a library, wherein at least one attribute of the 3D graph, including size, shape, and height, is defined in the library according to the features of the vertebra.

12. A spinal surgery planning device comprising: a memory for storing a two-dimensional (2D) spinal image of a patient captured by a medical imaging device; a register, configured to calculate a registration relationship between coordinates in the image space and coordinates in the surgical space by registering the image space for the 2D spinal image and the surgical space in which the spinal surgery is performed on the patient; a graphics generator for generating a virtual three-dimensional (3D) graphics having a cylindrical shape in the surgical space, and projecting the 3D graphics onto the 2D spinal image based on the registration relationship; a display for displaying the projected 3D graphics overlapping with the 2D spinal image; a graphic adjuster for adjusting the 3D graphic so that the projected 3D graphic corresponds to a predetermined landmark on the 2D spinal image; as well as A planner is used to set an implantation position and an implantation path of a spinal prosthesis based on the 2D spinal image and the 3D graphics.

13. The spinal surgery planning apparatus according to claim 12, further comprising a 3D view generator configured to generate a 3D image for the 3D graphic according to the registration relationship. in, The planner controls the changes of the spinal prosthesis on one of the 2D spinal image and the 3D image of the 3D graph to be reflected on each other on the 2D spinal image and the 3D image based on the registration relationship.

Images