Multi-planar reconstruction-based adjustable geometric controller for three-dimensional surgical planning

Abstract

Disclosed in the present invention is a multi-planar reconstruction (MPR)-based adjustable geometric controller for three-dimensional surgical planning. The adjustable geometric controller comprises a three-dimensional MPR geometric controller. The three-dimensional MPR geometric controller comprises three planes which are perpendicular to each other. Each plane comprises four edges and four vertices for circumscribing a DICOM image; by means of dragging the edges of each plane, the plane is translated along the plane normal vector direction; by means of dragging the vertices, the area size of the plane is changed; and after a rotation axis is selected, the edge perpendicular to the rotation axis of the plane where the rotation axis is located is dragged, thereby synchronously adjusting the plane and the normal vector direction of the orthogonally constrained planes for the plane. The multi-planar reconstruction-based adjustable geometric controller for three-dimensional surgical planning which is disclosed in the present invention aims to optimize operations such as translation and rotation of a 3D object in a three-dimensional space, expand applications of the 3D modeling technology in medical images and surgical planning, and solves the problem about performing accurate positioning and operations by means of mouse interaction in a three-dimensional space.

Description

Adjustable geometry controller for 3D surgical planning based on multi-planar reconstruction

[0001] This patent application claims priority from the following Chinese patent applications:

[0002] Filing date: March 25, 2024; Application number: 202410344000.7; Invention title: Adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction;

[0003] The entirety of the above application is incorporated herein by reference. Technical Field

[0004] The present invention belongs to the technical field of three-dimensional surgical planning, and in particular relates to an adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction. Background Art

[0005] In orthopedics, doctors often rely on imaging to observe a patient's bone structure for accurate diagnosis and treatment planning. While traditional two-dimensional images can provide some information, they can have limitations when observing complex bone structures, such as occlusion, overlap, and blurring.

[0006] With the continuous advancement of medical imaging technology, doctors can obtain more accurate and detailed patient skeletal data and use computer software to accurately reconstruct the skeleton in three dimensions. Orthopedic three-dimensional volumetric reconstruction technology generates personalized bone models and volume calculations based on the patient's specific circumstances, effectively reducing human error and helping doctors develop more accurate and sophisticated personalized surgical plans.

[0007] 3D surgical planning can provide more accurate anatomical information, including skeletal structure, joint morphology, and soft tissue. Through 3D reconstruction and simulation, doctors can better understand the patient's specific condition, better select surgical methods and instruments, and thus make more precise surgical plans. 3D surgical planning can help doctors determine the optimal surgical path and method in advance, better assess surgical risks and postoperative outcomes, and reduce surgical risks while improving outcomes and shortening operative time and trauma.

[0008] Rotation and translation are fundamental operations in 3D surgical planning, and 3D geometry controllers are essential tools for manipulating 3D objects. Currently, 3D surgical planning software faces numerous challenges that need to be addressed. For example, precise positioning in spinal correction, precise positioning of mouse interactions in 3D space, and more accurate measurement of Cobb angles and intervertebral distances are key challenges.

[0009] The standard geometric controller in 3D surgical planning consists of three mutually perpendicular axes to perform operations such as translation and rotation of objects. The controller is generally bound to the local coordinate system of the object being manipulated. The object geometry controller in traditional 3D surgical planning has the following problems:

[0010] (1) Rotation can only be performed by selecting the X, Y, and Z axes of the object's local coordinate system. This is equivalent to decomposing the three-dimensional rotation operation into the Euler angles of the X, Y, and Z axes to achieve the final rotation state of the object. During the operation, it is necessary to iterate and fine-tune each rotation axis one by one.

[0011] (2) Translation operations can only be performed through the X, Y, and Z axes of the local coordinate system of the selected object. This is equivalent to decomposing the translation operation of the object in three-dimensional space into translation operations in the X, Y, and Z axis directions. During the operation, it is necessary to iteratively fine-tune each direction one by one to move to the final appropriate position.

[0012] (3) Regardless of rotation or translation, the direction and magnitude of the displacement of the object in the 3D surgical simulation, the direction of the rotation axis, the position and the magnitude of the rotation angle, all need to be determined by comparing with the relative positions of other bones and organs in 3D space. Since the two-dimensional computer display can only observe the relative position between the selected object and the reference object from a specified camera direction, in order to confirm the position and direction of the selected object after movement, it is necessary to change the camera direction and modify and confirm from different angles. This process often requires repeated iterations.

[0013] In 3D surgical planning software, how to correctly and reasonably map mouse operations to the position and orientation changes of the operated object in 3D is crucial to the overall UI design and user experience, and further affects the accuracy and practicality of the overall surgical planning operation. Traditional 3D object geometry controllers cannot solve the above problems well, so it is often necessary to use three view windows in the coronal, sagittal, and axial directions at the same time, providing observations in different directions through the linkage of three views, and rotating and translating the object in three directions. The linkage of three views consumes three times the computer resources and is not intuitive in operation. The user needs to observe the other two views while operating one view to obtain position reference information. It often requires repeated iteration and fine-tuning in three directions to determine the final position and orientation of the controlled object.

[0014] Therefore, further improvements are made to the above problems.

[0015] Summary of the Invention

[0016] The main purpose of the present invention is to provide an adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction, which is committed to optimizing operations such as translation and rotation of 3D objects in three-dimensional space, expanding the application of 3D modeling technology in medical imaging and surgical planning, and solving the problem of precise positioning and operation through mouse interaction in three-dimensional space.

[0017] To achieve the above objectives, the present invention provides an adjustable geometry controller for three-dimensional surgical planning based on multi-planar reconstruction, including a three-dimensional MPR geometry controller. The three-dimensional MPR geometry controller includes three mutually perpendicular planes, each of which includes four edges and four endpoints circumscribing a DICOM image. By dragging the edges of a plane (using a mouse), the plane is displaced in the direction of its normal vector. By dragging the endpoints (using a mouse), the area of ​​the plane is changed. After selecting a rotation axis, the edge of the plane containing the rotation axis and perpendicular to the rotation axis is dragged to adjust the normal vector direction of the plane and the plane orthogonal to the plane in a coordinated manner.

[0018] In the initial state, three orthogonal planes parallel to the coronal, sagittal, and axial planes are used as the initial adjustment planes. The intersection of the three planes is the center control point. The center control point is provided with a direction vector with an arrow parallel to the three planes. At the center control point, a straight line parallel to the normal vectors of each plane is drawn through the center point, which is the center adjustment axis. The center adjustment axis of the 3D MPR geometry controller is adjusted by the translation and rotation of the plane, and can be used as the rotation axis and translation direction axis for 3D object operations.

[0019] In terms of operational interaction, the 3D MPR geometric controller is used to implement operations including translation and rotation of objects.

[0020] During measurement, point selection measurement and plane measurement are realized through the three-dimensional MPR geometry controller.

[0021] As a further preferred technical solution of the above technical solution, for operation interaction:

[0022] First, the position and direction of the controller's translation and rotation axes are adjusted by translating and rotating the MPR plane. The 2D cross-section of the DICOM image is used by the MPR plane to assist in confirming and calibrating the translation direction and rotation axis direction of the manipulated object.

[0023] When you need to translate an object, select the target object, then hold down the preset key (preferably the Shift key) and drag the translation axis or the corresponding plane border with the mouse to achieve the translation function of the object;

[0024] When an object needs to be rotated, first click to select the rotation axis, hold down the preset key (preferably the Shift key) and use the mouse to drag the edge of the plane where the rotation axis is located and perpendicular to the rotation axis to adjust the rotation angle. The closer to the center point, the greater the rotation amplitude, and the farther from the center point, the smaller the rotation amplitude (which facilitates precise control by the doctor), thereby realizing the rotation function of the object.

[0025] As a further preferred technical solution of the above technical solution, a reference plane projection method is used to implement specific operations of the mouse on the MPR plane, wherein:

[0026] The 3D MPR geometry controller is composed of the DICOM bounding box B(x min , x max ,y min ,y max , z min , z max ), the controller center point O and the normal vectors (n1, n2, n3) of the three orthogonal planes are determined. Each MPR section is determined by the normal vector n, viewX, viewUP, and the plane vertices (p0, p1, p2) are calculated based on the tangent points of the plane and the bounding box. The rotation axis and translation axis of the plane are determined by the two intersection points (t0, t1) of the center point and the bounding box B in the direction of the normal vector;

[0027] In the initial state, the normal vectors of the three planes are set to the (x, y, z) coordinate axes of the bounding box by default. When the MPR plane is transformed, the new tangent points of the plane and the bounding box are calculated based on the transformed center point O and the plane vectors n, viewX, and viewUP, and the plane vertices (p0, p1, p2) and intersection point (t0, t1) are updated.

[0028] As a further preferred technical solution of the above technical solution, under the translation operation, the mouse selects the border of the MPR plane, and the normal vector of the plane passing through the center point is used as the translation axis. The selection of the MPR plane border is determined by detecting whether the projection of the world coordinate point clicked by the mouse along the camera direction (dop) intersects with the border of the MPR plane. At the same time, it is necessary to consider the spatial relationship between the bounding box and the MPR plane border, and calculate the intersection and distance of the mouse world coordinate point along the camera direction in the six planes of the DICOM bounding box. When the border of the MPR plane intersects with the projection of the mouse world coordinate point along the camera direction and the intersection distance is less than the intersection distance of all the six planes of the bounding box, the plane is considered to be selected, and its intersection point is set as the initial coordinate point pw0 of the mouse operation;

[0029] During the rotation operation, directly calculate whether the projection of the world coordinate point clicked by the mouse along the camera direction intersects with the rotation axis vertices t0 and t1 and the intersection distance L. At the same time, calculate the intersection and distance of the mouse world coordinate point along the camera direction in the six planes of the DICOM bounding box. When the intersection distance L is less than the intersection distance of all six planes of the bounding box, the rotation axis is considered to be selected;

[0030] Assume that the initial world coordinate point when the mouse is pressed is pw0, and the world coordinate point after the mouse is moved is pw1. Regardless of translation or rotation operation, you first need to determine a suitable reference plane and project points pw0 and pw1 onto the reference plane according to the camera projection direction. In the translation operation, the translation vector is obtained based on the relationship between the projection point and the translation axis. In the rotation operation, the rotation angle is obtained based on the positional relationship between the projection point and the rotation axis.

[0031] As a further preferred technical solution of the above technical solution, in the translation operation, when the mouse moves, the initial world coordinate point pw0 and the current world coordinate point pw1 are projected onto the reference plane according to the camera projection direction dop, and the obtained projection points pw0' and pw1' are used to determine the size of the translation vector by the inner product of the displacement vector pw0'-pw1' and the translation axis vector;

[0032] In the translation operation, the center point O of the MPR geometry controller is moved according to the translation vector, and the plane vertices (p0, p1, p2) and the rotation axis vertices (t0, t1) of each MPR plane are updated;

[0033] During the rotation operation, the projection plane is selected as a plane with the rotation axis as the normal vector, and the center point of the plane is the intersection point O of the perpendicular line from the mouse world coordinate point pw0 to the rotation axis. t Sure;

[0034] During the rotation operation, the initial world coordinate point pw0 and the current world coordinate point pw1 are projected onto the reference plane according to the camera projection direction dop, and the projection points pw0' and pw1' are obtained. The rotation angle is given by the vector u = pw1-O t and v = pw0-O t Through the inverse cosine function:

[0035] Solve to obtain;

[0036] Given the rotation axis normal vector n = (a, b, c) and the rotation angle θ, the transformation of any spatial point by the rotation operation can be performed by the following steps:

[0037] Translation operation:

[0038] Assuming the origin is (x0, y0, z0), the formula for translating a point (x, y, z) to a new coordinate (x', y', z') is: x' = x - x0 y' = y - y0 z' = z - z0

[0039] Rotation operation:

[0040] The rotation matrix for rotating around the normal vector (a, b, c) by an angle θ can be expressed as:

[0041] Reverse translation operation:

[0042] The formula for translating the coordinates (x', y', z') to the original position (x", y", z") is: x"=x'+x0 y"=y'+y0 z"=z'+z0

[0043] During the rotation operation, the normal vectors of each plane of the MPR geometry controller are transformed according to the rotation operation, and then the plane vertices (p0, p1, p2) and the rotation axis vertices (t0, t1) of each MPR plane are updated.

[0044] As a further preferred technical solution of the above technical solution, for the point selection measurement method of the controller, the point is selected by adjusting the center point position of the three-dimensional MPR geometric controller, and the center point of the intersection of the three planes is directly marked as the endpoint and adjusted.

[0045] As a further preferred technical solution of the above technical solution, for the plane measurement method, the three-dimensional MPR geometry controller inserts an adjustment plane in any direction and marks it.

[0046] To achieve the above objectives, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the program, the adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction is implemented.

[0047] To achieve the above objectives, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] FIG1 shows an adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction.

[0049] Figure 2 shows the actual operation interface of the geometry controller.

[0050] FIG3 shows the geometry controller switching from planar mode to mesh mode.

[0051] FIG4 shows the geometry controller in an enlarged state, where precise control can also be achieved by operating the adjustment axis.

[0052] Figure 5 shows a scene where an object is moved via a geometry controller.

[0053] Figure 6 shows a scene where an object is rotated using a geometry controller.

[0054] FIG7 shows a standard MPR three-view diagram.

[0055] FIG8 shows the structure of the MPR plane in the MPR geometry controller.

[0056] FIG9 illustrates the translation operation of the MPR geometry controller. DETAILED DESCRIPTION

[0057] The following description is intended to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are for illustrative purposes only, and those skilled in the art will readily appreciate other obvious variations. The basic principles of the present invention defined in the following description may be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the present invention.

[0058] In the preferred embodiments of the present invention, those skilled in the art should note that the objects and the like involved in the present invention may be considered as prior art.

[0059] Preferred embodiment.

[0060] Multi-Planar Reconstruction (MPR) is a medical image processing technique that converts three-dimensional medical image data into two-dimensional images displayed on different planes. Commonly used in imaging modalities such as CT and MRI, MPR can present images in multiple planes, including the coronal, sagittal, and axial planes, by selecting different reconstruction algorithms. This improves doctors' ability to observe and diagnose a patient's internal tissue structures.

[0061] The MPR plane is generated by slicing and reconstructing three-dimensional medical imaging data. Different slicing planes can be selected according to the doctor's needs to provide more comprehensive and detailed anatomical information.

[0062] The MPR plane can provide plane reference and positioning in a specific direction, helping doctors to more accurately evaluate the three-dimensional structure of the spine, assisting doctors in surgical navigation, implant selection and surgical path planning to minimize surgical risks and improve surgical accuracy and safety.

[0063] This solution introduces the MPR geometry controller into three-dimensional surgical planning to control three-dimensional objects.

[0064] As shown in Figures 1-9, the present invention discloses an adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction, including a three-dimensional MPR geometric controller. The three-dimensional MPR geometric controller includes three mutually perpendicular planes, each plane including four edges and four endpoints circumscribing a DICOM image. The plane is displaced in the direction of the plane normal vector by dragging the edge of the plane (using a mouse), and the area of ​​the plane is changed by dragging the endpoint (using a mouse). After selecting the rotation axis, the edge of the plane where the rotation axis is located and perpendicular to the rotation axis is dragged, thereby adjusting the normal vector direction of the plane and the plane orthogonal to the plane in a linked manner (the plane can adjust the opacity or switch to a grid mode to reduce occlusion while still providing a reference, making it convenient for doctors to observe the three-dimensional cross-section in real time and directly adjust the target).

[0065] In the initial state, three orthogonal planes parallel to the coronal, sagittal, and axial planes are used as the initial adjustment planes. The intersection of the three planes is the center control point. The center control point is provided with a direction vector with an arrow parallel to the three planes (to facilitate the doctor to have the ability to make quick and accurate adjustments under magnification). In addition, at the center control point, a straight line parallel to the normal vectors of each plane is drawn through the center point, which is the center adjustment axis. The center adjustment axis of the 3D MPR geometry controller is adjusted by the translation and rotation of the plane, and can be used as the rotation axis and translation direction axis for 3D object operations.

[0066] In terms of operational interaction, the 3D MPR geometric controller is used to implement operations including translation and rotation of objects.

[0067] During measurement, point selection measurement and plane measurement are realized through the three-dimensional MPR geometry controller.

[0068] Specifically, for operation interactions:

[0069] First, the position and direction of the controller's translation and rotation axes are adjusted by translating and rotating the MPR plane. The 2D cross-section of the DICOM image is used by the MPR plane to assist in confirming and calibrating the translation direction and rotation axis direction of the manipulated object.

[0070] When you need to translate an object, select the target object, then hold down the preset key (preferably the Shift key) and drag the translation axis or the corresponding plane border with the mouse to achieve the translation function of the object;

[0071] When an object needs to be rotated, first click to select the rotation axis, hold down the preset key (preferably the Shift key) and use the mouse to drag the edge of the plane where the rotation axis is located and perpendicular to the rotation axis to adjust the rotation angle. The closer to the center point, the greater the rotation amplitude, and the farther from the center point, the smaller the rotation amplitude (which facilitates precise control by the doctor), thereby realizing the rotation function of the object.

[0072] In a specific implementation, the operational interaction of the three-dimensional MPR geometry controller requires the correct and reasonable mapping of the mouse operation to the position and direction changes of the operated object in three dimensions to achieve an accurate user operation experience. The mouse operation of the MPR geometry controller includes the selection operation of the MPR plane and the rotation axis and the drag and shift operation of the mouse after selection. In the traditional MPR algorithm, it is necessary to display the two-dimensional views of the three MPR planes at the same time, and adjust the position and direction of the other two orthogonally linked MPR planes by controlling the coordinate axis in the two-dimensional view, as shown in Figure 7. Manipulating the MPR plane through a two-dimensional view will bring about problems such as unintuitive operation and poor user interaction experience. The present invention adopts a reference surface projection method to realize the specific operation of the mouse on the MPR plane.

[0073] The 3D MPR geometry controller is composed of the DICOM bounding box B(x min , x max ,y min ,y max , z min , z max ), the controller center point O and the normal vectors (n1, n2, n3) of the three orthogonal planes are determined. Each MPR section is determined by the normal vector n, viewX, viewUP, and the plane vertices (p0, p1, p2) are calculated based on the tangent points of the plane and the bounding box. The rotation axis and translation axis of the plane are determined by the two intersection points (t0, t1) of the center point and the bounding box B in the direction of the normal vector;

[0074] In the initial state, the normal vectors of the three planes are set to the (x, y, z) coordinate axes of the bounding box by default. When the MPR plane is transformed, the new tangent points of the plane and the bounding box are calculated based on the transformed center point O and the plane vectors n, viewX, and viewUP, and the plane vertices (p0, p1, p2) and intersection point (t0, t1) are updated.

[0075] During a translation operation, the mouse selects the bounding box of the MPR plane, using the normal vector of the plane through the center point as the translation axis. The selection of the MPR plane bounding box is determined by detecting whether the projection of the world coordinate point clicked by the mouse along the camera direction intersects with the bounding box of the MPR plane. At the same time, the spatial relationship between the bounding box and the MPR plane bounding box needs to be considered, and the intersection and distance of the mouse world coordinate point along the camera direction with the six planes of the DICOM bounding box are calculated. When the bounding box of the MPR plane intersects with the projection of the mouse world coordinate point along the camera direction and the intersection distance is less than the intersection distance of all six planes of the bounding box, the plane is considered selected, and its intersection point is set as the initial coordinate point for the mouse operation.

[0076] During the rotation operation, directly calculate whether the projection of the world coordinate point clicked by the mouse along the camera direction intersects with the rotation axis vertices t0 and t1 and the intersection distance L. At the same time, calculate the intersection and distance of the mouse world coordinate point along the camera direction in the six planes of the DICOM bounding box. When the intersection distance is less than the intersection distance of all six planes of the bounding box L, the rotation axis is considered to be selected.

[0077] Assume that the initial world coordinate point when the mouse is pressed is pw0, and the world coordinate point after the mouse is moved is pw1. Regardless of translation or rotation operation, you first need to determine a suitable reference plane and project points pw0 and pw1 onto the reference plane according to the camera projection direction. In the translation operation, the translation vector is obtained based on the relationship between the projection point and the translation axis. In the rotation operation, the rotation angle is obtained based on the positional relationship between the projection point and the rotation axis.

[0078] The reference projection plane in the translation operation is selected as one of the six faces of the DICOM bounding box. The specific method is as follows:

[0079] 1. First, get the direction of the camera dop and the initial world coordinate point where the mouse is pressed as pw0.

[0080] 2. Then, traverse the six faces and filter out the planes that meet the conditions through a series of criteria as the target planes for transformation.

[0081] 3. Screening criteria include:

[0082] 3.1 If the dot product of the face normal and the camera projection direction is less than the given threshold EPSILON, the face is perpendicular to or deviates from the camera direction and is removed.

[0083] 3.2 If the dot product of the face normal and the slice normal is close to 1, the face is removed to ensure that the selected face and the slice are not in the same plane.

[0084] 3.3 If the above screening conditions are met, the face index number and the dot product value of the face normal and the camera projection direction are added to the candidate list.

[0085] 4. If the candidate list length is greater than or equal to 2, sort the candidate lists in descending order based on the inner product of the plane normal vector and the camera projection vector. Select the first face in the candidate list that is also the face closest to the initial world coordinate point as the reference plane.

[0086] During the translation operation, as shown in Figure 9, when the mouse moves, the initial world coordinate point pw0 and the current world coordinate point pw1 are projected onto the reference plane according to the camera projection direction dop. The resulting projection points pw0' and pw1' are used to determine the size of the translation vector by the inner product of the displacement vector pw0'-pw1' and the translation axis.

[0087] During the translation operation, the center point O of the MPR geometry controller is moved according to the translation vector, and the plane vertices (p0, p1, p2) and the rotation axis vertices (t0, t1) of each MPR plane are updated.

[0088] During the rotation operation, the projection plane is selected as a plane with the rotation axis as the normal vector, and the center point of the plane is the intersection point O of the perpendicular line from the mouse world coordinate point pw0 to the rotation axis. t Sure;

[0089] During the rotation operation, the initial world coordinate point pw0 and the current world coordinate point pw1 are projected onto the reference plane according to the camera projection direction dop, and the projection points pw0' and pw1' are obtained. The rotation angle is given by the vector u = pw1-O t and v = pw0-O t Through the inverse cosine function:

[0090] Solve to obtain;

[0091] Given the rotation axis normal vector n = (a, b, c) and the rotation angle θ, the transformation of any spatial point by the rotation operation can be performed by the following steps:

[0092] 1. Translation operation:

[0093] Assuming the origin is (x0, y0, z0), the formula for translating a point (x, y, z) to a new coordinate (x', y', z') is: x' = x - x0 y' = y - y0 z' = z - z0

[0094] 2. Rotation operation:

[0095] The rotation matrix for rotating around the normal vector (a, b, c) by an angle θ can be expressed as:

[0096] 3. Reverse translation operation:

[0097] The formula for translating the coordinates (x', y', z') to the original position (x", y", z") is: x"=x'+x0 y"=y'+y0 z"=z'+z0

[0098] During the rotation operation, the normal vectors of each plane of the MPR geometry controller are transformed according to the rotation operation, and then the plane vertices (p0, p1, p2) and the rotation axis vertices (t0, t1) of each MPR plane are updated.

[0099] More specifically, for the point selection measurement method of the controller, the point is selected by adjusting the center point position of the three-dimensional MPR geometric controller, and the center point of the intersection of the three planes is directly marked as the endpoint and adjusted.

[0100] Furthermore, for the plane measurement method, the three-dimensional MPR geometry controller inserts an adjustment plane in any direction and marks it. A parallel and close adjustment plane is inserted at the first position required by the object (for example, the lower end plate of the vertebra), marked as the lower end plate, and a parallel and close adjustment plane is inserted at the second position required by the object (the upper end plate of the next vertebra), marked as the upper end plate. The average distance between the two adjustment planes is the distance between the two positions (that is, the intervertebral distance between the two vertebrae). The measurement of angles such as the Cobb angle can also be completed using a plane. After determining the target vertebra and setting the corresponding adjustment plane, the angle between the projections of the two planes on the coronal plane of the human body is the Cobb angle.

[0101] Preferably, all measurement data in this solution can be tracked in real time. The geometric controller of this solution allows the inserted measurement points and measurement planes to be linked and bound to the target object. If the measured object undergoes displacement, rotation, etc., the measurement points and measurement planes bound to it will also undergo linkage operations of displacement and rotation in the same manner, and the corresponding measurement values ​​will also change in real time.

[0102] This scheme can be used not only for surgical planning of STL models, but also for volume reconstruction models based on volume rendering.

[0103] For the attached pictures:

[0104] Figure 1 shows an adjustable geometric controller for 3D surgical planning based on multi-planar reconstruction. The controller uses the plane mode by default, which consists of three mutually perpendicular planes, control axes, and control points.

[0105] FIG2 shows the actual operation interface of the geometric controller, in which the STL, volume rendering model and controller can be adjusted to complete the relevant operations of surgical planning.

[0106] Figure 3 shows the grid mode of the geometry controller. The geometry controller has three main modes: grid, plane, and transparent. The opacity of each plane can be adjusted, and users can choose according to their needs.

[0107] Figure 4 shows the geometry controller in a zoomed-in state. The line segments generated by the intersection of each two planes in this geometry controller are the adjustment axes. Even in a zoomed-in state, adjusting the axes allows for precise control of the target object. In this solution, before moving or rotating an object, the geometry controller must be adjusted to determine the object's translation direction or rotation axis. Once this is determined, subsequent operations can be performed.

[0108] Figure 5 shows a scene of moving an object using a geometry controller, which specifically includes the following steps:

[0109] First, determine the target direction for the object to move. Next, move or rotate the planes of the geometry controller so that one of them is parallel to the target direction. Next, select the target object to be moved. Holding down the Shift key, drag the plane or its axis perpendicular to the target direction in the geometry controller to move the object in the target direction. Once the desired position is reached, release the mouse button to complete the object's move.

[0110] Figure 6 shows a scene of rotating an object using a geometry controller, which specifically includes the following steps:

[0111] First, confirm the target rotation direction and rotation axis of the object. Secondly, move or rotate the plane of the geometry controller so that one of the adjustment axes is consistent with the expected rotation axis position and angle. At this time, the adjustment axis is the target rotation axis. Then, select the target object to be rotated, click the target rotation axis, hold down the "shift" key, and drag the plane or adjustment axis of the geometry controller through the target rotation axis to rotate the target object around the target rotation axis. After reaching the expected position, release the mouse to complete the object rotation operation.

[0112] Figure 7 shows the MPR plane manipulation scheme for the standard three-view MPR. The position and orientation of the other two MPR planes are adjusted by the control points of the coordinate axes in the 2D view. Manipulating MPR planes in 2D views can lead to unintuitive operations and poor user interaction experience in 3D surgical planning.

[0113] Figure 8 shows the structure of the MPR plane in the MPR geometry controller, including boundary points, coordinate vectors, and coordinate axis vertices. Based on the transformed center point O and the plane vectors n, viewX, and viewUP, the new tangent points between the plane and the bounding box are calculated, and the plane vertices (p0, p1, p2) and intersection point (t0, t1) are updated.

[0114] Figure 9 illustrates the MPR geometry controller's algorithm for 3D mouse translation using a dynamically selected projection plane. First, the mouse selects the translation axis. The mouse's reference projection plane is dynamically calculated and selected. The displacement vector on the reference plane is calculated from the projection points on the reference plane before and after the mouse movement. The translation vector is then multiplied by the dot product with the translation axis vector to update the selected MPR plane to its new position.

[0115] For the present invention:

[0116] 1. In this approach, the object's translation direction or rotation axis is predefined by adjusting the MPR geometry controller. Because translation and rotation operations are performed based on this fixed axis, mouse interactions are converted into single-dimensional operations, such as translation along the axis or rotation along the axis. There's no need to decompose translation and rotation operations into their directional components in rectangular coordinates. As a result, 3D operations based on the MPR geometry controller are simple, clear, and efficient, performed in one step, eliminating the iterative adjustment required by traditional geometry controllers for each direction.

[0117] 2. In this solution, the rotation axis or translation direction of the object operation is obtained through the intersection line between the MPR planes. Since each MPR plane is a cross-section of the overall 3D space, the MPR plane provides an effective spatial position reference plane in three-dimensional space, which can be used intuitively to determine the position relationship between 3D space objects in different direction cross-sections, such as distance, alignment, angle, etc. MPR contains three planes in orthogonal directions in 3D space, so this solution only needs a single view window to accurately and effectively provide position references in different directions. Similar solutions require three view windows to be opened at the same time to provide viewing angles in different directions. At the same time, due to the lack of plane reference, the positioning of the object can only be adjusted and operated iteratively by observing in different directions.

[0118] 3. Objects in medical 3D surgical planning are based on STL models derived from DICOM medical images. This solution uses MPR sections generated from raw DICOM data to perform 3D positioning based on clinically important reference planes. It also calibrates and confirms the movement direction and rotation point of objects in 3D planning, achieving 2D and 3D operational reference and association through a multi-data, multi-modal approach.

[0119] In summary, this solution uses the MPR plane to provide 2D image references and 3D spatial position references that are medically significant in 3D surgical planning, making up for the deficiency of traditional geometric controllers that lack references and can only rely on observation for spatial calibration. At the same time, the reference provided by the adjusted MPR plane is used to pre-plan the translation direction or rotation angle and direction of the object before operation, avoiding the traditional geometric controller's inability to adjust the position and direction of its own coordinate axis, which can only be operated in the rectangular coordinate direction and repeatedly undergo an iterative process of 3D spatial calibration. Therefore, the MPR adjustable geometric controller proposed in this solution can provide more accurate and efficient operation of the controlled object in 3D surgical planning, which is an important development and innovation in 3D geometric controller technology.

[0120] The present invention also discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction is implemented.

[0121] The present invention also discloses a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction is implemented.

[0122] It is worth mentioning that the technical features such as objects involved in the patent application of this invention should be regarded as prior art. The specific structure, working principle and possible control method and spatial layout method of these technical features can be selected by conventional means in the field and should not be regarded as the inventive point of the patent of this invention. The patent of this invention will not be further elaborated.

[0123] For those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned embodiments, or to make equivalent replacements for some of the technical features therein. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction, characterized in that: The system comprises a three-dimensional MPR geometry controller, which comprises three mutually perpendicular planes, each of which comprises four edges and four endpoints circumscribing the DICOM image. The plane is displaced in the direction of the plane normal vector by dragging the edge of the plane, and the area of ​​the plane is changed by dragging the endpoint. After selecting the rotation axis, the edge of the plane where the rotation axis is located and perpendicular to the rotation axis is dragged to adjust the normal vector direction of the plane and the plane orthogonal to the plane in a linked manner, wherein: In the initial state, three orthogonal planes parallel to the coronal, sagittal, and axial planes are used as the initial adjustment planes. The intersection of the three planes is the center control point. The center control point is provided with a direction vector with an arrow parallel to the three planes. At the center control point, a straight line parallel to the normal vectors of each plane is drawn through the center point, which is the center adjustment axis. The center adjustment axis of the 3D MPR geometry controller is adjusted by the translation and rotation of the plane, and can be used as the rotation axis and translation direction axis for 3D object operations. In terms of operational interaction, the 3D MPR geometric controller is used to implement operations including translation and rotation of objects. During measurement, point selection measurement and plane measurement are realized through the three-dimensional MPR geometry controller.

2. The adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction according to claim 1, characterized in that: For operational interactions: First, the position and direction of the controller's translation and rotation axes are adjusted by translating and rotating the MPR plane. The 2D cross-section of the DICOM image is used by the MPR plane to assist in confirming and calibrating the translation direction and rotation axis direction of the manipulated object. When you need to translate an object, select the target object, then hold down the preset button and drag the translation axis or the corresponding plane border with the mouse to achieve the translation function of the object; When you need to rotate an object, first click to select the rotation axis. Press and hold the preset button while dragging the edge of the plane where the rotation axis is located that is perpendicular to the rotation axis with the mouse to adjust the rotation angle. The closer to the center point, the greater the rotation amplitude, and the farther from the center point, the smaller the rotation amplitude, thus realizing the rotation function of the object.

3. The adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction according to claim 2, characterized in that: The reference plane projection method is used to realize the specific operation of the mouse on the MPR plane, where: The 3D MPR geometry controller is composed of the DICOM bounding box B(x min , x max ,y min ,y max , z min , z max ), the controller center point O and the normal vectors (n1, n2, n3) of the three orthogonal planes are determined. Each MPR section is determined by the normal vector n, viewX, viewUP, and the plane vertices (p0, p1, p2) are calculated based on the tangent points of the plane and the bounding box. The rotation axis and translation axis of the plane are determined by the two intersection points (t0, t1) of the center point and the bounding box B in the direction of the normal vector; In the initial state, the normal vectors of the three planes are set to the (x, y, z) coordinate axes of the bounding box by default. When the MPR plane is transformed, the new tangent points of the plane and the bounding box are calculated based on the transformed center point O and the plane vectors n, viewX, and viewUP, and the plane vertices (p0, p1, p2) and intersection points (t0, t1) are updated.

4. The adjustable geometry controller for three-dimensional surgical planning based on multi-planar reconstruction according to claim 3, characterized in that: During the translation operation, the mouse selects the border of the MPR plane and uses the normal vector of the plane through the center point as the translation axis. The selection of the MPR plane border is determined by detecting whether the projection of the world coordinate point clicked by the mouse along the camera direction intersects with the border of the MPR plane. At the same time, the spatial relationship between the bounding box and the MPR plane border needs to be considered. The intersection and distance of the mouse world coordinate point along the camera direction in the six planes of the DICOM bounding box are calculated. When the border of the MPR plane intersects with the projection of the mouse world coordinate point along the camera direction and the intersection distance is less than the intersection distance of all the six planes of the bounding box, the plane is considered to be selected, and its intersection point is set as the initial coordinate point pw0 of the mouse operation; During the rotation operation, directly calculate whether the projection of the world coordinate point clicked by the mouse along the camera direction intersects with the rotation axis vertices t0 and t1 and the intersection distance L. At the same time, calculate the intersection and distance of the mouse world coordinate point along the camera direction in the six planes of the DICOM bounding box. When the intersection distance L is less than the intersection distance of all six planes of the bounding box, the rotation axis is considered to be selected; Assume that the initial world coordinate point when the mouse is pressed is pw0, and the world coordinate point after the mouse is moved is pw1. Regardless of translation or rotation operation, you first need to determine a suitable reference plane and project points pw0 and pw1 onto the reference plane according to the camera projection direction. In the translation operation, the translation vector is obtained based on the relationship between the projection point and the translation axis. In the rotation operation, the rotation angle is obtained based on the positional relationship between the projection point and the rotation axis.

5. The adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction according to claim 4, characterized in that: During translation, when the mouse moves, the initial world coordinate point pw0 and the current world coordinate point pw1 are projected onto the reference plane according to the camera projection direction dop. The resulting projection points pw0' and pw1' are used to determine the size of the translation vector by the inner product of the displacement vector pw0'-pw1' and the translation axis. In the translation operation, the center point O of the MPR geometry controller is moved according to the translation vector, and the plane vertices (p0, p1, p2) and the rotation axis vertices (t0, t1) of each MPR plane are updated; During the rotation operation, the projection plane is selected as a plane with the rotation axis as the normal vector, and the center point of the plane is the intersection point O of the perpendicular line from the mouse world coordinate point pw0 to the rotation axis. t Sure; During the rotation operation, the initial world coordinate point pw0 and the current world coordinate point pw1 are projected onto the reference plane according to the camera projection direction dop, and the projection points pw0' and pw1' are obtained. The rotation angle is given by the vector u = pw1-O t and v = pw0-O t Through the inverse cosine function: Solve to obtain; Given the rotation axis normal vector n = (a, b, c) and the rotation angle θ, the rotation operation transforms any spatial point by the following steps: Translation operation: Assuming the origin is (x0, y0, z0), the formula for translating a point (x, y, z) to a new coordinate (x', y', z') is: x'=x-x0 y'=y-y0 z'=z-z0; Rotation operation: The rotation matrix for rotating around the normal vector (a, b, c) by an angle θ can be expressed as: Reverse translation operation: The formula for translating the coordinates (x', y', z') to their original position (x", y", z") is: x"=x'+x0 y"=y'+y0 z"=z'+z0 During the rotation operation, the normal vectors of each plane of the MPR geometry controller are transformed according to the rotation operation, and then the plane vertices (p0, p1, p2) and the rotation axis vertices (t0, t1) of each MPR plane are updated.

6. The adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction according to claim 5, characterized in that: For the point selection measurement method of the controller, the point is selected through the center point position of the three-dimensional MPR geometric controller, and the center point of the intersection of the three planes is directly marked as the endpoint and adjusted.

7. The adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction according to claim 6, characterized in that: For plane measurement, the 3D MPR geometry controller inserts adjustment planes in any direction and marks them.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction as described in any one of claims 1 to 7 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction as claimed in any one of claims 1 to 7 is implemented.

Images