An Adjustable Geometric Controller for Three-Dimensional Surgical Planning Based on Multi-Plane Reconstruction

Abstract

The present invention discloses an adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction, including a three-dimensional MPR geometric controller, wherein the three-dimensional MPR geometric controller includes three mutually perpendicular planes, each plane includes four edges and four endpoints of an external DICOM image, and the plane is displaced in the direction of the plane normal vector by dragging the edge of the plane, and the area size of the plane is changed by dragging the endpoints. After selecting the rotation axis, the edge of the plane where the rotation axis is located and perpendicular to the rotation axis is dragged, thereby adjusting the normal vector direction of the plane and the orthogonal constraint plane of the plane in linkage. The adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction disclosed by the present invention is dedicated to optimizing operations such as translation and rotation of 3D objects in three-dimensional space, expanding the application of 3D modeling technology in medical imaging and surgical planning, and solving the problem of accurate positioning and operation through mouse interaction in three-dimensional space.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional surgical planning, and particularly relates to an adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction. Background Art

[0002] In orthopedic medicine, doctors usually need to observe the patient's bone structure through imaging to accurately diagnose and formulate treatment plans. Although traditional two-dimensional images can provide certain information, there may be some limitations when observing complex bone structures, such as occlusion, overlap, and unclear images.

[0003] With the continuous progress of medical imaging technology, doctors can obtain more accurate and detailed patient bone data, and use computer software to perform precise three-dimensional volume reconstruction of the bones. The orthopedic three-dimensional volume reconstruction technology generates individualized bone models and volume calculation results according to the specific situation of the patient, which can effectively reduce human errors and help doctors formulate more accurate and refined personalized surgical planning solutions.

[0004] Three-dimensional surgical planning can provide more accurate anatomical information, including bone structure, joint morphology, soft tissues, etc. Through three-dimensional reconstruction and simulation operations, doctors can better understand the specific situation of the patient, better select surgical methods and surgical instruments, and thus make more precise surgical plans. Three-dimensional surgical planning can help doctors determine the best surgical path and operation method in advance, better evaluate the surgical risk and postoperative effect, reduce the surgical risk while improving the surgical effect, and reduce the surgical time and trauma.

[0005] Rotation and translation are basic operations in three-dimensional surgical planning, and 3D geometric controllers are basic tools for realizing 3D object operations. At present, there are many problems that need to be solved urgently in three-dimensional surgical planning software. For example, how to accurately position in spinal orthopedics, how to accurately position mouse interaction operations in three-dimensional space, and how to more accurately measure Cobb angles, vertebral space distances, etc.

[0006] The standard geometric controller in 3D surgical planning consists of three mutually perpendicular axes to perform operations such as translation and rotation on objects. The controller is generally bound to the local coordinate system of the object to be operated. The object geometric controller in traditional 3D surgical planning has the following problems:

[0007] (1) It can only perform rotation operations by selecting the X, Y, and Z axes of the local coordinate system of the object, which is equivalent to decomposing the rotation operation in three-dimensional space into Euler angles of the X, Y, and Z axes to reach the final rotation state of the object. When operating, it is necessary to iteratively fine-tune each rotation axis one by one.

[0008] (2) Translation operations can only be performed by selecting the X, Y, and Z axes of the local coordinate system of the object, which is equivalent to decomposing the translation operation of the object in three-dimensional space into translation operations in the X, Y, and Z axis directions. During the operation, it is necessary to iteratively fine-tune each direction one by one to move to the final appropriate position.

[0009] (3) Regardless of rotation and translation operations, the direction and magnitude of the displacement of the object in 3D surgical simulation, the direction of the rotation axis, the position, and the magnitude of the rotation angle all need to be determined by referring to and comparing with the relative positions of other bones and organs in 3D space. Since the 2D computer display can only observe the relative positions between the selected object and the reference object from a specified camera direction, to confirm the position and direction of the selected object after movement, it is necessary to change the camera direction and modify and confirm from different angles, and this process often requires repeated iteration.

[0010] In 3D surgical planning software, how to correctly and reasonably map mouse operations to the changes in the position and direction of the object being operated in three dimensions is crucial for the overall UI design and the user experience. Furthermore, it affects the accuracy and practicality of the overall surgical planning operation. Traditional 3D object geometric controllers cannot well solve the above problems. Therefore, it is often necessary to use the view windows in three directions of the coronal plane, sagittal plane, and axial plane simultaneously, and provide observations in different directions through the linkage of the three views, and perform rotation and translation operations on the object in the three directions respectively. The way of the linkage of the three views consumes 3 times the computer resources, is not intuitive in operation, and the user needs to observe the other two views while operating one view to obtain position reference information, and often also needs to repeatedly iterate and fine-tune in the three directions to determine the final position and direction of the controlled object.

[0011] Therefore, in view of the above problems, further improvements are made. Summary of the Invention

[0012] The main purpose of the present invention is to provide an adjustable geometric controller for 3D surgical planning based on multi-planar reconstruction, which is committed to optimizing operations such as translation and rotation of 3D objects in three-dimensional space, expanding the application of 3D modeling technology in medical imaging and surgical planning, and solving the problem of precise positioning and operation through mouse interaction in three-dimensional space.

[0013] To achieve the above objectives, the present invention provides an adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction, including a three-dimensional MPR geometric controller. The three-dimensional MPR geometric controller includes three mutually perpendicular planes, each plane includes four edges and four endpoints of the circumscribed DICOM image, and the plane is displaced in the direction of the plane normal vector by dragging the edge of the plane (using a mouse), and the area size of the plane is changed by dragging the endpoint (using a mouse). After selecting the rotation axis, the edge of the plane where the rotation axis is located and perpendicular to the rotation axis is dragged, so as to adjust the normal vector direction of the plane and the orthogonal constraint plane of the plane in a linked manner, wherein:

[0014] In the initial state, three orthogonal planes parallel to the coronal plane, sagittal plane, and axial plane are used as the initial adjustment planes. The intersection of the three planes is the center control point. The center control point is provided with a direction vector with an arrow parallel to the three planes. At the center control point, a straight line parallel to the normal vectors of each plane is drawn through the center point, which is the center adjustment axis. The center adjustment axis of the 3D MPR geometry controller is adjusted by the translation and rotation of the plane, and can be used as the rotation axis and translation direction axis of the 3D object operation.

[0015] In terms of operational interaction, the 3D MPR geometric controller is used to implement operations including translation and rotation of objects.

[0016] During measurement, point selection measurement and plane measurement are realized through the three-dimensional MPR geometry controller.

[0017] As a further preferred technical solution of the above technical solution, for operation interaction:

[0018] First, the position and direction of the controller's translation axis and rotation axis are adjusted by translating and rotating the MPR plane, and the translation direction and rotation axis direction of the controlled object are confirmed and calibrated by the 2D cross-section of the DICOM image using the MPR plane;

[0019] When you need to translate an object, select the target object, then hold down a preset key (preferably the Shift key) and drag the translation axis or the corresponding plane border with the mouse to achieve the translation function of the object;

[0020] When the object needs to be rotated, first click to select the rotation axis, hold down the preset key (preferably the Shift key) and use the mouse to drag the edge of the plane where the rotation axis is located and perpendicular to the rotation axis to adjust the rotation angle. The closer the distance to the center point is, the greater the rotation amplitude, and the farther the distance from the center point is, the smaller the rotation amplitude (which is convenient for doctors to perform precise control), thereby realizing the rotation function of the object.

[0021] As a further preferred technical solution of the above technical solution, a reference plane projection method is adopted to implement the specific operation of the mouse on the MPR plane, where:

[0022] The three-dimensional MPR geometric controller is determined by the DICOM bounding box B(x min ,x max ,y min ,y max ,z min ,z max ), the center point O of the controller, and the normal vectors (n1, n2, n3) of the three orthogonal planes. Each MPR section is determined by the normal vector n, viewX, and viewUP, and the plane vertices (p0, p1, p2) are calculated according to the outer tangent points of the plane and the bounding box. The rotation axis and translation axis of the plane are determined by the two intersection points (t0, t1) of the center point in the direction of the normal vector and the bounding box B;

[0023] In the initial state, the normal vectors of the three planes are default set to the (x, y, z) coordinate axes of the bounding box. When a transformation is made to the MPR plane, the new outer tangent points of the plane and the bounding box are calculated according to the transformed center point O and the plane vectors n, viewX, and viewUP, and the plane vertices (p0, p1, p2) and intersection points (t0, t1) are updated.

[0024] As a further preferred technical solution of the above technical solution, in the translation operation, the mouse selects the border of the MPR plane, and uses the normal vector passing through the center point of the plane as the translation axis. The selection of the MPR plane border is determined by detecting whether the projection of the world coordinate point clicked by the mouse along the camera direction (dop) intersects with the border of the MPR plane. At the same time, the spatial relationship between the bounding box and the border of the MPR plane needs to be considered, and the intersection points and distances of the mouse world coordinate point along the camera direction on the six planes of the DICOM bounding box are calculated. When the border of the MPR plane intersects with the projection of the mouse world coordinate point along the camera direction and the intersection distance is less than the intersection distances of all six planes of the bounding box, the plane is considered to be selected, and its intersection point is set as the initial coordinate point pw0 of the mouse operation;

[0025] In the rotation operation, directly calculate whether the projection of the world coordinate point clicked by the mouse along the camera direction intersects with the rotation axis vertices t0 and t1 and the intersection distance L, and at the same time calculate the intersection points and distances of the mouse world coordinate point along the camera direction on the six planes of the DICOM bounding box. When the intersection distance L is less than the intersection distances of all six planes of the bounding box, the rotation axis is considered to be selected;

[0026] Assume that the initial world coordinate point when the mouse is pressed is pw0, and the world coordinate point after movement is pw1. Whether it is a translation or rotation operation, it is first necessary to determine an appropriate reference plane, project the points pw0 and pw1 onto the reference plane in the camera projection direction. In the translation operation, the translation vector is obtained according to the mutual relationship between the projection points and the translation axis, and in the rotation operation, the rotation angle is obtained through the positional relationship between the projection points and the rotation axis.

[0027] As a further preferred technical solution of the above technical solution, in the translation operation, when the mouse moves, the initial world coordinate point pw0 and the current world coordinate point pw1 are projected onto the reference plane in the camera projection direction dop, and the obtained projection points pw′0 and pw′1 are used to determine the magnitude of the translation vector through the inner product of the displacement vector pw′0 - pw′1 and the translation axis vector;

[0028] In the translation operation, the center point O of the MPR geometric controller is moved according to the translation vector, and the plane vertices (p0, p1, p2) and the rotation axis vertices (t0, t1) of each MPR plane are updated;

[0029] In the rotation operation, the projection plane is selected as a plane with the rotation axis as the normal vector, and the center point of the plane is the intersection point O of the perpendicular line from the world coordinate point pw0 of the mouse to the rotation axis t determined;

[0030] In the rotation operation, the initial world coordinate point pw0 and the current world coordinate point pw1 are projected onto the reference plane in the camera projection direction dop, and the projection points pw′0 and pw′1 are obtained. The rotation angle is obtained from the vectors u = pw1 - O t and v = pw0 - O t through the inverse cosine function:

[0031]

[0032] solved and obtained;

[0033] Given the rotation axis normal vector n = (a, b, c) and the rotation angle θ, the transformation of any spatial point in the rotation operation can be carried out through the following steps:

[0034] Translation operation:

[0035] Assume that the origin is (x0, y0, z0), then the formula for translating the point (x, y, z) to the new coordinates (x′, y′, z′) is:

[0036] x′ = x - x0

[0037] y′ = y - y0

[0038] Z′ = z - z0;

[0039] Rotation operation:

[0040] The rotation matrix for rotating by an angle θ about the normal vector (a, b, c) can be expressed as:

[0041]

[0042] Reverse translation operation:

[0043] The formula for translating the coordinates (x′, y′, z′) to the original position (x″, y″, z″) is:

[0044] x″ = x′ + x0

[0045] y″ = y′ + y0

[0046] z″ = z′ + z0;

[0047] In the rotation operation, after transforming the normal vectors of each plane of the MPR geometric controller according to the rotation operation, the plane vertices (p0, p1, p2) and the rotation axis vertices (t0, t1) of each MPR plane are updated.

[0048] As a further preferred technical solution of the above technical solution, for the point selection measurement method of the controller, by adjusting the center point position of the three-dimensional MPR geometric controller for point selection, the center point of the intersection of the three planes is directly marked as an endpoint and adjusted.

[0049] As a further preferred technical solution of the above technical solution, for the plane measurement method, the three-dimensional MPR geometric controller is inserted into the adjustment plane in any direction and marked.

[0050] To achieve the above object, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction.

[0051] To achieve the above object, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 Shows an adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction.

[0053] Figure 2 Shows the actual operation interface of the geometric controller.

[0054] Figure 3Shows a geometric controller that switches from a planar mode to a grid mode.

[0055] Figure 4 Shows the geometric controller in a magnified situation, and precise control can also be achieved by operating the adjustment axis.

[0056] Figure 5 Shows a scene where an object is moved by the geometric controller.

[0057] Figure 6 Shows a scene where an object is rotated by the geometric controller.

[0058] Figure 7 Shows the standard MPR three-view drawings.

[0059] Figure 8 Shows the structure of the MPR plane in the MPR geometric controller.

[0060] Figure 9 Shows the translation operation of the MPR geometric controller. Detailed implementation

[0061] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments in the following description are only examples, and those skilled in the art can think of other obvious variations. The basic principles defined in the following description of the present invention can be applied to other implementation schemes, variation schemes, improvement schemes, equivalent schemes, and other technical schemes without departing from the spirit and scope of the present invention.

[0062] In the preferred embodiment of the present invention, those skilled in the art should note that the objects and the like involved in the present invention can be regarded as the prior art.

[0063] Preferred embodiment.

[0064] MPR (Multi-Planar Reconstruction) technology is a medical image processing technology that can convert three-dimensional medical image data into two-dimensional images displayed on different planes. This technology is commonly used in imaging modalities such as CT and MRI. By selecting different reconstruction algorithms, images can be presented on multiple planes such as the coronal plane, sagittal plane, and axial plane, thereby improving doctors' observation and diagnosis levels of patients' internal tissue structures.

[0065] The MPR plane is generated by slicing and reconstructing three-dimensional medical image data, and different slicing planes can be selected according to doctors' needs to provide more comprehensive and detailed anatomical information.

[0066] The MPR plane can provide plane reference and positioning in a specific direction, helping doctors to more accurately evaluate the three-dimensional structure of the spine, assist doctors in surgical navigation, implant selection and surgical path planning to minimize surgical risks and improve surgical accuracy and safety.

[0067] This scheme introduces the MPR geometry controller into three-dimensional surgical planning to control three-dimensional objects.

[0068] like Figures 1-9 As shown, the present invention discloses an adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction, including a three-dimensional MPR geometric controller, wherein the three-dimensional MPR geometric controller includes three mutually perpendicular planes, each plane includes four edges and four endpoints of the circumscribed DICOM image, and the plane is displaced in the direction of the plane normal vector by dragging the edge of the plane (using a mouse), and the area size of the plane is changed by dragging the endpoint (using a mouse), and after selecting the rotation axis, the edge of the plane where the rotation axis is located and perpendicular to the rotation axis is dragged, so as to adjust the normal vector direction of the plane and the orthogonal constraint plane of the plane in linkage (the plane can adjust the opacity, and can also be switched to a grid mode, which can reduce occlusion while still providing a reference, so that the doctor can observe the three-dimensional section in real time and directly adjust the target), wherein:

[0069] In the initial state, three orthogonal planes parallel to the coronal plane, sagittal plane, and axial plane are used as the initial adjustment planes. The intersection of the three planes is the center control point. The center control point is provided with a direction vector with an arrow parallel to the three planes (to facilitate the doctor to have the ability to make quick and accurate adjustments under magnification), and at the center control point, a straight line parallel to the normal vectors of each plane is drawn through the center point, which is the center adjustment axis. The center adjustment axis of the 3D MPR geometry controller is adjusted by the translation and rotation of the plane, and can be used as the rotation axis and translation direction axis of the 3D object operation;

[0070] In terms of operational interaction, the 3D MPR geometric controller is used to implement operations including translation and rotation of objects.

[0071] During measurement, point selection measurement and plane measurement are realized through the three-dimensional MPR geometry controller.

[0072] Specifically, for operation interactions:

[0073] First, the position and direction of the controller's translation axis and rotation axis are adjusted by translating and rotating the MPR plane, and the translation direction and rotation axis direction of the controlled object are confirmed and calibrated by the 2D cross-section of the DICOM image using the MPR plane;

[0074] When it is necessary to translate an object, select the target object, and then while holding down the preset button (preferably the Shift key), drag the translation axis or the corresponding plane border with the mouse to achieve the translation function of the object;

[0075] When it is necessary to rotate an object, first click to select the rotation axis, and while holding down the preset button (preferably the Shift key), drag the side perpendicular to the rotation axis in the plane where the rotation axis is located with the mouse to adjust the rotation angle. When adjusting, the closer to the center point, the greater the rotation amplitude; the farther from the center point, the smaller the rotation amplitude (which is convenient for doctors to perform precise control), so as to achieve the rotation function of the object.

[0076] In specific implementation, the operation interaction of the three-dimensional MPR geometric controller requires correctly and reasonably mapping the mouse operation to the change of the position and direction of the object to be operated in three dimensions to achieve a precise user operation experience. The mouse operations of the MPR geometric controller include the selection operation of the MPR plane and the rotation axis and the dragging and shifting operation of the mouse after selection. In the traditional MPR algorithm, it is necessary to simultaneously display the two-dimensional views of the three MPR planes, and by adjusting the control points of the coordinate axes in the two-dimensional views, the positions and directions of the other two orthogonally linked MPR planes are adjusted, as Figure 7 shown. Operating the MPR plane through the two-dimensional view will bring problems such as non-intuitive operation and poor user interaction experience. The present invention adopts a reference plane projection method to realize the specific operation of the mouse on the MPR plane.

[0077] The three-dimensional MPR geometric controller is determined by the DICOM bounding box B(x min ,x max ,y min ,y max ,z min ,z max ), the center point O of the controller and the normal vectors (n1, n2, n3) of the three orthogonal planes, as Figure 8 shown. Each MPR section is determined by the normal vector n, viewX, and viewUP, and the plane vertices (p0, p1, p2) are calculated according to the outer tangent points of the plane and the bounding box. The rotation axis and translation axis of the plane are determined by the two intersection points (t0, t1) of the center point in the direction of the normal vector and the bounding box B.

[0078] In the initial state, the normal vectors of the three planes are default set to the (x, y, z) coordinate axes of the bounding box. When a transformation is made to the MPR plane, the new outer tangent points of the plane and the bounding box are calculated according to the transformed center point O and the plane vectors n, viewX, and viewUP, and the plane vertices (p0, p1, p2) and intersection points (t0, t1) are updated;

[0079] Under the translation operation, the mouse selects the border of the MPR plane, and uses the normal vector of the plane passing through the center point as the translation axis. The selection of the MPR plane border is determined by detecting whether the projection of the world coordinate point clicked by the mouse along the camera direction intersects the border of the MPR plane. At the same time, the spatial relationship between the bounding box and the MPR plane border needs to be considered, and the intersection points and distances of the mouse world coordinate point along the camera direction on the six planes of the DICOM bounding box are calculated. When the border of the MPR plane intersects the projection of the mouse world coordinate point along the camera direction and the intersection distance is less than the intersection distances of all six planes of the bounding box, this plane is considered selected, and its intersection point is set as the initial coordinate point pw0 of the mouse operation.

[0080] Under the rotation operation, directly calculate whether the projection of the world coordinate point clicked by the mouse along the camera direction intersects the rotation axis vertices t0 and t1 and the intersection distance L. At the same time, calculate the intersection points and distances of the mouse world coordinate point along the camera direction on the six planes of the DICOM bounding box. When the intersection distance L is less than the intersection distances of all six planes of the bounding box, this rotation axis is considered selected.

[0081] Assume that the initial world coordinate point when the mouse is pressed is pw0, and the world coordinate point after movement is pw1. Regardless of the translation or rotation operation, it is first necessary to determine an appropriate reference plane, project the points pw0 and pw1 onto the reference plane according to the camera projection direction. In the translation operation, the translation vector is obtained according to the mutual relationship between the projection point and the translation axis. In the rotation operation, the rotation angle is obtained through the positional relationship between the projection point and the rotation axis.

[0082] In the translation operation, the reference projection plane is selected as one of the six faces of the DICOM bounding box. The specific method is as follows:

[0083] 1. First, obtain the direction of the camera dop and the initial world coordinate point when the mouse is pressed as pw0.

[0084] 2. Then, traverse the six faces and screen out the qualified plane as the target plane for transformation through a series of criteria.

[0085] 3. The screening criteria include:

[0086] 3.1 If the dot product of the face normal and the camera projection direction is less than the given threshold EPSILON, then this face is perpendicular or facing away from the camera direction, and this face is excluded.

[0087] 3.2 If the dot product of the face normal and the slice normal is close to 1, this face is excluded to ensure that the selected face is not the same plane as the slice.

[0088] 3.3 If the above screening conditions are passed, add the face index number and the dot product value of the face normal and the camera projection direction to the candidate list.

[0089] 4. If the length of the candidate list is greater than or equal to 2, sort it in descending order according to the inner product of the plane normal vector and the camera projection vector. Select the first face in the candidate list, which is also the face closest to the initial world coordinate point, as the reference plane.

[0090] In the translation operation, as Figure 9 shown, when the mouse moves, project the initial world coordinate point pw0 and the current world coordinate point pw1 onto the reference plane in the camera projection direction dop to obtain the projected points pw′0 and pw′1, and determine the magnitude of the translation vector through the inner product of the displacement vector pw′0 - pw′1 and the translation axis vector.

[0091] In the translation operation, move the center point O of the MPR geometric controller according to the translation vector, and update the plane vertices (p0, p1, p2) and the rotation axis vertices (t0, t1) of each MPR plane.

[0092] In the rotation operation, select the projection plane as a plane with the rotation axis as the normal vector, and the center point of the plane is the intersection point O of the perpendicular line from the world coordinate point pw0 of the mouse to the rotation axis t determined.

[0093] In the rotation operation, project the initial world coordinate point pw0 and the current world coordinate point pw1 onto the reference plane in the camera projection direction dop to obtain the projected points pw′0 and pw′1, and the rotation angle is obtained by the vectors u = pw1 - O t and v = pw0 - O t through the inverse cosine function:

[0094]

[0095] solved and obtained.

[0096] Given the rotation axis normal vector n = (a, b, c) and the rotation angle θ, the transformation of any spatial point in the rotation operation can be carried out through the following steps:

[0097] 1. Translation operation:

[0098] Assume the origin is (x0, y0, z0), then the formula for translating the point (x, y, z) to the new coordinates (x′, y′, z′) is:

[0099] x′ = x - x0

[0100] y′ = y - y0

[0101] z′ = z - z0

[0102] 2. Rotation operation:

[0103] The rotation matrix for rotating around the normal vector (a, b, c) by the angle θ can be expressed as:

[0104]

[0105] 3. Reverse translation operation

[0106] The formula for translating the coordinates (x′, y′, z′) back to their original position (x″, y″, z″) is as follows:

[0107] x″ = x′ + x0

[0108] y″ = y′ + y0

[0109] z″ = z′ + z0

[0110] In the rotation operation, after transforming the normal vectors of each plane of the MPR geometric controller according to the rotation operation, update the plane vertices (p0, p1, p2) and the rotation axis vertices (t0, t1) of each MPR plane.

[0111] More specifically, for the point selection measurement method of the controller, select points by adjusting the center point position of the three-dimensional MPR geometric controller, and directly mark the center point of the intersection of the three planes as the end point and make adjustments.

[0112] Furthermore, for the plane measurement method, insert an adjustment plane in any direction into the three-dimensional MPR geometric controller and mark it. Insert a parallel and closely attached adjustment plane at the first required position of the object (such as the lower endplate of the vertebral segment), mark it as the lower endplate, and insert a parallel and closely attached adjustment plane at the second required position of the object (the upper endplate of the next vertebral segment), mark it as the upper endplate. The average distance between the two adjustment planes is the distance between these two positions (i.e., the intervertebral space between these two vertebral segments). The measurement of angles such as the Cobb angle can also be completed using planes. After determining the target vertebral segment and setting the corresponding adjustment planes, the included angle between the projections of the two planes on the human coronal plane is the Cobb angle.

[0113] Preferably, all measurement data in this solution can be tracked in real time. The geometric controller of this solution allows the measurement points and measurement planes inserted to be linked and bound to the target object. If the measured object undergoes operations such as displacement and rotation, the bound measurement points and measurement planes will also perform linkage operations of displacement and rotation in the same way, and their corresponding measured values will change in real time.

[0114] This solution can not only be used for surgical planning of STL models, but also apply to volume reconstruction models based on volume rendering.

[0115] Regarding the attached drawings:

[0116] Figure 1Shows an adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction. By default, the controller uses the planar mode, which consists of three mutually perpendicular planes, control axes, and control points.

[0117] Figure 2 Shows the actual operation interface of the geometric controller, where the STL, volume rendering model, and the controller can be adjusted to complete relevant operations for surgical planning.

[0118] Figure 3 Shows the mesh mode of the geometric controller. The geometric controller mainly has three modes: mesh, planar, and transparent, and the opacity of each plane can be adjusted, allowing users to choose according to their own needs.

[0119] Figure 4 Shows the geometric controller in the magnified state. The line segment generated by the intersection of every two planes in the geometric controller is the adjustment axis. In the magnified state, precise control of the target object can also be achieved through the adjustment axis. In this solution, before moving or rotating an object, the geometric controller needs to be adjusted first to determine the translation direction or rotation axis of the object, and then subsequent operations can be carried out.

[0120] Figure 5 Shows the scenario of moving an object through the geometric controller, which specifically includes the following steps:

[0121] First, confirm the target movement direction of the object. Second, move or rotate the planes of the geometric controller so that one of the planes is parallel to the target direction. Subsequently, select the target object to be moved, hold down the "shift" key, and drag the plane perpendicular to the target direction or its adjustment axis in the geometric controller to move the target object along the target direction. After reaching the expected position, release the mouse, and the movement operation of the object is completed.

[0122] Figure 6 Shows the scenario of rotating an object through the geometric controller, which specifically includes the following steps:

[0123] First, confirm the target rotation direction and rotation axis of the object. Second, move or rotate the planes of the geometric controller so that one of the adjustment axes is in the same position and angle as the expected rotation axis, and at this time, this adjustment axis is the target rotation axis. Subsequently, select the target object to be rotated, then click on the target rotation axis, hold down the "shift" key, and drag the plane passing through the target rotation axis or its adjustment axis in the geometric controller to rotate the target object around the target rotation axis. After reaching the expected position, release the mouse, and the rotation operation of the object is completed.

[0124] Figure 7The MPR plane manipulation scheme of the standard MPR three-view is shown, and the position and direction of the other two MPR planes are adjusted by the control points of the coordinate axes in the 2D view. MPR plane manipulation through two-dimensional views will bring problems such as unintuitive operation and poor user interaction experience in 3D surgical planning.

[0125] Figure 8 The structure of the MPR plane in the MPR geometry controller is shown, including boundary points, coordinate vectors, and coordinate axis vertices. According to the transformed center point O and the plane vectors n, viewX, viewUP, the new tangent points between the plane and the bounding box are calculated, and the plane vertices (p0, p1, p2) and intersection points (t0, t1) are updated.

[0126] Figure 9 The MPR geometry controller uses a three-dimensional translation operation algorithm of the mouse with a dynamically selected projection plane. First, the mouse selects the translation axis, dynamically calculates and selects the reference projection plane of the mouse, calculates the displacement vector on the reference plane through the projection points on the reference plane before and after the mouse moves, and obtains the translation vector through the dot product with the translation axis vector, thereby updating the selected MPR plane to the new position.

[0127] For the present invention:

[0128] 1. In this solution, the translation direction or rotation axis of the object is first determined in advance by adjusting the MPR geometry controller. Since the translation and rotation operations of the object are based on the determined movement axis or rotation axis, the mouse interaction is converted into single-dimensional operations such as translation along the movement axis or rotation along the rotation axis, and there is no need to decompose the translation and rotation operations into the directional components of the rectangular coordinates. Therefore, the 3D operation based on the MPR geometry controller is concise, clear and efficient, and can be done in one step, avoiding the iterative adjustment process of the traditional geometry controller in various directions.

[0129] 2. In this solution, the rotation axis or translation direction of the object operation is obtained through the intersection line between the MPR planes. Since each MPR plane is a cross-section of the overall 3D space, the MPR plane provides an effective spatial position reference plane in three-dimensional space, which can be used intuitively to determine the position relationship between 3D space objects in cross-sections in different directions, such as distance, alignment, angle, etc. MPR contains planes in three orthogonal directions in 3D space, so this solution only needs a single view window to accurately and effectively provide position references in different directions. Similar solutions require three view windows to be opened at the same time to provide viewing angles in different directions. At the same time, due to the lack of plane reference, the positioning of objects can only be adjusted and operated iteratively by observing in different directions.

[0130] 3. The object in 3D medical surgical planning is an STL model based on the modeling of DICOM medical images. This solution uses MPR cross-sections generated from the original DICOM data, can perform 3D positioning according to the reference plane with important clinical significance, and calibrate and confirm the moving direction and rotation point of the object in 3D planning, realizing the operation reference and association between 2D and 3D in a multi-data and multi-modal manner.

[0131] In summary, this solution uses the MPR plane to provide 2D image reference and 3D spatial position reference with medical significance in 3D surgical planning, making up for the deficiency of the traditional geometric controller that lacks reference and can only perform spatial calibration by observation. At the same time, by using the reference provided by the adjusted MPR plane to pre-plan the translation direction or rotation angle and direction of the object before operation, it avoids the iterative process of the traditional geometric controller that can only disassemble the operation from the rectangular coordinate direction and repeatedly perform 3D spatial calibration because it cannot adjust the position and direction of its own coordinate axes. Therefore, the adjustable geometric controller based on MPR proposed in this solution can provide more accurate and efficient operation of the controlled object in 3D surgical planning, which is an important development and innovation of 3D geometric controller technology.

[0132] The present invention also discloses an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction.

[0133] The present invention also discloses a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction.

[0134] It is worth mentioning that the technical features such as the object involved in this patent application for invention should be regarded as the prior art. The specific structures, working principles, and possible control methods and spatial layout methods of these technical features can be selected conventionally in the art and should not be regarded as the invention points of this patent for invention. This patent for invention will not be further specifically elaborated.

[0135] For those skilled in the art, it is still possible to modify the technical solutions recorded in the foregoing embodiments or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction, characterized in that, The invention comprises a three-dimensional MPR geometry controller, wherein the three-dimensional MPR geometry controller comprises three mutually perpendicular planes, each plane comprises four edges and four endpoints of the circumscribed DICOM image, and the plane is displaced in the direction of the plane normal vector by dragging the edge of the plane, and the area size of the plane is changed by dragging the endpoints. After selecting the rotation axis, the edge of the plane where the rotation axis is located and perpendicular to the rotation axis is dragged, so as to adjust the normal vector direction of the plane and the orthogonal constraint plane of the plane in linkage, wherein: In the initial state, three orthogonal planes parallel to the coronal plane, sagittal plane, and axial plane are used as the initial adjustment planes. The intersection of the three planes is the center control point. The center control point is provided with a direction vector with an arrow parallel to the three planes. At the center control point, a straight line parallel to the normal vectors of each plane is drawn through the center point, which is the center adjustment axis. The center adjustment axis of the 3D MPR geometry controller is adjusted by the translation and rotation of the plane, and can be used as the rotation axis and translation direction axis of the 3D object operation. In terms of operational interaction, the 3D MPR geometric controller is used to implement operations including translation and rotation of objects. During measurement, point selection and plane measurement are realized through the 3D MPR geometry controller; For operational interactions: First, the position and direction of the controller's translation axis and rotation axis are adjusted by translating and rotating the MPR plane, and the translation direction and rotation axis direction of the controlled object are confirmed and calibrated by the 2D cross-section of the DICOM image using the MPR plane; When you need to translate an object, select the target object, then hold down the preset button and drag the translation axis or the corresponding plane border with the mouse to achieve the translation function of the object; When you need to rotate an object, first click to select the rotation axis, hold down the preset button and use the mouse to drag the edge of the plane where the rotation axis is located and perpendicular to the rotation axis to adjust the rotation angle. The closer the distance to the center point is, the greater the rotation amplitude is, and the farther the distance from the center point is, the smaller the rotation amplitude is, thus realizing the rotation function of the object; In the translation operation, the mouse selects the border of the MPR plane, and the normal vector of the plane through the center point is used as the translation axis. The selection of the MPR plane border is determined by detecting whether the projection of the world coordinate point clicked by the mouse along the camera direction intersects with the border of the MPR plane. At the same time, the spatial relationship between the bounding box and the MPR plane border needs to be considered, and the intersection and distance of the mouse world coordinate point along the camera direction in the six planes of the DICOM bounding box are calculated. When the border of the MPR plane intersects with the projection of the mouse world coordinate point along the camera direction and the intersection distance is less than the intersection distance of all the six planes of the bounding box, the plane is considered to be selected, and its intersection point is set as the initial coordinate point pw0 of the mouse operation; In the rotation operation, directly calculate whether the projection of the world coordinate point clicked by the mouse along the camera direction intersects with the rotation axis vertices t0 and t1 and the intersection distance L. At the same time, calculate the intersection and distance of the mouse world coordinate point along the camera direction in the six planes of the DICOM bounding box. When the intersection distance L is less than the intersection distance of all the six planes of the bounding box, the rotation axis is considered to be selected; Assume that the initial world coordinate point when the mouse is pressed is pw0, and the world coordinate point after movement is pw1. Whether it is a translation or rotation operation, first, an appropriate reference plane needs to be determined. Points pw0 and pw1 are projected onto the reference plane according to the camera projection direction. In the translation operation, the translation vector is obtained based on the mutual relationship between the projection points and the translation axis. In the rotation operation, the rotation angle is obtained through the positional relationship between the projection points and the rotation axis.

2. The adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction according to claim 1, wherein The reference plane projection method is adopted to implement the specific operation of the mouse on the MPR plane, where: The three-dimensional MPR geometric controller is determined by the DICOM bounding box B(x min , x max , y min , y max , z min , z max ), the center point O of the controller, and the normal vectors (n1, n2, n3) of the three orthogonal planes. Each MPR section is determined by the normal vector n, viewX, and viewUP, and the plane vertices (p0, p1, p2) are calculated based on the outer tangent points of the plane and the bounding box. The rotation axis and translation axis of the plane are determined by the two intersection points (t0, t1) of the center point in the direction of the normal vector and the bounding box B; In the initial state, the normal vectors of the three planes are default set to the (x, y, z) coordinate axes of the bounding box. When a transformation occurs to the MPR plane, according to the transformed center point O and the plane vectors n, viewX, viewUP, the new outer tangent points of the plane and the bounding box are calculated, and the plane vertices (p0, p1, p2) and intersection points (t0, t1) are updated.

3. The adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction according to claim 2, wherein: In the translation operation, when the mouse moves, the initial world coordinate point pw0 and the current world coordinate point pw1 are projected onto the reference plane according to the camera projection direction dop, obtaining the projection points pw′0 and pw′1. The magnitude of the translation vector is determined by the inner product of the displacement vector pw′0 - pw′1 and the translation axis vector; In the translation operation, the center point O of the MPR geometric controller is moved according to the translation vector, and the plane vertices (p0, p1, p2) and the rotation axis vertices (t0, t1) of each MPR plane are updated; During the rotation operation, the projection plane is selected as a plane with the rotation axis as the normal vector, and the center point of the plane is determined by the intersection point O of the perpendicular line from the world coordinate point pw0 of the mouse to the rotation axis. t Determined; During the rotation operation, the initial world coordinate point pw0 and the current world coordinate point pw1 are projected onto the reference plane in the camera projection direction dop, obtaining the projection points pw′0 and pw′1, and the rotation angle is determined by the vectors u = pw1 - O t and v = pw0 - O t through the arccosine function: Solve and obtain; Given the rotation axis normal vector n = (a, b, c) and the rotation angle θ, the transformation of any spatial point in the rotation operation can be carried out through the following steps: Translation operation: Assume the origin is (x0, y0, z0), then the formula for translating the point (x, y, z) to the new coordinates (x′, y′, z′) is: x′ = x - x0 y′ = y - y0 z′ = z - z0; Rotation operation: The rotation matrix for rotating by an angle θ around the normal vector (a, b, c) can be expressed as: Reverse translation operation: The formula for translating the coordinates (x′, y′, z′) back to the original position (x″, y″, z″) is: x″ = x′ + x0 y″ = y′ + y0 z″ = z′ + z0; In the rotation operation, after the normal vectors of each plane of the MPR geometric controller are transformed according to the rotation operation, the plane vertices (p0, p1, p2) and the rotation axis vertices (t0, t1) of each MPR plane are updated.

4. The adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction according to claim 3, wherein For the point selection measurement method of the controller, points are selected through the center point position of the three-dimensional MPR geometric controller, and the center point of the intersection of the three planes is directly marked as an endpoint and adjusted.

5. The adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction according to claim 4, wherein, For the plane measurement method, an adjustment plane in any direction is inserted into the three-dimensional MPR geometric controller and marked.

6. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction according to any one of claims 1 to 5.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When this computer program is executed by the processor, it implements the adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction according to any one of claims 1 to 5.

Images